Inceptions and Inspirations

Gurus and Readings

Gurus

• Rod Johnson
  • Spring Framework
• Doug Lea
  • Concurrent Programming in Java(1997), java.util.concurrent package
• Joshua Bloch
  • Effective Java(2001)
• Douglas Crockford
  • JSON, JSLint, JavaScript: The Good Parts(O'Reilly. 2008)

Books

• XSLT Cookbook: Solutions and Examples for XML and XSLT Developers, 2nd Edition, Sal Mangano, 2005
• JavaScript: The Good Parts, Douglas Crockford, 2008

Articles

• The Ghost Who Codes: How Anonymity is Killing Your Programming Career(02.13.2013)

Java

Basic

• Constructor

  • The first statement of a constructor body may be an explicit invocation of another constructor of the same class or of the direct superclass.

  • If a constructor body does not begin with an explicit constructor invocation and the constructor being declared is not part of the primordial class Object, then the constructor body implicitly begins with a superclass constructor invocation "super();", an invocation of the constructor of its direct superclass that takes no arguments.

  • source : 8.8.7. Constructor Body of The Java Language Specification of Java SE 8 Edition

Guidelines for Designing API

• Minimize public members.

• Prefer stateless over stateful

• Provide factory, builder, facade or something that acts as an entrance point of the library.

• Utilize JSR 305 annotations to clearly specify nullness and concurrency constraints.

• Apply consistent pattern on the choice of primitive or wrapper for the return type and parameter type.

• Apply consistent pattern on the choice of null or empty collection/array.

• Apply consistent way to manage configuration.
  • Define and describe the meaning of each configuration item.

• Consider using with Spring framework.

• Avoid get or set for the methods that's NOT trivial.

• Don't miss the Javadoc for non-trivial public methods.
  • Explain boundary conditions, contraints, or exceptional behavior as possible.

• Use English in the source as possible.

Using Pattern

• Design Patterns
• GRASP (General Responsibility Assignment Software Patterns)
• SOLID (Single responsibility, Open-closed, Liskov substitution, Interface segregation and Dependency inversion)
• DRY (Don't Repeat Yourself)
• Divide and Conquer

• Design Patterns: Elements of Reusable Object-Oriented Software (by Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides)
• Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and Iterative Development (by Craig Larman)

Core Patterns

• High Cohesion
• Low Coupling
• Don't Talk to Strangers

• Singleton pattern
• Factory pattern
• Builder pattern

• Adapter pattern
• Proxy pattern
• Facade pattern

• Observer pattern (Listener, Reacter, Callback)
• Command pattern
• Strategy pattern

• Open-closed pattern

Builder Pattern

• Builder pattern(Wikipedia)
• The freeze thaw pattern

• To make a class thread-safe escaping the state change during initializing, you can use *Builder pattern*. By delegating the thread unsafety to the builder object, the product object created by the builder can be thread safe.

Examples

• jmxtrans
  • com.googlecode.jmxtrans.model.Query
  • com.googlecode.jmxtrans.model.Query.Builder
  • com.googlecode.jmxtrans.model.Server
  • com.googlecode.jmxtrans.model.Server.Builder

Concurrency

• Immutables are thread-safe and shareable
• Some mutables can be thread-safe and shareable
  • When the mutables contains only global or shared variables
• Partially thread-safe
  • Thread safe after initial configuration
    • com.fasterxml.jackson.databind.ObjectMapper
    • Freeze/Thaw pattern

Formatting, conversion and validating of string, number or date type data

Available packages

• Apache Commons Lang 3
  • Homepage : http://commons.apache.org/lang/
  • API : http://commons.apache.org/proper/commons-lang/javadocs/api-3.3.2/
  • Lang provides a host of helper utilities for the java.lang API, notably String manipulation methods, basic numerical methods, object reflection, concurrency, creation and serialization and System properties. Additionally it contains basic enhancements to java.util.Date and a series of utilities dedicated to help with building methods, such as hashCode, toString and equals.

• Apache Commons Validator
  • Homepage : http://commons.apache.org/validator/
  • API : http://commons.apache.org/proper/commons-validator/javadocs/api-1.4.0/
  • Framework to define validators and validation rules in an xml file and include some much reusable method to validate and convert strings in terms of number, integer, date, time, IP address, e-mail address, credit card number and so on.

Available utilities by case

• String manipulation
  • org.apache.commons.lang3.StringUtils
    • general util. to investigate, join, split, substring or trim string
    • static methods, NullPointerException free
    • API
  • org.apache.commons.lang3.StringEscapeUtils
    • escape or unescape string in terms of HTML, XML, Java, JavaScript or CVS
    • static methods, NullPointerException free and thread-safe
    • API
  • org.apache.commons.lang3.text.WordUtils
    • capitalize, uncapitalize or wrap the string
    • static methods, NullPointerException free
    • API

• Character manipulation
  • org.apache.commons.lang3.CharUtils
    • static methods, NullPointerException free
    • API

• Date manipulation and calculation
  • org.apache.commons.lang3.DateUtils
    • java.util.Date type based basic calculations (add/subtract years/months/weeks/days/hours/minutes/seconds)
    • static method, NullPointerException free
    • API

• Date formatting
  • org.apache.commons.lang3.DateFormatUtils
    • one line formatting
    • static methods, NullPointerException free and thread-safe
    • API

• String -> Number conversion
  • org.apache.commons.lang3.math.NumberUtils
    • static methods, NullPointerException free, default definable style (like C)
    • API

• String -> Date validation and conversion
  • org.apache.commons.validator.routines.DateValidator
    • validate and parse string in terms of date
    • API
  • org.apache.commons.validator.routines.TimeValidator
    • validate and parse string in terms of time
    • API

• String -> Number validation and conversion
  • org.apache.commons.validator.routines.DoubleValidator
    • validate and parse string in terms of double-precision floating-point number(double).
    • API
  • org.apache.commons.validator.routines.FloatValidator
    • validate and parse string in terms of single-precision floating-point number(float).
    • API
  • org.apache.commons.validator.routines.LongValidator
    • validate and parse string in terms of integer in long type.
    • API
  • org.apache.commons.validator.routines.IntegerValidator
    • validate and parse string in terms of integer in int type.
    • API

General data container model

For meta modeling or untyped data transfer object(DTO), general data container model is necessary. A few real world samples are :

• Open MBean descriptor classes.
  • javax.management.openmbean package
  • type for general data : CompositeData, TabularData
  • type for general meta : SimpleType, ArrayType, CompositeType, TabularType

JavaScript

Coding Guideline

• Define all variables at the top of the function.

• Use {} instead of new Object()
• Use [] instead of new Array()

• Avoid with clause.
• Avoid eval function.
• Avoid for-in clause, specially with array.
  • for-in loop over all the present keys in the object and its prototype chain.

• Be careful with this.

• Distinguish function declaration and function expression
  • Function declaration : function foo(){}
  • Function expression : var foo = function(){}

• Avoid wrapper objects for primitive types.

• Distinguish RegExp.test(), String.search() and String.match(), specially in performance.

C/C++

Visual C++

Build guideline

• You should use /RTC for development builds; /RTC should not be used for a retail build. /RTC cannot be used with compiler optimizations (/O Options (Optimize Code)). (Compiler Options > /RTC (Run-Time Error Checks)

Modules guideline

• Directories where fundamental Windows modules live

On Hadoop

Interesting point

• Full landscape of Hadoop echo-system and the detailed features of each component in the echo-system

• Cloudera
  • the features and architectures of service and configuration management tool.

Leading products analysis

Cloudera Manager

• Features
  • Statement management
    • manage model state(shape) and runtime state
  • Configuration management
  • Process management
  • Software distribution management
  • Management services
    • Metric Collection and Display
    • Health Tests
    • Events and Alerts
• Interfaces
  • Web based admin tool
  • Java API

Related topics

• Process supervision software and tools

Requisite

• Full list of command line or API in Hadoop
  • especially necessary or deeply related to admin or control the Hadoop system.

• Running shell script from Java
  • ProcessBuilder in JDK
  • more elegant libraries or wrapper in open-source world ?

• Distributed agent framework or library in Java

• Data visualization UI framework or library
  • Especially JavaScript
  • other than D3.js

• Monitoring and control Zookeeper from the application

On OpenShift

Background

Pain-point

Expectation

Workflow

Concepts

• Applications, Cartridges, Gears, Nodes
  • What are the cardinalies among concepts
    • application : cartridge = ?
    • cartridge : gear = ?
    • gear : node = ?
  • "By default, for scalable applications, each cartridge resides on its own gears within its own group instance. However, sometimes it is required/preferred to have two cartridges be located together on the same set of gears." -> cartridge (instance) : gear = n : 1
  • Considering domain scope cartridge, application : cartridge = n : m
  • So, an application can be installed on top of multiple gears, and a gear can be involved with multiple applications.

• Scope of cartridge
  • domain scope cartridge seems to be suitable for shared services such as database and middle-wares.

Commands

Questions

• Generally sources on a system consist of several source projects. Then how to map source projects on to applications of OpenShift ?
  • Should consider the source configuration management view-points.

• Can I use Subversion instead of built-in Git with OpenShift and how can I do that?

• Can I share one database instance among multiple applications in OpenShift and how can I do that?
  • Generally environment of software development process consist of
    • multiple source projects
    • single database
    • single application server
    • single configuration repository

• Can I use my own value for $OPENSHIFT_GEAR_UUID ?

• Is it possible to share Maven local repository among gears ?

Necessary Cartridges

• Oracle database cartridge
  • domain scope

• Domain scope MySQL database cartridge
• Domain scope PostgreSQL database cartridge

• Jetty cartridge
• Tomcat cartridge

• Client env. cartridge
  • contains install script of Eclipse and necessary plug-ins.

• Source projects cartridge
  • basically multiple source project.
  • should be able to customized on the number and names of source projects at cartridge install time.

Front-end

On Hyperledger Fabric

• Fabric CA
  • 기본 설치 및 설정
  • TCert, ECert 발급
  • 인증서 갱신/재발급
  • CA Server Clustering
  • LDAP 연계

• Fabric Network/Channel
  • Fabric Peer 기본 설치/설정/구동
  • Fabric Orderer 기본 설치/설정/구동
  • Channel 설계/적용
  • Endorsement Policy 설계/적용
  • CouchDB 적용/최적화
  • Kafka 적용/최적화
  • Fabric Orderer Clustering
  • TLS 적용
  • Docker Swarm 구성
  • Kubernete 구성
  • ELK Stack 기반 Log 분석
  • Gossip Protocol 분석
  • gPRC 분석
  • Transaction Tracing

• 성능/가용성 테스트
  • JMeter 기본 테스트 방안
  • JMeter 분산 테스트 방안
  • 시스템 자원사용율 모니터링 - CPU
  • 시스템 자원사용율 모니터링 - Memory
  • 시스템 자원사용율 모니터링 - Network I/O
  • 시스템 자원사용율 모니터링 - Disk I/O
  • 서버 자원사용율 모니터링 - Fabric Peer
  • 서버 자원사용율 모니터링 - Fabric Orderer
  • 서버 자원사용율 모니터링 - Kafka
  • 서버 자원사용율 모니터링 - CouchDB
  • Go Profiling 적용
  • Go Debugging 적용
  • 표준 성능테스트 케이스 정의
  • 표준 가용성테스트 케이스 정의

• 20 Node Test Network
  • 기본 구성 (LevelDB, Solo Orderer, No TLS)
  • 기본 구성 성능테스트
  • CouchDB 적용 성능테스트
  • Production 구성 (CouchDB, Clustered Orderer, TLS)
  • Production 구성 성능 테스트
  • Production 구성 가용성 테스트

• 100 Node Test Network
  • 시스템 구성 설계
  • 장비 구성
  • 구축 (CouchDB, Clustered Orderer, TLS)
  • 성능 테스트
  • 가용성 테스트

Provisioning scripts design

• Use JSON format metadata.
• Use GNU parallel.
• Use headquarter-station pattern.
• Minimize file transfer.
• Confirm pre-conditions and post-conditions.
• Generate final report including architecture and useful endpoints of the network or cluster.

Architecture decision

• What is the optimal ratio between Orders, Leader Peers and Nonleader Peers ?
• What is the optimal values of block creation criteria ?
• How to optimize the connection or link between Peer container and CouchDB container on the same host machine ?
• What is the optimal values of gossip communications ?
• How to optimize CouchDB ?
• How to optimize Kafka ?
• How to measure and optimize the network performance between machines ?
• How to measure and optimize the disk I/O performance of machines ?

On OpenCV

IplImage Anatomy

• Definition

• Member priority
  • 1st : width, height,
  • 2nd : depth, nChannels
  • 3rd : origin, dataOrder

• Constants for depth

• Constants for origin

References

• ipl.h
• OpenCV API Reference > IplImage

On Red5

• Real Time Messaging Protocol
  • Packet structure
  • The protocol
• RTMP Specification

RTMPClient

On VirtualBox

Tips for CentOS on VirtualBox

• Absolutely install the guest addition.
  • CentOS as a Guest OS in VirtualBox
• Use F10 for the Host Key.
  • [VirtualBox Manager > File > Preference > Input > Virtual Machine > Host Key Combination]
• Make Mini ToolBars visible as possible
  • [Virtual Machine > Machine > Settings > General > Advanced > Mini ToolBars]
    • check "Show in Fullscreen/Seamless"
    • check "Show at Top of Screen"
  • Use "Seamless Mode" as possible
  • [Virtual Machine > View]

On Docker

Key Concerns

• How to manage configuration per installation/instance, operation ?
• How to manage logs ?
• How to manage application deployments ?
• How to manage persistent data ?
• How to debug or monitor application in container ?
• How to provide or access development tools ?
• How to make containers collaborate or bind containers ?

Strategys and Tactics

• Include development tools and environment into the docker image for developer local.
• Make working copy of source code in local stored externalized storage or shared folder.
• Externalize configuration. (using what ?)
• Separate docker image from application source.
  • Make docker image which can clone, build and deploy an application in some predefined format from source repository.
• Make and enhance own management service for pods, containers and so on.

On Redmine

Customization

Target files

• page template : $REDMINE_HOME/app/views/layouts/base.html.erb
• initial page : $REDMINE_HOME/app/views/welcome/index.html.erb
• i18n message bundle : $REDMINE_HOME/config/locales/*.yml
• default style : $REDMINE_HOME/public/stylesheets/application.css

Custom style

On TestLink

Installation

Overview

• TestLink 1.9 needs PHP 5.2 or 5.3, MySQL database 4.1.x or higher, and Apache web server 2.x.
• All of these can be prepared in oneshot using EasyPHP.

Procedure

1. Download "EasyPHP 12.0 with PHP 5.2.17" from www.easyphp.org/download.php
   • Other than version 1.9 of TestLink, refer the installation guide or something like that to confirm the exact dependency of TestLink
2. Run the setup executable
   • EasyPHP installation file is in forms of exe file like EasyPHP-12.0_with_PHP-5.2.17-setup.exe
3. Follow the instructions and specify the installation directory.
   • All the files for PHP, Apache httpd server, MySQL database and those related files will be located under the specified directory.
   • For windows, c:\servers\EasyPHP-12.0 would be proper and avoid to include white-space in the path like "c:\Program Files\EasyPHP-12.0"
4. Check the installation directory
   • After the installation completed, you can see the following files or directories under the directory you specified during installation. Exact list could be different depending on the version and edition of EasyPHP

c:\servers\EasyPHP-12.0\ ----+--- apache\
                             |--- conf_files\
                             |--- home\
                             |--- modules\
                             |--- mysql\
                             |--- php\
                             |--- tmp\
                             |--- www\
                             |--- xdebug\
                             |--- EasyPHP-12.0.exe
                             |--- easyphp.ini
                             |--- EasyPHP.log
                             |--- gpl.txt
                             |--- langues.txt
                             |--- readme.txt
                             |--- unins000.dat
                             +--- unins000.exe

1. Run the EasyPHP and check around
   • To run EasyPHP, execute the short cut of [Start > Programs > EasyPHP 12.0 > EasyPHP 12.0]
   • Easy PHP will run Apache web server, MySQL database and make an icon appear on the right/bottom side of the Windows task bar nearby clock. The icon is like black lower-case alphabet "e", When you click the right mouse button on the icon, you can see meues like 'Help', 'Log Files', 'Configuration', 'Explorer', 'Administration', and so on. For more on these menus, refer http://www.easyphp.org/introduction.php

On Solidity

Compiler

The return of solc.compile function

After the following code executed

The structure of output

Smart Contract

• Account White List
• Whitelisted Token
• Distingushiable Account
• Distingushiable Polling

On System Management

Production Systems Management Considerations

• Don't use SSH users same with those on development or test environment.
• Don't use SSH key/cert same with those on development or test environment.
• Don't use sudoers

• Don't use default port for system softwares or middle-wares.
• Don't use default account for system softwares or middle-wares.
• Close unnecessary ports.

Business Concepts

misc

File abstraction

• path, directory, name, extension
  • path = directory + file-separator + name + '.' + extension

• full-name = path
• short-name = name + '.' + extension

• Samples
  • c:\windows\system32\kernel32.dll
    • path : c:\windows\system32\kernel32.dll
    • directory : c:\windows\system32
    • name : kernel32
    • extension : dll

• References
  • Filename (Wikipedia)
  • java.nio.file.Path API
  • java.io.File API

Message scheme

General message header

Message mapping

• Mapping rules
  • unit rules
    • concatenation
      • concatenation of <emp>n</emp> items of source
    • split
      • split by index
      • split by delimiter
    • replacement
    • formatting
      • pre defined formatting
      • user defined formatting using formatt pattern
    • format conversion
      • pre defined conversion : eg) 2013/01/01 -> 2013-01-01
      • user defined - is it possible ?
      • two step : unformat and apply new format
    • arithmetic calculations
      • addition of two items
      • subtraction of two items
      • multiplication of item and constant number
      • division of item
    • logical calculations
    • code to code conversion
    • to boolean conversion : different from logical calculation ?
    • regex conversion : ?
  • combined rules
  • free-form using expression directly
    • how to validate

• Mapping rule unit test case

Things you MUST know about DEBUGGING

• Eclipse debugger
  • How to setup debugging.
    • on server-side
    • on Eclipse
  • How to start debugging.
    • when launched by Eclipse
    • when externally launched.
  • How to make or delete break point.
  • How to view static fields in the Variables view.
  • How to view inherited fields in the Variable view.
  • How to view only logical structure of collection or map objects.
  • How to use Display view.
  • What is stack trace and how to trace it.
  • How to move forward.
    • resume, step into, step over, step return
  • How to list breakpoints while debugging
  • How to config debugging step filter.
  • How to config debugging stack filter. Is it possible?
  • How to change variable value and execute forward while debugging.
  • How to add or remove breakpoints on-the-fly (while debugging).

General strategy and tactics on development environment

Source projects

• Concepts
  • portfolio = product+
  • product = module+
    • module ~ package
  • module = class+, resource+

• Tactics
  • Construct source project per module not product.
  • Construct source repository per product.
    • Do not ...
    • Dependencies among products should be defined on binary repository, not source repository of dependent products.
  • The structure of trunk and tag or release directory in source repository.
  • Eclipse launcher and configuration
    • eclipse.bat and eclipse.ini under the trunk or the tag/0.0.0 directory.
    • JAVA_6_HOME, ECLIPSE_422_HOME
  • Readme files
    • under the trunk or the tag/0.0.0 directory for Readme on scale of product.
    • under the per module Eclipse project directory for Readme on scale of module.
    • contents
      • requirement on environment. specially necessary environment variables and their meaning.

Configuration management

• Naming environment variables, configuration property or variable placeholder
  • an item (environment variable or configuration property) whose value is a directory should have a name that ends in _DIR or .dir and the value should have a trailing slash.
  • an item whose data-type is boolean should have a name that includes ....
    • example ; ALLOWS_DELETE

Software Distributable Packaging

Concepts and terms

• source, binary, resource
• library, runtime
• file types
  • C/C++ : *.lib, *.dll, *.pdb, *.h
  • Java : *.jar

Directory layout of deployed

• JBoss AS 4.x directory structure
  • bin, client, docs, lib, conf, data, log, tmp, work

UI(User Interface) design

• Delete or Reset action should be reconfirmed by the user.
• Internatiolization(i18n) or Localization(l10n) should be applied intrinsically for buttons, labels and titles.

Software development guideline

• Guides source project layout
• Guides selection of programming languages, software frameworks, and development tools
• Guides coding convention
• Guides design patterns and common libraries
• Guides software build and packaging
• Guides unit testing
• Guides debugging
• Guides static code analysis
• Guides technical documentation
• Provides boilerplate or scaffold artifacts