Thirdstage's Wiki
39 Pages
Add new page

Computer Programming

View source
Comments Share

Contents

Programming Fundamental

Category Description Examples
Static Type Checking the process of verifying the type safety of a program based on analysis of a program's text (source code). C/C++, Java, Rust
Dynamic Type Checking the process of verifying the type safety of a program at runtime. JavaScript, Python, Ruby
Type Paradigm Description Remarks
Imperative Imperative programming uses statements that change a program's state
Procedural Programming program is built from one or more procedures (also termed subroutines or functions) Fortran, Pascal, C
Object-Oriented Programming Java, C++, C#, Scala, Smalltalk
Declarative Declarative Programming expresses the logic of a computation without describing its control flow - logic without control flow SQL, XQuery, Regex
Functional Programming treats computation as the evaluation of mathematical functions and avoids changing-state and mutable data Clojure, Erlang, Haskell
Reactive programming concerned with data streams and the propagation of change
  • Functional Programming Concepts
    • Higher-order function : (mathematical) functions that can either take other functions as arguments or return them as results
    • First-class function : (computer science) treats functions as first-class citizens
    • First-class citizen : an entity which supports all the operations including being passed as an argument, returned from a function, and assigned to a variable.
    • Lambda : a function definition that is not bound to an identifier (lambda abstraction, lambda expression, function literal, anonymous function)
    • Closure : a record storing a function together with an environment
  • Predicate : commonly understood to be a Boolean-valued function
  • Parameter
    • A parameter or a formal argument, is a special kind of variable, used in a subroutine to refer to one of the pieces of data provided as input to the subroutine.
    • The argument in computer science is thus the actual input expression passed/supplied to a function, procedure, or routine in the invocation/call statement.
    • The term parameter (sometimes called formal parameter) is often used to refer to the variable as found in the function definition, while argument (sometimes called actual parameter) refers to the actual input supplied at function call.
    • Parameters appear in procedure definitions; arguments appear in procedure calls. Loosely, a parameter is a type, and an argument is an instance.
    • An output parameter, also known as an out parameter or return parameter, is a parameter used for output, rather than the more usual use for input. Output parameters is an idiom in some languages, notably C and C++.
  • Variable Shadowing

Data Structure and Algorithm

Higher-order Function

Function Description Sample Remark
Filter processes a data structure (usually a list) in some order to produce a new data structure containing exactly those elements of the original data structure for which a given predicate returns the boolean value true. stream.filter(pred),
filter(func, list)
Map applies a given function to each element of a functor, e.g. a list, returning a list of results in the same order. stream.map(pred),
map(func, list)
Fold analyze a recursive data structure and through use of a given combining operation, recombine the results of recursively processing its constituent parts, building up a return value stream.reduce(initval, func),
reduce(func, list, initval) 		Reduce, Aggregate, Inject

Tree

Metasyntax

  • ASN.1 (Abstract Syntax Notation One)
    • a standard and notation that describes rules and structures for representing, encoding, transmitting, and decoding data in telecommunications and computer networking

Extended BNF

Usage Notation
Definition =
Concatenation ,
Termination ;
Alternation |
Optional [ ... ]
Repetition { ... }
Grouping ( ... )
Comment (* ... *)
Special Sequence ? ... ?
Exception -

Examples

(* syntax for event signature of Solidity *)
(* https://solidity.readthedocs.io/en/v0.5.15/abi-spec.html#events *)
event-signature = event-name, "(", [ parameter-type, { ",", parameter-type } ], ")"

Pattern matching

Coding Convention

Style Description Sample Remarks
Camel Case Spaces and punctuation are removed and the first letter of each word is capitalised. accessToke, AccessToken Pascal Case
Snake Case Punctuation is removed and spaces are replaced by single underscores. access_token, ACCESS_TOKEN
Kebab Case Punctuation is removed and spaces are replaced by single hyphens. access-token Spinal Case, Lisp Case, Dash Case
Language Guide Remarks
Java Code Conventions for the Java Programming Language
Code Conventions for the JavaServer Pages Technology
Naming Conventions for Enterprise Java Applications
Scala Scala Style Guide
Python PEP 8 -- Style Guide for Python Code
Google Python Style Guide
Rust Rust Coding Conventions
Rust Naming Conventions
Rust API Guidelines / Naming RFC 430
JavaScript Google JavaScript Style Guide
Firefox JavaScript Coding style
Airbnb JavaScript Style Guide
Code Conventions for the JavaScript Programming Language by Douglas Crockford
JavaScript Style Guide from jQuery's Style Guides
WordPress JavaScript Coding Standards
Airbnb React/JSX Style Guide
C/C++ ide.googlecode.com/svn/trunk/cppguide.html Google C++ Style Guide
C++11 use in Chromium
OpenStack C++ Coding Standards
FFmpeg Coding Rules
HTML/CSS Google HTML/CSS Style Guide
HTML Style Guide
WordPress HTML Coding Standards
WordPress CSS Coding Standards

Object Oriented Programming

Concept Description Supported by Remarks
interface definitions of methods and values which the objects agree upon, in order to co-operate, as part of an API. define one or more behaviors via method signatures
trait both provide a set of methods that implement behaviour to a class, and require that the class implement a set of methods that parameterize the provided behaviour. Rust, Scala defines behaviors via full method definitions but not states
mix-in a class that contains methods for use by other classes without having to be the parent class of those other classes. described as being "included" rather than "inherited".

Glossary

Programming Language

Language Appeared in Style Typing Systme Remarks
C 1972 Imperative, Procedural Static, Weak
C++ 1985 Imperative, Object-oriented, Functional, Procedural Static
Python 1990 Imperative, Object-oriented, Functional, Procedural, Reflective Dynamic
Java 1995 Object-oriented, Imperative, Reflective Static, Strong
JavaScript 1995 Functional, Imperative Dynamic
Ruby 1995 Functional, Imperative, Object-oriented, Reflective Dynamic, Strong
Go 2009 Imperative, Functional, Object-oriented Static, Strong, Inferred
Rust 2010 Functional, Imperative Static, Strong, Inferred
Kotlin 2011 Object-oriented, Functional, Imperative Static, Strong, Inferred
  • Language Server Protocol (LSP)
    • A Language Server is meant to provide the language-specific smarts and communicate with development tools over a protocol that enables inter-process communication.
Feature Java Python Rust
Type Checking Static Type Checking Dynamic Type Checking Static Type Checking
Primitive Types boolean,
char,
short, int, long,
float,
double 		int,
float,
complex,
str 		bool, char,
i8, i16, i32, i64, isize,
u8, u16, u32, u64, usize,
f32, f64
Variable Default Mutability mutable mutable immutable
Constant static final MAX const MAX
Integer Literals 31, -31,
10_000_000_000,
0x33FE (hexadecimal),
0b101010 (binary)
Floating Point Literals 0.5, -0.5,
10f, 89d,
3.1415e0, 1.0e-6d
Character Literals
String Literals
Formatted String Literals N/A f'Hi, {name}'
Arithmetic Operators + (addition)
- (substraction)
* (multiplication)
/ (division)
% (modulus) 		+ (addition)
- (substraction)
* (multiplication)
/ (division)
% (modulus)
** (exponent)
// (floor division) 		+ (addition)
- (substraction)
* (multiplication)
/ (division)
% (modulus)
Regex Support
Container Types Array, Set, List, Map set, list, tuple, dict
Array Literals
List Literals ['dot', 'cat', 'snake']
Map Literals N/A {'dog': 'bark', 'cat': 'meow', 'snake': 'hiss'}
Set Literals N/A {'dog', 'cat', 'snake'}
Multiple Return No Yes Yees
Destructuring Assignment No Yes Yes
Lamda Expression (a, b) -> (a*a + b*b) lamda a, b: a*a + b*b |x| x * 2

Java

Python

JavaScript

Scala

Rust

References

Primitive Types
Category Type Definition Size Literals Remarks
Boolean bool 1 Byte true, false
Character char unicode scalar value 4 Bytes 'a', 'Z'
Integer i8 8-bit signed integer type 1 Byte [-128, 127]
i16 16-bit signed integer type 2 Bytes
i32 32-bit signed integer type 4 Bytes
i64 64-bit signed integer type 8 Bytes
isize
u8 8-bit unsigned integer type 1 Byte [0, 255]
u16 16-bit unsigned integer type 2 Bytes
u32 32-bit unsigned integer type 4 Bytes
u64 64-bit unsigned integer type 8 Bytes
usize
Floating Point f32 32-bit floating point type 4 Bytes
f64 64-bit floating point type 8 Bytes
API Documentation
Crate Description Remarks
std the foundation of portable Rust software, a set of minimal and battle-tested shared abstractions for the broader Rust ecosystem. core types, primitives, standard macros, I/O, multithreading
test support code for rustc's built in unit-test and micro-benchmarking framework.
rand utilities for random number generation.
Cargo CLI
Command Description Remarks
cargo build Compile local packages and all of their dependencies.
cargo update Update dependencies as recorded in the local lock file(Cargo.lock).

Go

References

command description remarks
go build compile packages and dependencies
go install compile and install packages and dependencies
go get download and install packages and dependencies
Category Package Description Remarks
standard fmt implements formatted I/O with functions analogous to C's printf and scanf.
standard http provides HTTP client and server implementations.
standard path/filepath implements utility routines for manipulating filename paths in a way compatible with the target operating system-defined file paths.
go-resiliency retrier implements the "retriable" resiliency pattern for Go

Readings

Fundamental
OOP
Documenting
Profiling
Networking

Frameworks

Libraries

Gorilla WebSocket
Viper
Cobra
  • https://github.com/spf13/cobra
  • Desc. : both a library for creating powerful modern CLI applications as well as a program to generate applications and command files.
  • License : Apache License Version 2.0
Golang logging library

Tools

Gocode
Delve
Commands
command description remarks
dlv attach Attach to running process and begin debugging
dlv exec Execute a precompiled binary, and begin a debug session
Readings
pprof

C, C++

C#

Ruby

Readings

Metaprogramming
Design Pattern
misc
  • SSL CERTIFICATE UPDATES
    • to solve errors with SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed

Libraries

Rouge

TypeScript

CoffeeScript

ActionScript

Erlang

  • http://www.erlang.org/
  • Desc. : a programming language used to build massively scalable soft real-time systems with requirements on high availability.

Haskell

Lua

  • http://www.lua.org
  • Desc. : a powerful, efficient, lightweight, embeddable scripting language

R

Code

  • Unicode code point
    • "U+" followed by its hexadecimal number.
    • e.g. U+0058, U+E0001, U+10FFFD
  • HTML numeric character reference
    • "&#" + decimal number + ";" or "&#x" + hexadecimal number + ";"
    • e.g. ¨, ¨, ♠, ♠
ISO Title Description Remarks
ISO 8601 Data elements and interchange formats an international standard covering the exchange of date- and time-related data.

Punctuation

Name Symbol HTML 4.0 Entity Remarks
parentheses ( ) round brackets, curved brackets, oval brackets
square brakets [ ] brakets
braces { } curly brackets, flower brackets
guillemets « » « »
single quotation mark ‘ ’ ‘ ’
double quotation mark “ ” “ ”
period . full stop, dot, point
ellipses … …
question mark ?
exclamation mark !
comma ,
colon :
semicolon ;
dash – — – —
hyphen ‐
apostrophe '
slash /

Typography

Name Symbol HTML 4.0 Entity Remarks
tilde ~ ˜
grave accent ` backtick, backquote
at sign @
number sign # pound sign, hash mark
percent %
caret ^
circumflex ˆ ˆ
ampersand & &
asterisk ∗ ∗ star
underscore _   understroke
backslash \
vertical bar | pipe, stick, vertical slash, thick colon
pilcrow ¶ paragraph mark, paragraph sign
bullet • •
section sign § §
prime ′ ′ minutes, feet
double prime ″ ″ seconds, inches
therefore sign ∴ ∴

Unicode

  • Unicode (more than 120,000 characters)
  • Universal Coded Character Set (ISO/IEC 10646)
    • positions 0 through 255 of ISO/IEC 10646 and Unicode are the same as in ISO-8859-1, alias ISO Latin 1.
    • positions 0 through 127 of ISO/IEC 10646 and Unicode are the same as in ISO/IEC 646.
    • the 128 ASCII and 256 ISO-8859-1 (Latin 1) characters are assigned Unicode/UCS code points that are the same as their codes in the earlier standards.

Date Time

Timezone

Locale

Currency

UUID

Color Space

Encoding

Encoding Efficiency Description Characters Remarks
Base64 75% Represent binary data in an ASCII string format by translating it into a radix-64 representation. [A-Za-z0-9], +, / padding: =
URL Encoding a mechanism for encoding information in a Uniform Resource Identifier (URI) [A-Za-z0-9], -, _, ~, . Percent Encoding

Online Encoder/Decoder

Site Description Remarks
Hex decoder Online hexadecimal to text converter

Codec

JPEG

PNG

GIF

BMP

FLV and F4V

Misc

  • The Swift Codes : Swift Codes or BIC Codes for all the Banks in the world.
  • ISO 9362
    • Defines a standard format of Business Identifier Codes (also known as SWIFT-BIC, BIC, SWIFT ID or SWIFT code) approved by the ISO.

Data Format

JSON

JSON Schema

JSON Path

JSON Pointer

JMESPath

References
Readings

YAML

References

Readings

Tips and Tricks

Abbreviated form of dictionaries and lists
martin: {name: Martin D'vloper, job: Developer, skill: Elite}
fruits: ['Apple', 'Orange', 'Strawberry', 'Mango']

Protocol Buffers

HOCON

TOML

HCL

Markup

  • MediaWiki
  • Markdown
  • MultiMarkdown
    • a tool to help turn minimally marked-up plain text into well formatted documents, including HTML, PDF (by way of LaTeX), OPML, or OpenDocument (specifically, Flat OpenDocument or ‘.fodt’, which can in turn be converted into RTF, Microsoft Word, or virtually any other word-processing format).
  • AsciiDoc
    • a text document format for writing notes, documentation, articles, books, ebooks, slideshows, web pages, man pages and blogs.
  • CSV Converter
    • allows you to enter a table in CSV-format and convert it to HTML or the WikiMedia format for tables.

MediaWiki Markup

  • Built-in classes for table
Class Description Remarks
wikitable Basic styling (light gray background, borders, padding and align left)
sortable Makes the table rows sortable by the selected/clicked header cell
mw-collapsible Adds a link to collapse the table collapsible
mw-collapsed Makes table to default to collapsed state collapsed
mw-datatable Allows for row highlighting

Magic Words

Behavior Switches Description Remarks
__TOC__ Places a table of contents at the word's current position
Variable Description Remarks
{{CURRENTYEAR}} Current Year
Function Description Remarks
{{formatnum:unformatted number}} Takes an unformatted number and outputs it in the localized digit script and formatted with decimal and decimal group separators, according to the wiki's default locale.

Extensions

Extension Description Remarks
Extension:SyntaxHighlight Provides rich formatting of source code using the <syntaxhighlight> tag. Powered by the Pygments library
Extension:Header Tabs Transforms top-level MediaWiki headers into tabs using the jQuery UI JavaScript library.
Extension:Math Provides support for rendering mathematical formulae.
Extension:SyntaxHighlight
  • Parameters
Parameter Description Remark
lang defines what lexer should be used
line enables line numbers
start defines the first line number of the code block
highlight specifies one or more lines that should be marked
inline indicates that the source code should be inline as part of a paragraph
class
style allows CSS attributes to be included directly

Real-world Examples

Markdown

kramdown

Readings

Textile

reStructuredText

LaTeX

Element Description Syntax/Sample Remarks
environments 
\begin{environmentname}
text to be influenced
\end{environmentname}
document, math
commands 
\commandname[option1,option2,...]{argument1}{argument2}...
  • Math Mode
    • There are a few ways to enter math mode, however the most common is $....$, where the text within the dollar signs is in the math mode environment. You have already been using math mode unknowingly by using the \begin{equation} and \end{equation} commands.
  • Whitespaces in math mode
Command Description Remarks
\quad space equal to the current font size (= 18 mu)
\, 3/18 of \quad (= 3 mu)
\: 4/18 of \quad (= 4 mu)
\; 5/18 of \quad (= 5 mu)
\! -3/18 of \quad (= -3 mu)
\qquad twice of \quad (= 36 mu)
\\ new line
\\~\\ double new line

Sample LaTex Codes

Simple equation with fraction
\begin{equation}
\\
difficulty = \frac{2^{256}}{target}

\\~\\

\frac{1}{difficulty} = \frac{target}{2^{256}} = mining \ probability
\\
\end{equation}

Multimedia

Name Width Height Aspect Ratio Remarks
VGA 640 480 4:3 Video Graphics Array
SVGA 800 600 4:3 Super VGA
HD 1280 720 16:9 High-Definition
FHD 1920 1080 16:9 Full HD
QHD 2560 1440 16:9 Quad HD (22 x HD)
4K UHD 3840 2160 16:9 Ultra HD (32 x HD, 22 x FHD), 4K
8K UHD 7680 4320 16:9 8K

Video

Disk Image

Format Extension Description Supported By Remarsk
ISO .iso, udf an archive file that contains everything that would be written to an optical disc, sector by sector, including the optical disc file system.
Media Descriptor File .mdf, .mds a proprietary disc image file format developed for Alcohol 120% Daemon Tools, CDemu, MagicISO, PowerDVD, WinCDEmu support Mixed Mode CD
IMG .img store raw disk images of floppy disks, hard drives, and optical discs or a vector image.
CUE Sheet .cue a metadata file which describes how the tracks of a CD or DVD[citation needed] are laid out. support Mixed Mode CD
CloneCD Control File .ccd a text descriptor with the extension .ccd used by CloneCD to mark the properties of a CD/DVD image. support Mixed Mode CD
Program Can Burn Can Mount Input Format Output Format OS
CDBurnerXP Yes No ISO, MDF Windows
ImgBurn Yes No ISO, IMG, CUE Windows
Virtual CloneDrive No Yes ISO, BIN, IMG, UDF, DVD, CCD Windows
WinCDEmu No Yes ISO, BIN, IMG, MDF, CCD, NRG Windows

Architecture and Model

Architecture

  • Twelve-factor App
    1. One codebase tracked in revision control, many deploys
    2. Explicitly declare and isolate dependencies
    3. Store config in the environment
    4. Treat backing services as attached resources
    5. Strictly separate build and run stages
    6. Execute the app as one or more stateless processes
    7. Export services via port binding
    8. Scale out via the process model
    9. Maximize robustness with fast startup and graceful shutdown
    10. Keep development, staging, and production as similar as possible
    11. Treat logs as event streams
    12. Run admin/management tasks as one-off processes

Pattern

UML

Profile, Constraint

Class Diagram

Activity Diagram

Sequence Diagram

Data Modeling

Common Data-types

Datatype Facet Standard/Guideline Remarks
Address BS7666 1994
Date Max 10 Characters in the format CCYY-MM-DD
e-Mail Address len < 255 IETF RFC2822
Datatype Facet Standard/Guideline Remarks
Person Family Name Alphanumeric, len <= 100
Person Given Name Alphanumeric, len <= 100

Common Glossary

Glossary Description Remarks
state describe a stage in a process
status describe an outcome of an operation HTTP Status Codes, Amazon Order Status, JIRA Ticket Status

Common Abbreviation

Full Abbreviation Description Remarks
account acct.
approval apprv.
authentication auth.
authorization authz.
business biz
campaign cmpgn
corporation corp.
contract contr.
customer cust
description descr desc is reserved word of Oracle and MySQL
document(s) doc.
payment pymt.
private pvt prv
public pub
receipt recpt
reference ref
representative rep
request req.
response resp.
transfer transf
value val.

Process Modeling

BPMN

CORBA

REST

Microservices

  • MonolithFirst (Martin Fowler, 3 June 2015)
    • Almost all the successful microservice stories have started with a monolith that got too big and was broken up
    • Almost all the cases where I've heard of a system that was built as a microservice system from scratch, it has ended up in serious trouble.
  • MicroservicePremium (Martin Fowler, 13 May 2015)
  • The Death of Microservice Madness in 2018 (2018)
  • Why microservices may not be for everybody (February 4, 2017)
  • The Microservices Hype (Aug 25, 2017)
    • Microservices give a lot of benefits, but with those benefits comes a lot of complexity, and this kind of complexity would drive you off the road of focusing on the business, which at the end is what really matters. Distribution, communication, discovery, isolation, health, are some of the topics that come hand in hand with this approach, and every each of them is its own monster to control.

Software Development Process

  • Rational Unified Process
    • RUP (Wikipedia)
    • Four project life-cycle phases
      • Inception phase
      • Elaboration phase
      • Construction phase
      • Transition phase
    • Six engineering disciplines
      • Business modelling
      • Requirements
      • Analysis and design
      • Implementation
      • Test
      • Deployment
    • Three supporting disciplines
      • Configuration and change management
      • Project management
      • Environment

Testing

  • Unit Testing Guidelines
    • Keep testing at unit level
    • Keep tests independent
    • Focus on execution coverage first
    • Cover boundary cases
    • Provide a random generator : a simple way to improve test coverage further is to generate random parameters so that the tests can be executed with different input every time.
    • Test each feature once

UI, UX

Real-time graph

Data Warehouse and Business Intelligence

Software Configuration Management

Issue Tracking

Contents Management

Networking

Procotol Full Name Description Layer Remarks
ARP Address Resolution Protocol a communication protocol used for discovering the link layer address, such as a MAC address, associated with a given internet layer address, typically an IPv4 address. Link IPv4 over IEEE 802.3 and IEEE 802.11
ICMP Internet Control Message Protocol used by network devices, including routers, to send error messages and operational information indicating, for example, that a requested service is not available or that a host or router could not be reached. Internet
BGP Border Gateway Protocol a standardized exterior gateway protocol designed to exchange routing and reachability information among autonomous systems (AS) on the Internet Application
SNMP Simple Network Management Protocol an Internet Standard protocol for collecting and organizing information about managed devices on IP networks and for modifying that information to change device behavior. Application
MQTT Message Queuing Telemetry Transport an ISO standard publish-subscribe-based messaging protocol. Application ISO/IEC PRF 20922
Name Pattern Description Remarks
eno1 On-board device with Firmware/BIOS provided index number
ens1 Off-board device with Firmware/BIOS provided PCI Express hotplug slot index number
enp2s0 Device with physical/geographical location of the connector of the hardware
enx78e7d1ea46da Device with the interfaces's MAC address
eth0 Classic, unpredictable kernel-native ethX naming
  • Networking Devices
Device Description Remarks
Router a networking device that forwards data packets between computer networks.

TCP/IP

Fundamentals

  • Classless Inter-Domain Routing(CIDR)
    • a method for allocating IP addresses and IP routing
    • IP address = most significant bits + least significant set
    • most significant bits : the network prefix, a whole network or subnet identifier
    • least significant set : the host identifier
classful name IP address range # of addresses subnet mask host id size mask bits RFC1918 name
single class A network 10.0.0.0 – 10.255.255.255 16,777,216 255.0.0.0 24 bits 8 bits 24-bit block
16 contiguous class B networks 172.16.0.0 – 172.31.255.255 1,048,576 255.240.0.0 20 bits 12 bits 20-bit block
256 contiguous class C networks 192.168.0.0 – 192.168.255.25 65,536 255.255.0.0 16 bits 16 bits
  • Priviliged ports
    • The TCP/IP port numbers below 1024 are special in that normal users are not allowed to run servers on them
  • Ephemeral port
    • a short-lived transport protocol port for Internet Protocol (IP) communications.
    • IANA : 49152 to 65535
    • Linux : 32768 to 61000 (/proc/sys/net/ipv4/ip_local_port_range)
TCP Socket Options
Option Description Remarks
TCP_USER_TIMEOUT When the value is greater than 0, it specifies the maximum amount of time in milliseconds that transmitted data may remain unacknowledged before TCP will forcibly close the corresponding connection and return ETIMEDOUT to the application. If the option value is specified as 0, TCP will use the system default.
TCP States
                   CLOSED
		            |  |
                    |  |
	      	       LISTEN
 	                |  |
 SYN RECEIVED ------+  +-------- SYN SENT
     |                               |
     +--------- ESTABLISHED ---------+
                    |  |
     +--------------+  +-------------+
     |                               |
     |                               |
 FIN WAIT 1 ------ CLOSING      CLOSE WAIT
     |                |              |
     |                |              |
 FIN WAIT 2 ----- TIME WAIT	     LAST ACT
                      |              |
                      |              |
                    CLOSED ----------+

Readings

DNS

Type Description Remarks
A record maps a name to one or more IP addresses
CNAME record maps a name to another name
ALIAS record maps a name to another name, but can coexist with other records on that name
URL record redirects the name to the target name using the HTTP 301 status code

LAN

HTTP

RFC Title Description Remarks
RFC-2616 Hypertext Transfer Protocol -- HTTP/1.1
RFC-7235 Hypertext Transfer Protocol (HTTP/1.1): Authentication
RFC-7239 RFC 7239 Forwarded HTTP Extension

Status Codes

Code Description Remarks
400 Bad Request The server cannot or will not process the request due to an apparent client error
401 Unauthorized Similar to 403 Forbidden, but specifically for use when authentication is required and has failed or has not yet been provided. Unauthenticated
403 Forbidden The request was valid, but the server is refusing action. Unauthorized
404 Not Found The requested resource could not be found but may be available in the future.
407 Proxy Authentication Required The client must first authenticate itself with the proxy.

Header Fields

General Header Fields
  • Header fields which have general applicability for both request and response messages, but which do not apply to the entity being transferred.
Header Description Remarks
Connection: keep-alive|close Controls whether or not the network connection stays open after the current transaction finishes.
Request Header Fields
Header Description Remarks
Authorization: type credentials Contains the credentials to authenticate a user agent with a server, usually after the server has responded with a 401 Unauthorized status and the WWW-Authenticate header.
Proxy-Authorization: type credentials Contains the credentials to authenticate a user agent to a proxy server, usually after the server has responded with a 407 Proxy Authentication Required status and the Proxy-Authenticate header.
Origin: scheme://hostname:port Initiates a request for cross-origin resource sharing (asks server for Access-Control-* response fields). CORS
Host: host:port The domain name of the server (for virtual hosting), and (optionally) the TCP port number on which the server is listening.
Forwarded: by=identifier;for=identifier;host=host;proto=http|https Contains information from the client-facing side of proxy servers that is altered or lost when a proxy is involved in the path of the request. X-Forwarded-For, X-Forwarded-Host
X-Forwarded-For: client, proxy1, proxy2 A de-facto standard header for identifying the originating IP address of a client connecting to a web server through an HTTP proxy or a load balancer. XFF
X-Forwarded-Host: host A de-facto standard header for identifying the original host requested by the client in the Host HTTP request header. XFH
X-Forwarded-Proto: protocol a de-facto standard header for identifying the protocol (HTTP or HTTPS) that a client used to connect to your proxy or load balancer. XFP
X-Request-ID Correlates HTTP requests between a client and server.
Response Header Fields
Header Description Remarks
WWW-Authenticate: type realm=realm Defines the authentication method that should be used to gain access to a resource. Basic, Digest, OAuth, HOBA
Proxy-Authenticate: type realm=realm Defines the authentication method that should be used to gain access to a resource behind a proxy server.
Access-Control-Allow-Credentials: true Tells browsers whether to expose the response to frontend JavaScript code when the request's credentials mode (Request.credentials) is "include". CORS
Access-Control-Allow-Origin: *|origin|null Indicates whether the response can be shared with requesting code from the given origin. CORS
Access-Control-Allow-Methods: method, method, ... Specifies the method or methods allowed when accessing the resource in response to a preflight request. CORS
Access-Control-Allow-Headers: header-name, header-name, ... Indicate which HTTP headers can be used during the actual request. CORS
Access-Control-Max-Age: delta-seconds Indicates how long the results of a preflight request can be cached. CORS
Content-Security-Policy: policy-directive; policy-directive allows web site administrators to control resources the user agent is allowed to load for a given page. CSP

Authentication

  • Related Status Codes
Status Code Description Remarks
401 Unauthorized indicates that the request has not been applied because it lacks valid authentication credentials for the target resource.
407 Proxy Authentication Required indicates that the client needs to authenticate itself in order to use a proxy.
  • Related HTTP Headers
Header Type Description Remarks
Authorization Request allows a user agent to authenticate itself with an origin server
Proxy-Authorization Request allows the client to identify itself (or its user) to a proxy that requires authentication.
WWW-Authenticate Response indicates the authentication scheme(s) and parameters applicable to the target resource.
Proxy-Authenticate Response indicates the authentication scheme(s) and parameters applicable to the proxy for this effective request URI
HTTP Authentication Scheme
Name Full Name Description Remarks
Basic Basic HTTP Authentication the client needs to authenticate itself with a user-id and a password for each protection space ("realm").
Digest HTTP Digest Access Authentication A valid response contains an unkeyed digest of the username, the password, the given nonce value, the HTTP method, and the requested URI.
OAuth OAuth 1.0
Bearer OAuth 2.0 Bearer Token A security token with the property that any party in possession of the token (a "bearer") can use the token in any way that any other party in possession of it can.

Mechanisms

Mechanism Description Related Remark
Cross-Origin Resource Sharing (CORS) a mechanism that uses additional HTTP headers to tell browsers to give a web application running at one origin, access to selected resources from a different origin. Origin, Access-Control-Allow-Origin
Content Security Policy (CSP) an added layer of security that helps to detect and mitigate certain types of attacks, including Cross Site Scripting (XSS) and data injection attacks. Content-Security-Policy

HTTP/2

WebSocket

  • WebSocket (on Wikipedia)
    • a computer communications protocol, providing full-duplex communication channels over a single TCP connection

SMTP

  • DomainKeys Identified Mail(DKIM)
    • an email authentication method designed to detect forged sender addresses in emails (email spoofing), a technique often used in phishing and email spam.
    • allows the receiver to check that an email claimed to have come from a specific domain was indeed authorized by the owner of that domain.

SNMP

  • Simple Network Management Protocol: an Internet Standard protocol for collecting and organizing information about managed devices on IP networks and for modifying that information to change device behavior.

MQTT

  • Message Queuing Telemetry Transport : an ISO standard (ISO/IEC PRF 20922) publish-subscribe-based messaging protocol.

RTSP

  • Real Time Streaming Protocol
  • RTSP (Wikipedia)
  • RTSP (MultimediaWiki)

RTMP

JSON-RPC

gRPC

  • an open source remote procedure call (RPC) system initially developed at Google.
  • https://www.grpc.io/

Security

Glossary

Category Term Description Remarks
Attack Vulnerability a weakness which can be exploited by a Threat Actor, such as an attacker, to perform unauthorised actions within a computer system at Longman
Eavesdropping secretly or steathily listening to the private conversation of others without their consent at Longman
Session hijacking the exploitation of a valid computer session—sometimes also called a session key—to gain unauthorized access to information or services in a computer system
Man-in-the-middle attack an attack where the attacker secretly relays and possibly alters the communication between two parties who believe they are directly communicating with each other
Rainbow table a precomputed table for reversing cryptographic hash functions, usually for cracking password hashes.
Phishing the fraudulent attempt to obtain sensitive information such as usernames, passwords and credit card details by disguising oneself as a trustworthy entity in an electronic communication. fake website, social engineering
Scam an attempt to defraud a person or group after first gaining their confidence, used in the classical sense of trust.
Sybil Attack the attacker subverts the reputation system of a network service by creating a large number of pseudonymous identities and uses them to gain a disproportionately large influence.
Defense Cipher an algorithm for performing encryption or decryption
Key stretching techniques used to make a possibly weak key, typically a password or passphrase, more secure against a brute-force attack by increasing the resources (time and possibly space) it takes to test each possible key. PBKDF2, bcrypt, scrypt, Argon2
Credential an attestation of qualification, competence, or authority issued to an individual by a third party with a relevant or de facto authority or assumed competence to do so. diplomas, academic degrees, certifications, identification documents, badges, passwords, keys, ...
MAC(Message Authentication Code) a short piece of information used to authenticate a message—in other words, to confirm that the message came from the stated sender (its authenticity) and has not been changed. MACs differ from digital signatures as MAC values are both generated and verified using the same secret key.
Tamperproofing a methodology used to hinder, deter or detect unauthorised access to a device or circumvention of a security system. misnomer

Concepts

A digital signature is a mathematical scheme for verifying the authenticity of digital messages or documents. A valid digital signature, where the prerequisites are satisfied, gives a recipient very strong reason to believe that the message was created by a known sender (authentication), and that the message was not altered in transit (integrity)
A key generation algorithm that selects a private key uniformly at random from a set of possible private keys. The algorithm outputs the private key and a corresponding public key.
A key generation algorithm that selects a private key uniformly at random from a set of possible private keys. The algorithm outputs the private key and a corresponding public key.
A signature verifying algorithm that, given the message, public key and signature, either accepts or rejects the message's claim to authenticity.

The DSA algorithm involves four operations: key generation (which creates the key pair), key distribution, signing and signature verification.

The Elliptic Curve Digital Signature Algorithm (ECDSA) offers a variant of the Digital Signature Algorithm (DSA) which uses elliptic curve cryptography.

A public key certificate, also known as a digital certificate or identity certificate, is an electronic document used to prove the ownership of a public key.

A public key infrastructure (PKI) is a set of roles, policies, hardware, software and procedures needed to create, manage, distribute, use, store and revoke digital certificates and manage public-key encryption.

Standards

standard description remarks
Rainbow Series a series of computer security standards and guidelines published by the United States government in the 1980s and 1990s
Trusted Computer System Evaluation Criteria (TCSEC) a United States Government Department of Defense (DoD) standard that sets basic requirements for assessing the effectiveness of computer security controls built into a computer system Orange Book
Common Criteria(CC) an international standard (ISO/IEC 15408) for computer security certification Common Criteria for Information Technology Security Evaluation
FIPS 140-2 Security Requirements for Cryptographic Modules
X.500 a series of computer networking standards covering electronic directory services
X.509 a standard that defines the format of public key certificates

Standard Algorithm Names

  • Cipher Algorithm Names
Cipher Algorithm Description Spec Remark
AES a 128-bit block cipher supporting keys of 128, 192, and 256 bits. FIPS 197 AES_128, AES_192, AES_256
DES a symmetric-key algorithm for the encryption of digital data. FIPS PUB 46-3
Blowfish a symmetric-key block cipher, designed in 1993 by Bruce Schneier
  • Cipher Algorithm Modes
  • Cipher Algorithm Paddings
  • Key Generator Algorithm
  • Key Pair Generator Algorithm
Algorithm Description Remarks
DiffieHellman Generates keypairs for the Diffie-Hellman KeyAgreement algorithm.
DSA Generates keypairs for the Digital Signature Algorithm.
RSA Generates keypairs for the RSA algorithm (Signature/Cipher).
EC Generates keypairs for the Elliptic Curve algorithm.
  • MAC Algorithms
  • Message Digest Algorithms
  • Signature Algorithms
    • <digest>with<encryption>
Algorithms Description Spec Remarks
NONEwithRSA Does not use a digesting algorithm before performing the RSA operation PKCS #1
MD2withRSA Uses the MD2 digest algorithm and RSA to create and verify RSA digital signatures PKCS #1
MD5withRSA Uses the MD5 digest algorithm and RSA to create and verify RSA digital signatures PKCS #1

Fundamentals

Hashing

  • Salt
    • Random data that is used as an additional input to a one-way function that hashes data, a password or passphrase.
  • How to store salt?

Access Control

Misc

  • CAPTCHA (Completely Automated Public Turing Test to tell Computers and Humans Apart)
    • a type of challenge–response test used in computing to determine whether or not the user is human.

Cryptography

Key Concepts

  • Cryptography
    • about constructing and analyzing protocols that prevent third parties or the public from reading private messages
  • Cryptosystem
    • a suite of cryptographic algorithms needed to implement a particular security service, most commonly for achieving confidentiality (encryption)
    • key generation algorithm + encryption algorithm + decryption algorithm
  • Cipher(Cypher)
    • an algorithm for performing encryption or decryption
  • Block cipher mode of operation
    • ECB, CBC, PCBC, CFB, OFB, ...

PKCS

  • Public Key Cryptography Standards : a group of public-key cryptography standards devised and published by RSA Security Inc
  • PKCS (on Wikipedia)
Standard Name Description Remarks
PKCS #3 Diffie–Hellman Key Agreement Standard A cryptographic protocol that allows two parties that have no prior knowledge of each other to jointly establish a shared secret key over an insecure communications channel.
PKCS #5 Password-based Encryption Standard RFC 8018, PBKDF2
PKCS #6 Extended-Certificate Syntax Standard defines extensions to the old v1 X.509 certificate specification. Obsoleted
PKCS #8 Private-Key Information Syntax Standard a standard syntax for storing private key information.
PKCS #11 Cryptographic Token Interface defining a generic interface to cryptographic tokens Cryptoki
PKCS #12 Personal Information Exchange Syntax Standard defines an archive file format for storing many cryptography objects as a single file
PKCS#11

SHA

  • Implementations
Category Implementation Remarks
Java java.security.MessageDigest class JDK
org.apache.commons.codec.digest.Sha2Crypt class Apache Commons
static String sha256Crypt(byte[] keyBytes, String salt)
static String sha512Crypt(byte[] keyBytes, String salt)
Python 2.x hashlib module md5(), sha1(), sha224(), sha256(), sha384(), sha512()
Python 3.x hashlib module md5(), sha1(), sha224(), sha256(), sha384(), sha512(), ...
Node.js crypto.Hash class sha256, sha512
MySQL SHA1() function
SHA2() function

Code Snippet

  • Python 2.x
>>> # Hash 'Hello, World!' with SHA-256 into hex representation
>>> import hashlib
>>> s = hashlib.sha256()
>>> s.update("Hello, World!")
>>> s.hexdigest()
  • JavaScript / Node.js
> // Hash 'Hello, World!' with SHA-256 into hex representation
> crypto.createHash('sha256').update('Hello, World!').digest('hex');

PEM

Digital Signature

Element Description Remarks
Key Generation Algorithm generates a private key and a corresponding public key
Signing Algorithm given a message and a private key, produces a signature
Signature Verifying Algorithm given the message, public key and signature, either accepts or rejects the message's claim to authenticity
Scheme Description Remarks
RSA one of the first public-key cryptosystems based on the practical difficulty of the factorization of the product of two large prime numbers. factoring problem
RSA-PSS a cryptographic signature scheme designed by Mihir Bellare and Phillip Rogaway PKCS#1 v2.1
DSA a Federal Information Processing Standard for digital signatures, based on the mathematical concept of modular exponentiation and the discrete logarithm problem
ECDSA a variant of the DSA which uses elliptic curve cryptography

ECDSA





Step Equation Remarks
calculate message hash
calculate
select secure random cryptographically secure random
calculate the curve point public key
calculate
calculate
Symbol Concept Remarks
elliptic curve base point
integer order of prime number
private key randomly selected
public key calculated by elliptic curve
message
signature

PKI

Concepts

Extension Description Remarks
.pem (Privacy-enhanced Electronic Mail) Base64 encoded DER certificate, enclosed between "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----"
.cer, .crt, .der usually in binary DER form, but Base64-encoded certificates are common too
.p12 PKCS#12, may contain certificate(s) (public) and private keys (password protected)
.pfx PFX, predecessor of PKCS#12

Readings

Concept Description Remarks
Authority Key Identifier The authority key identifier extension provides a means of identifying the public key corresponding to the private key used to sign a certificate
Issuer The issuer field identifies the entity that has signed and issued the certificate

HSM

Key Store

OAuth

HTTP Authentication

TLS

SSH

Elements

Element Description Remarks
passphrase password for the private key
~/.ssh/authorized_keys a list of public keys to allow to in-access into this account from outside used when accepting others
~/.ssh/known_hosts a list of public keys to allow for current user to out-access used when accessing outer server
~/.ssh/id_dsa, ~/.ssh/id_rsa a private key of current account
~/.ssh/id_dsa.pub, ~/.ssh/id_rsa.pub a public key of current account

Commands

Readings

ssh
scp
Port Forwarding
  • Port forwarding (on Wikipedia)
    • Local port forwarding
    • Remote port forwarding
    • Dynamic port forwarding
Local port forwarding

    Application --- Local Port ------- Bastion Host(Jump Server) ------- Destination Server
            Client side                                                     Remote side

Remote port forwarding
    
    @TODO

DID

JWT

  • https://jwt.io/
  • Desc. : an open, industry standard RFC 7519 method for representing claims securely between two parties.

Business Solutions

  • Business Processes
Process Description Remarks
Order Fulfillment The complete process from point of sales inquiry to delivery of a product to the customer.

SAP R/3

misc

Documentation Convention

HP Notebook

Software Licenses

Examples

Hardware

  • Open Compute Project
    • to develop servers and data centers following the model traditionally associated with open source software projects.

Computer Bus

Bus Full Name Description Remarks
PATA Parallel AT Attachment(ATA) an interface standard for the connection of storage devices such as hard disk drives, floppy disk drives, and optical disc drives in computers. ATA, IDE
SATA Serial AT Attachment(ATA) a computer bus interface that connects host bus adapters to mass storage devices such as hard disk drives, optical drives, and solid-state drives
PCIe Peripheral Component Interconnect(PCI) Express a high-speed serial computer expansion bus standard, designed to replace the older PCI, PCI-X and AGP bus standards. PCI Express, PCI-e
M.2 a specification for internally mounted computer expansion cards and associated connectors
NVMe Non-Volatile Memory(NVM) Express an open logical device interface specification for accessing non-volatile storage media attached via a PCI Express (PCIe) bus.
Thunderbolt the brand name of a hardware interface developed by Intel (in collaboration with Apple) that allows the connection of external peripherals to a computer.
Retrieved from "https://3rdstage.fandom.com/wiki/Computer_Programming?oldid=16729"
Community content is available under CC-BY-SA unless otherwise noted.

Fan Feed

More 3rdstage's Wiki
Explore Wikis