3rdstage's Wiki
Don't have an account?
Register
Register
in: Pages using deprecated enclose attributes

Changes: On HAProxy, Wireshark and tcpdump

Back to page
View source
Line 170: Line 170:
 
| style="white-space: nowrap;" | [http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-capture%20request%20header <tt>capture request header &lt;name&gt; len &lt;length&gt;</tt>]
  
| style="white-space: nowrap;" | [http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-capture%20request%20header <tt>capture request header &lt;name&gt; len &lt;length&gt;</tt>]
 
| Capture and log the last occurrence of the specified request header.
  
| Capture and log the last occurrence of the specified request header.
  +
| X/O/O/X || Not for Back-end
−
| ||
  
 
|-
  
|-
 
| [http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-source <tt><nowiki>source <addr>[:<port>] [usesrc ...]</nowiki></tt>]
  
| [http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-source <tt><nowiki>source <addr>[:<port>] [usesrc ...]</nowiki></tt>]

Revision as of 06:35, 4 June 2019

HAProxy

  • http://www.haproxy.org/
  • Desc. : a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications
  • License:
  • Written in:
  • Sources:

References

  
  Proxy ----------- Gateway
             |
             +----- Forward Proxy
             |
             +----- Reverse Proxy ----------- SSL Acceleration Proxy
                                       |
                                       +----- SSL Termination Proxy
                                       |
                                       +----- Load Balancer
  • HAProxy Configuration
config = global + defaults* + frontend* + backend* + listen*

global = process management and security parameters 
         + performance tuning parameters 
         + debugging parameters
         + user lists
         + peers
         + mailers

Global Parameters

Process management and security parameters
Parameter Description Remarks
cpu-map Specifies CPU sets for process or thread sets Linux 2.6+,
log Adds a global syslog server
nbproc <number> Creates <number> processes when going daemon default: 1
nbthread <number> Creates <number> threads for each created processes
ulimit-n Sets the maximum number of per-process file-descriptors Recommended not to use this option
ssl-default-bind-options Sets default ssl-options to force on all "bind" lines.
Performance tuning parameters
Parameter Description Remarks
tune.ssl.cachesize <number> Sets the size of the global SSL session cache, in a number of blocks. default: 20,000
tune.ssl.lifetime <timeout> Sets how long a cached SSL session may remain valid in seconds. default: 300s (5 min)
Access control parameters
Parameter Description Remarks
user <username> [password|insecure-password <password>] [groups <group>,<group>,(...)] Adds user <username> to the current userlist.

Proxy Keywords

Keywords Description DF/FE/LI/BE Remarks
mode { tcp|http|health } Set the running mode or protocol of the instance
bind [<address>]:<port_range> [, ...] [param*] Define one or several listening addresses and/or ports in a frontend X/O/O/X
tcp-request inspect-delay <timeout> Set the maximum allowed time to wait for data during content inspection
tcp-response content <action> [{if | unless} <condition>] Perform an action on a session response depending on a layer 4-7 condition
option http_proxy Enable or disable plain HTTP proxy mode Forward Proxy
option http-keep-alive Enable or disable HTTP keep-alive from client to server
option httpclose Enable or disable passive HTTP connection closing "Connection: close" header
Deprecated
option forwardfor Enable insertion of the X-Forwarded-For header to requests sent to servers "X-Forwarded-For" header
option socket-stats Enable or disable collecting & providing separate statistics for each socket.
capture request header <name> len <length> Capture and log the last occurrence of the specified request header. X/O/O/X Not for Back-end
source <addr>[:<port>] [usesrc ...] Set the source address for outgoing connections
server <name> <address>[:[port]] [param*] Declare a server in a backend
  • http-request
    • defines a set of rules which apply to layer 7 processing.
Action Description Remarks
http-request allow Stops the evaluation of the rules and lets the request pass the check.
http-request deny Stops the evaluation of the rules and immediately rejects the request.
http-request auth Stops the evaluation of the rules and immediately responds with an HTTP 401 or 407 error code to invite the user to present a valid user name and password
Timeout
Keywords Description Remarks
timeout connect <timeout> Set the maximum time to wait for a connection attempt to a server to succeed
timeout server <timeout> Set the maximum inactivity time on the server side
timeout client <timeout> Set the maximum inactivity time on the client side
timeout tunnel <timeout> Set the maximum inactivity time on the client and server side for tunnels
timeout server-fin <timeout> Set the inactivity timeout on the server side for half-closed connections
timeout client-fin <timeout> Set the inactivity timeout on the client side for half-closed connections
timeout http-request <timeout> Set the maximum allowed time to wait for a complete HTTP request
timeout http-keep-alive <timeout> Set the maximum allowed time to wait for a new HTTP request to appear
Logging
Keywords Description Remarks
option httplog Enable logging of HTTP request, session state and timers
option dontlognull Enable or disable logging of null connections
option dontlog-normal Enable or disable logging of normal, successful connections
option log-health-checks Enable or disable logging of health checks status updates
log-format Specifies the log format string to use for traffic logs
Health Check
Keywords Description Remarks
option httpchk <method> <uri> <version> Enable HTTP protocol to check on the servers health
Statistics
Keywords Description DF/FE/LI/BE Remarks
stats enable Enable statistics reporting with default settings O/O/O/O
stats admin { if | unless } <cond> Enable statistics admin level if/unless a condition is matched X/O/O/O
stats auth <user>:<passwd> Enable statistics with authentication and grant access to an account O/O/O/O

Bind Options

Options Description Remarks
ca-ignore-err Sets a comma separated list of errorIDs to ignore during verify at depth > 0
crt-ignore-err Sets a comma separated list of errorIDs to ignore during verify at depth == 0
no-sslv3 Disables support for SSLv3 on any sockets instantiated from the listener when SSL is supported
ssl-min-ver Enforces use of <version> or upper on SSL connections instantiated

from this listener

TLSv1.0 | TLSv1.1 | TLSv1.2 | TLSv1.3
alpn Enables the TLS ALPN extension and advertises the specified protocol list as supported on top of ALPN. ALPN (on Wikipedia)
npn Enables the NPN TLS extension and advertises the specified protocol list as supported on top of NPN.

Server Options

Options Description Remarks
check Enables health checks on the server.
inter <delay> Sets the interval between two consecutive health checks to <delay> milliseconds. default: 2000ms
check-ssl Forces encryption of all health checks over SSL, regardless of whether the server uses SSL or not for the normal traffic.
ssl Enables SSL ciphering on outgoing connections to the server.
verify [none|required]

ACL

acl <aclname> <criterion> [flags] [operator] [<value>] ...

acl <aclname> <sample fetch method> [flags] [operator] [<value>] ...
Element Description Remarks
ACL Name (aclname) [-_.:A-Za-z0-9]*
Flags (flags) -i, -f, -m, -n, -M, -u, --
ACL Flags
Flag Description Remarks
-i ignore case during matching of all subsequent patterns.
-m found only check if the requested sample could be found in the stream, but do not compare it against any pattern.
-m bool check the value as a boolean.
-m int match the value as an integer.
-m len match the sample's length as an integer.
-m str exact match
-m sub substring match
-m reg regex match
-m beg prefix match
-m end suffix match

Logging

Log Fields
Field Name Type Description Remarks
%ci client_ip IP
%ft frontend_name_transport string '~' suffix for SSL
%b backend_name string
%si server_IP IP target address
%Tq Th + Ti + TR numeric total time to get the client request from the accept date or since the emission of the last byte of the previous response HTTP mode only
%Tw Tw numeric total time spent in the queues waiting for a connection slot.
%Tc Tc numeric total time to establish the TCP connection to the server.
%Tr response time numeric server response time HTTP mode only
%Tt Tt numeric total session duration time
%ST status_code numeric
%B bytes_read numeric from server to client
%ts termination_state string
%sq srv_queue numeric
%bq backend_queue numeric
%hr captured_request_headers string default style
%r http_request string HTTP mode only
Timings Events
                 first request               2nd request
      |<-------------------------------->|<-------------- ...
      t         tr                       t    tr ...
   ---|----|----|----|----|----|----|----|----|--
      : Th   Ti   TR   Tw   Tc   Tr   Td : Ti   ...
      :<---- Tq ---->:                   :
      :<-------------- Tt -------------->:
                :<--------- Ta --------->:
Event Name Description Remarks
Th handshakes total time to accept tcp connection and execute handshakes for low level protocols.
Ti idle the idle time before the HTTP request.
TR Request total time to get the client request.
Tq total time to get the client request from the accept date or since the emission of the last byte of the previous response. Th + Ti + TR
Tw waiting total time spent in the queues waiting for a connection slot.
Tc connection total time to establish the TCP connection to the server.
Tr response server response time
Td data the data transmission time Tt - (Th + Ti + TR + Tw + Tc + Tr)
Ta active total active time for the HTTP request
Tt total total session duration time

Sample Fetch Methods

Method Type Description Remarks
src ip This is the source IPv4 address of the client of the session
Method Type Description Remarks
req.payload_lv binary Extracts a binary block whose size is specified at <offset1> for <length> bytes
res.payload_lv binary extracts a binary block whose size is specified at <offset1> for <length> bytes, and which starts at <offset2> if specified or just after the length in the response buffer.
req.ssl_hello_type integer Returns an integer value containing the type of the SSL hello message found in the request buffer if the buffer contains data that parse as a complete SSL (v3 or superior) client hello message.
res.ssl_hello_type integer Returns an integer value containing the type of the SSL hello message found in the response buffer if the buffer contains data that parses as a complete SSL (v3 or superior) hello message
Method Type Description Remarks
req.hdr([<name>[,<occ>]]) string Extracts the last occurrence of header <name> in an HTTP request When used from an ACL, all occurrences are iterated over until a match is found
res.hdr([<name>[,<occ>]]) string Extracts the last occurrence of header <name> in an HTTP response.
hdr([<name>[,<occ>]]) string Equivalent to req.hdr() when used on requests, and to res.hdr() when used on responses
http_auth(<userlist>) boolean Returns a boolean indicating whether the authentication data received from the client match a username & password stored in the specified userlist.
http_auth_group(<userlist>) string Returns a string corresponding to the user name found in the authentication data received from the client if both the user name and password are valid according to the specified userlist. ACL derivatives

Readings

ACL

Proxying

SSL

WebSockets

HTTP/2

Load Balancing

Logging and Statistics

Health Check

Throttling

Samples

Typical Sample for Reverse Proxy

# References
#   http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#3
#   http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4
#   https://kvz.io/blog/2010/08/11/haproxy-logging/

global
  daemon
  log /var/lib/haproxy/dev/log local0 info
  log /var/lib/haproxy/dev/log local1 notice

  ca-base /etc/haproxy/ssl/trusted
  crt-base /etc/haproxy/ssl

  maxconn 2048
  tune.ssl.default-dh-param 2048 

  #https://disablessl3.com/
  ssl-default-bind-options no-sslv3
  
defaults
  mode http
  log global

  option forwardfor
  
  timeout connect 5s
  timeout client 10s
  timeout server 10s
  timeout tunnel 600s
  timeout server-fin 10s
  timeout client-fin 10s
  timeout http-request 5s
  timeout http-keep-alive 2s
  
  errorfile 400 /etc/haproxy/errors/400.http
  errorfile 403 /etc/haproxy/errors/403.http
  errorfile 408 /etc/haproxy/errors/408.http
  errorfile 500 /etc/haproxy/errors/500.http
  errorfile 502 /etc/haproxy/errors/502.http
  errorfile 503 /etc/haproxy/errors/503.http
  errorfile 504 /etc/haproxy/errors/504.http

listen stats
  bind 192.168.1.11:8070
  
  stats enable
  stats hide-version
  stats realm HAProxy\ Statistics
  stats uri /stats

frontend http
  bind 192.168.1.11:8080
  
  option httplog
  option dontlognull
  log-format %ci\ [%t]\ %ft\ %b\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %ts\ %sq/%bq\ %{+Q}r

  http-request set-header X-Haproxy-Current-Date %T
  http-request set-header X-Forwarded-Proto http
  default_backend http-rear


frontend https
  bind 192.168.1.11:8443 ssl crt server-tls.pem ca-file ca.crt
  # bind 192.168.1.11:8443 ssl crt server-tls.pem ca-file ca.crt verify optional crt-ignore-err all ca-ignore-err all   # For mutual auth

  option httplog
  option dontlognull
  log-format %ci\ %ft\ %b\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %ts\ %sq/%bq\ %{+Q}r

  #http-request deny deny_status 400 unless { ssl_fc }
  #http-request deny deny_status 400 unless { ssl_c_used }
  #http-request deny deny_status 400 unless { ssl_c_verify 0 }
  
  # setup request header for logging or backend usage
  # https://www.haproxy.com/blog/ssl-client-certificate-information-in-http-headers-and-logs/
  # https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4.2-http-request
  http-request set-header X-Haproxy-Current-Date %T
  http-request set-header X-SSL                  %[ssl_fc]
  http-request set-header X-SSL-Session_ID       %[ssl_fc_session_id,hex]
  http-request set-header X-SSL-Client-Verify    %[ssl_c_verify]    
  http-request set-header X-SSL-Client-DN        %{+Q}[ssl_c_s_dn]
  http-request set-header X-SSL-Client-CN        %{+Q}[ssl_c_s_dn(cn)]
  http-request set-header X-SSL-Issuer           %{+Q}[ssl_c_i_dn]
  http-request set-header X-SSL-Client-NotBefore %{+Q}[ssl_c_notbefore]
  http-request set-header X-SSL-Client-NotAfter  %{+Q}[ssl_c_notafter]
  http-request set-header X-Forwarded-Proto https

  default_backend http-rear


frontend ws
  bind 192.168.1.11:8090

  acl is_connection_upgrade hdr(Connection) -i upgrade
  acl is_upgrade_websocket hdr(Upgrade) -i websocket
  acl has_websocket_key hdr_cnt(Sec-WebSocket-Key) eq 1
  acl has_websocket_ver hdr_cnt(Sec-WebSocket-Version) eq 1

  http-request deny unless is_connection_upgrade is_upgrade_websocket has_websocket_key has_websocket_ver

  default_backend ws-rear


frontend wss
  bind 192.168.1.11:8453 ssl crt server-tls.pem ca-file ca.crt
  # bind 192.168.1.11:8453 ssl crt server-tls.pem ca-file ca.crt verify optional crt-ignore-err all ca-ignore-err all   # For mutual auth

  #http-request deny deny_status 400 unless { ssl_fc }
  #http-request deny deny_status 400 unless { ssl_c_used }
  #http-request deny deny_status 400 unless { ssl_c_verify 0 }

  acl is_connection_upgrade hdr(Connection) -i upgrade
  acl is_upgrade_websocket hdr(Upgrade) -i websocket
  acl has_websocket_key hdr_cnt(Sec-WebSocket-Key) eq 1
  acl has_websocket_ver hdr_cnt(Sec-WebSocket-Version) eq 1

  http-request deny unless is_connection_upgrade is_upgrade_websocket has_websocket_key has_websocket_ver

  default_backend ws-rear

backend http-rear
  server s1 127.0.0.1:80 check

backend ws-rear
  server s1 127.0.0.1:90 check

Typical Sample for Forward Proxy

# Sample HAProxy configuration for dedicated forward proxy

# References
#    https://stackoverflow.com/questions/49433417/setup-https-forward-proxy-with-haproxy
#    https://serverfault.com/questions/477642/using-haproxy-for-transparent-forwarding-and-selective-redirection

global
  daemon
  log /var/lib/haproxy/dev/log local0 info
  log /var/lib/haproxy/dev/log local1 notice

  ca-base /etc/haproxy/ssl/trusted
  crt-base /etc/haproxy/ssl

  maxconn 2048
  tune.ssl.default-dh-param 2048
  ssl-default-bind-options no-sslv3

userlist users
  group regular-users

  user tom password ... groups regular-users

defaults
  mode http
  log global

  timeout connect 5s
  timeout client 10s
  timeout server 10s
  timeout tunnel 600s
  timeout server-fin 10s
  timeout client-fin 10s
  timeout http-request 5s
  timeout http-keep-alive 2s

  errorfile 400 /etc/haproxy/errors/400.http
  errorfile 403 /etc/haproxy/errors/403.http
  errorfile 408 /etc/haproxy/errors/408.http
  errorfile 500 /etc/haproxy/errors/500.http
  errorfile 502 /etc/haproxy/errors/502.http
  errorfile 503 /etc/haproxy/errors/503.http
  errorfile 504 /etc/haproxy/errors/504.http

listen stats
  bind 192.168.1.11:8070

  stats enable
  stats hide-version
  stats realm HAProxy\ Statistics
  stats uri /stats
  stats auth proxyadmin:p8dp8d

resolvers dns
  nameserver dns1 8.8.8.8
  nameserver dns2 8.8.4.4
  hold valid 10s

frontend http
  bind 192.168.1.11:8080

  option http-use-proxy-header
  option httplog
  option dontlognull
  log-format %ci\ [%t]\ %ft\ %b\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %ts\ %sq/%bq\ %{+Q}r
  
  acl is-allowed-users http_auth_groups(users) regular-users
  http-request auth unless is-allowed-users   # 407 for unauthorized access

  default_backend outside

backend outside

  option httpclose
  option http_proxy

Wireshark

References

Operator Symbol Description Remarks
eq == Equal
ne != Not Equal
gt > Greater Than
lt < Less Than
ge >= Greater than or Equal to
le <= Less than or Equal to
contains Does the protocol, field or slice contain a value
matches ~ Does the protocol or text string match the given case-insensitive Perl-compatible regex
[i:j] Slices with i = start_offset, j = length
[i-j] Slices with i = start_offset, j = end_offset, inclusive
[i] Slices with i = start_offset, length = 1
[:j] Slices with start_offset = 0, length = j
[i:] Slices with start_offset = i, end_offset = end_of_field
and && Logical AND
or || Logical OR
not ! Logical NOT
Protocol Typical Fields Description Remarks
tcp port Transmission Control Protocol
ip addr, dst, src Internet Protocol Version 4
http Hypertext Transfer Protocol
ssl Secure Sockets Layer
websocket WebSocket

Readings

Tips and Tricks

Typical display filters

ip.src == 192.168.1.31 and ip.addr == 203.252.150.28 and http

tcpdump

tcpdump [ -AbdDefhHIJKlLnNOpqStuUvxX# ] [ -B buffer_size ] 

         [ -c count ] [ -C file_size ] 
         [ -E spi@ipaddr algo:secret,... ] 
         [ -F file ] [ -G rotate_seconds ] [ -i interface ] 
         [ --immediate-mode ] [ -j tstamp_type ] [ -m module ] 
         [ -M secret ] [ --number ] [ --print ] [ -Q in|out|inout ] 
         [ -r file ] [ -s snaplen ] [ -T type ] [ --version ] 
         [ -V file ] [ -w file ] [ -W filecount ] [ -y datalinktype ] 
         [ -z postrotate-command ] [ -Z user ] 
         [ --time-stamp-precision=tstamp_precision ] 
         [ expression ]

Options

Option Description Remarks
-n Don't convert addresses
-i interface Listen on interface
-A Print each packet (minus its link level header) in ASCII Handy for capturing web pages
-s n Snarf n bytes of data from each packet rather than the default of 262144 bytes Setting n to 0 sets it to the default of 262144
-x When parsing and printing, in addition to printing the headers of each packet, print the data of each packet (minus its link level header) in hex

Filter Expression

Primitive Description Remarks
host host True if either the IPv4/v6 source or destination of the packet is host
dst host host True if the IPv4/v6 destination field of the packet is host
src host host True if the IPv4/v6 source field of the packet is host
port port True if either the source or destination port of the packet is port.
dst port port True if the packet is ip/tcp, ip/udp, ip6/tcp or ip6/udp and has a destination port value of port.
src port port True if the packet has a source port value of port.
ip proto protocol True if the packet is an IPv4 packet of protocol type protocol.
tcp = ip proto tcp
Community content is available under CC-BY-SA unless otherwise noted.