FANDOM


HAProxy

  • http://www.haproxy.org/
  • Desc. : a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications
  • License:
  • Written in:
  • Sources:

References

 
  Proxy ----------- Gateway
             |
             +----- Forward Proxy
             |
             +----- Reverse Proxy ----------- SSL Acceleration Proxy
                                       |
                                       +----- SSL Termination Proxy
                                       |
                                       +----- Load Balancer
  • HAProxy Configuration
config = global + defaults* + frontend* + backend* + listen*

global = process management and security parameters
         + performance tuning parameters
         + debugging parameters
         + user lists
         + peers
         + mailers

Global Parameters

Process management and security parameters
Parameter Description Remarks
log Adds a global syslog server
ulimit-n Sets the maximum number of per-process file-descriptors Recommended not to use this option
ssl-default-bind-options Sets default ssl-options to force on all "bind" lines.
Access control parameters
Parameter Description Remarks
user <username> [password|insecure-password <password>] [groups <group>,<group>,(...)] Adds user <username> to the current userlist.

Proxy Keywords

Keywords Description DF/FE/LI/BE Remarks
mode { tcp|http|health } Set the running mode or protocol of the instance
bind [<address>]:<port_range> [, ...] [param*] Define one or several listening addresses and/or ports in a frontend X/O/O/X
tcp-request inspect-delay <timeout> Set the maximum allowed time to wait for data during content inspection
tcp-response content <action> [{if | unless} <condition>] Perform an action on a session response depending on a layer 4-7 condition
option http_proxy Enable or disable plain HTTP proxy mode Forward Proxy
option http-keep-alive Enable or disable HTTP keep-alive from client to server
option httpclose Enable or disable passive HTTP connection closing "Connection: close" header
Deprecated
option forwardfor Enable insertion of the X-Forwarded-For header to requests sent to servers "X-Forwarded-For" header
option socket-stats Enable or disable collecting & providing separate statistics for each socket.
capture request header <name> len <length> Capture and log the last occurrence of the specified request header.
source <addr>[:<port>] [usesrc ...] Set the source address for outgoing connections
server <name> <address>[:[port]] [param*] Declare a server in a backend
  • http-request
    • defines a set of rules which apply to layer 7 processing.
Action Description Remarks
http-request allow Stops the evaluation of the rules and lets the request pass the check.
http-request deny Stops the evaluation of the rules and immediately rejects the request.
http-request auth Stops the evaluation of the rules and immediately responds with an HTTP 401 or 407 error code to invite the user to present a valid user name and password
Timeout
Keywords Description Remarks
timeout connect <timeout> Set the maximum time to wait for a connection attempt to a server to succeed
timeout server <timeout> Set the maximum inactivity time on the server side
timeout client <timeout> Set the maximum inactivity time on the client side
timeout tunnel <timeout> Set the maximum inactivity time on the client and server side for tunnels
timeout server-fin <timeout> Set the inactivity timeout on the server side for half-closed connections
timeout client-fin <timeout> Set the inactivity timeout on the client side for half-closed connections
timeout http-request <timeout> Set the maximum allowed time to wait for a complete HTTP request
timeout http-keep-alive <timeout> Set the maximum allowed time to wait for a new HTTP request to appear
Logging
Keywords Description Remarks
option httplog Enable logging of HTTP request, session state and timers
option dontlognull Enable or disable logging of null connections
option dontlog-normal Enable or disable logging of normal, successful connections option log-health-checks Enable or disable logging of health checks status updates
log-format Specifies the log format string to use for traffic logs
Health Check
Keywords Description Remarks
option httpchk <method> <uri> <version> Enable HTTP protocol to check on the servers health
Statistics
Keywords Description DF/FE/LI/BE Remarks
stats enable Enable statistics reporting with default settings O/O/O/O
stats admin { if | unless } <cond> Enable statistics admin level if/unless a condition is matched X/O/O/O
stats auth <user>:<passwd> Enable statistics with authentication and grant access to an account O/O/O/O

Bind Options

Options Description Remarks
ca-ignore-err Sets a comma separated list of errorIDs to ignore during verify at depth > 0
crt-ignore-err Sets a comma separated list of errorIDs to ignore during verify at depth == 0
no-sslv3 Disables support for SSLv3 on any sockets instantiated from the listener when SSL is supported
ssl-min-ver Enforces use of <version> or upper on SSL connections instantiated

from this listener

TLSv1.0 | TLSv1.1 | TLSv1.2 | TLSv1.3

Server Options

Options Description Remarks
check Enables health checks on the server.
inter <delay> Sets the interval between two consecutive health checks to <delay> milliseconds. default: 2000ms
check-ssl Forces encryption of all health checks over SSL, regardless of whether the server uses SSL or not for the normal traffic.
ssl Enables SSL ciphering on outgoing connections to the server.
verify [none|required]

ACL

acl <aclname> <criterion> [flags] [operator] [<value>] ...

acl <aclname> <sample fetch method> [flags] [operator] [<value>] ...
Element Description Remarks
ACL Name (aclname) [-_.:A-Za-z0-9]*
Flags (flags) -i, -f, -m, -n, -M, -u, --

Sample Fetch Methods

Method Type Description Remarks
src ip This is the source IPv4 address of the client of the session
Method Type Description Remarks
req.payload_lv binary Extracts a binary block whose size is specified at <offset1> for <length> bytes
res.payload_lv binary extracts a binary block whose size is specified at <offset1> for <length> bytes, and which starts at <offset2> if specified or just after the length in the response buffer.
req.ssl_hello_type integer Returns an integer value containing the type of the SSL hello message found in the request buffer if the buffer contains data that parse as a complete SSL (v3 or superior) client hello message.
res.ssl_hello_type integer Returns an integer value containing the type of the SSL hello message found in the response buffer if the buffer contains data that parses as a complete SSL (v3 or superior) hello message
Method Type Description Remarks
req.hdr([<name>[,<occ>]]) string Extracts the last occurrence of header <name> in an HTTP request When used from an ACL, all occurrences are iterated over until a match is found
res.hdr([<name>[,<occ>]]) string Extracts the last occurrence of header <name> in an HTTP response.
hdr([<name>[,<occ>]]) string Equivalent to req.hdr() when used on requests, and to res.hdr() when used on responses
http_auth(<userlist>) boolean Returns a boolean indicating whether the authentication data received from the client match a username & password stored in the specified userlist.
http_auth_group(<userlist>) string Returns a string corresponding to the user name found in the authentication data received from the client if both the user name and password are valid according to the specified userlist. ACL derivatives

Readings

ACL

Proxying

SSL

WebSockets

HTTP/2

Load Balancing

Logging and Statistics

Health Check

Throttling

Samples

Typical Sample for Reverse Proxy

  1. # References
  2. #   http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#3
  3. #   http://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4
  4. #   https://kvz.io/blog/2010/08/11/haproxy-logging/
  5.  
  6. global
  7.   daemon
  8.   log /var/lib/haproxy/dev/log local0 info
  9.   log /var/lib/haproxy/dev/log local1 notice
  10.  
  11.   ca-base /etc/haproxy/ssl/trusted
  12.   crt-base /etc/haproxy/ssl
  13.  
  14.   maxconn 2048
  15.   tune.ssl.default-dh-param 2048
  16.  
  17.   #https://disablessl3.com/
  18.   ssl-default-bind-options no-sslv3
  19.  
  20. defaults
  21.   mode http
  22.   log global
  23.  
  24.   option forwardfor
  25.  
  26.   timeout connect 5s
  27.   timeout client 10s
  28.   timeout server 10s
  29.   timeout tunnel 600s
  30.   timeout server-fin 10s
  31.   timeout client-fin 10s
  32.   timeout http-request 5s
  33.   timeout http-keep-alive 2s
  34.  
  35.   errorfile 400 /etc/haproxy/errors/400.http
  36.   errorfile 403 /etc/haproxy/errors/403.http
  37.   errorfile 408 /etc/haproxy/errors/408.http
  38.   errorfile 500 /etc/haproxy/errors/500.http
  39.   errorfile 502 /etc/haproxy/errors/502.http
  40.   errorfile 503 /etc/haproxy/errors/503.http
  41.   errorfile 504 /etc/haproxy/errors/504.http
  42.  
  43. listen stats
  44.   bind 192.168.1.11:8070
  45.  
  46.   stats enable
  47.   stats hide-version
  48.   stats realm HAProxy\ Statistics
  49.   stats uri /stats
  50.  
  51. frontend http
  52.   bind 192.168.1.11:8080
  53.  
  54.   option httplog
  55.   option dontlognull
  56.   log-format %ci\ [%t]\ %ft\ %b\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %ts\ %sq/%bq\ %{+Q}r
  57.  
  58.   http-request set-header X-Haproxy-Current-Date %T
  59.   http-request set-header X-Forwarded-Proto http
  60.   default_backend http-rear
  61.  
  62.  
  63. frontend https
  64.   bind 192.168.1.11:8443 ssl crt server-tls.pem ca-file ca.crt
  65.   # bind 192.168.1.11:8443 ssl crt server-tls.pem ca-file ca.crt verify optional crt-ignore-err all ca-ignore-err all   # For mutual auth
  66.  
  67.   option httplog
  68.   option dontlognull
  69.   log-format %ci\ %ft\ %b\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %ts\ %sq/%bq\ %{+Q}r
  70.  
  71.   #http-request deny deny_status 400 unless { ssl_fc }
  72.   #http-request deny deny_status 400 unless { ssl_c_used }
  73.   #http-request deny deny_status 400 unless { ssl_c_verify 0 }
  74.  
  75.   # setup request header for logging or backend usage
  76.   # https://www.haproxy.com/blog/ssl-client-certificate-information-in-http-headers-and-logs/
  77.   # https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4.2-http-request
  78.   http-request set-header X-Haproxy-Current-Date %T
  79.   http-request set-header X-SSL                  %[ssl_fc]
  80.   http-request set-header X-SSL-Session_ID       %[ssl_fc_session_id,hex]
  81.   http-request set-header X-SSL-Client-Verify    %[ssl_c_verify]    
  82.   http-request set-header X-SSL-Client-DN        %{+Q}[ssl_c_s_dn]
  83.   http-request set-header X-SSL-Client-CN        %{+Q}[ssl_c_s_dn(cn)]
  84.   http-request set-header X-SSL-Issuer           %{+Q}[ssl_c_i_dn]
  85.   http-request set-header X-SSL-Client-NotBefore %{+Q}[ssl_c_notbefore]
  86.   http-request set-header X-SSL-Client-NotAfter  %{+Q}[ssl_c_notafter]
  87.   http-request set-header X-Forwarded-Proto https
  88.  
  89.   default_backend http-rear
  90.  
  91.  
  92. frontend ws
  93.   bind 192.168.1.11:8090
  94.  
  95.   acl is_connection_upgrade hdr(Connection) -i upgrade
  96.   acl is_upgrade_websocket hdr(Upgrade) -i websocket
  97.   acl has_websocket_key hdr_cnt(Sec-WebSocket-Key) eq 1
  98.   acl has_websocket_ver hdr_cnt(Sec-WebSocket-Version) eq 1
  99.  
  100.   http-request deny unless is_connection_upgrade is_upgrade_websocket has_websocket_key has_websocket_ver
  101.  
  102.   default_backend ws-rear
  103.  
  104.  
  105. frontend wss
  106.   bind 192.168.1.11:8453 ssl crt server-tls.pem ca-file ca.crt
  107.   # bind 192.168.1.11:8453 ssl crt server-tls.pem ca-file ca.crt verify optional crt-ignore-err all ca-ignore-err all   # For mutual auth
  108.  
  109.   #http-request deny deny_status 400 unless { ssl_fc }
  110.   #http-request deny deny_status 400 unless { ssl_c_used }
  111.   #http-request deny deny_status 400 unless { ssl_c_verify 0 }
  112.  
  113.   acl is_connection_upgrade hdr(Connection) -i upgrade
  114.   acl is_upgrade_websocket hdr(Upgrade) -i websocket
  115.   acl has_websocket_key hdr_cnt(Sec-WebSocket-Key) eq 1
  116.   acl has_websocket_ver hdr_cnt(Sec-WebSocket-Version) eq 1
  117.  
  118.   http-request deny unless is_connection_upgrade is_upgrade_websocket has_websocket_key has_websocket_ver
  119.  
  120.   default_backend ws-rear
  121.  
  122. backend http-rear
  123.   server s1 127.0.0.1:80 check
  124.  
  125. backend ws-rear
  126.   server s1 127.0.0.1:90 check

Typical Sample for Forward Proxy

  1. # Sample HAProxy configuration for dedicated forward proxy
  2.  
  3. # References
  4. #    https://stackoverflow.com/questions/49433417/setup-https-forward-proxy-with-haproxy
  5. #    https://serverfault.com/questions/477642/using-haproxy-for-transparent-forwarding-and-selective-redirection
  6.  
  7. global
  8.   daemon
  9.   log /var/lib/haproxy/dev/log local0 info
  10.   log /var/lib/haproxy/dev/log local1 notice
  11.  
  12.   ca-base /etc/haproxy/ssl/trusted
  13.   crt-base /etc/haproxy/ssl
  14.  
  15.   maxconn 2048
  16.   tune.ssl.default-dh-param 2048
  17.   ssl-default-bind-options no-sslv3
  18.  
  19. userlist users
  20.   group regular-users
  21.  
  22.   user tom password ... groups regular-users
  23.  
  24. defaults
  25.   mode http
  26.   log global
  27.  
  28.   timeout connect 5s
  29.   timeout client 10s
  30.   timeout server 10s
  31.   timeout tunnel 600s
  32.   timeout server-fin 10s
  33.   timeout client-fin 10s
  34.   timeout http-request 5s
  35.   timeout http-keep-alive 2s
  36.  
  37.   errorfile 400 /etc/haproxy/errors/400.http
  38.   errorfile 403 /etc/haproxy/errors/403.http
  39.   errorfile 408 /etc/haproxy/errors/408.http
  40.   errorfile 500 /etc/haproxy/errors/500.http
  41.   errorfile 502 /etc/haproxy/errors/502.http
  42.   errorfile 503 /etc/haproxy/errors/503.http
  43.   errorfile 504 /etc/haproxy/errors/504.http
  44.  
  45. listen stats
  46.   bind 192.168.1.11:8070
  47.  
  48.   stats enable
  49.   stats hide-version
  50.   stats realm HAProxy\ Statistics
  51.   stats uri /stats
  52.   stats auth proxyadmin:p8dp8d
  53.  
  54. resolvers dns
  55.   nameserver dns1 8.8.8.8
  56.   nameserver dns2 8.8.4.4
  57.   hold valid 10s
  58.  
  59. frontend http
  60.   bind 192.168.1.11:8080
  61.  
  62.   option http-use-proxy-header
  63.   option httplog
  64.   option dontlognull
  65.   log-format %ci\ [%t]\ %ft\ %b\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %ts\ %sq/%bq\ %{+Q}r
  66.  
  67.   acl is-allowed-users http_auth_groups(users) regular-users
  68.   http-request auth unless is-allowed-users   # 407 for unauthorized access
  69.  
  70.   default_backend outside
  71.  
  72. backend outside
  73.  
  74.   option httpclose
  75.   option http_proxy

Wireshark

References

Operator Symbol Description Remarks
eq == Equal
ne != Not Equal
gt > Greater Than
lt < Less Than
ge >= Greater than or Equal to
le <= Less than or Equal to
contains Does the protocol, field or slice contain a value
matches ~ Does the protocol or text string match the given case-insensitive Perl-compatible regex
[i:j] Slices with i = start_offset, j = length
[i-j] Slices with i = start_offset, j = end_offset, inclusive
[i] Slices with i = start_offset, length = 1
[:j] Slices with start_offset = 0, length = j
[i:] Slices with start_offset = i, end_offset = end_of_field
and && Logical AND
or || Logical OR
not ! Logical NOT
Protocol Typical Fields Description Remarks
tcp port Transmission Control Protocol
ip addr, dst, src Internet Protocol Version 4
http Hypertext Transfer Protocol
ssl Secure Sockets Layer
websocket WebSocket

Readings

Tips and Tricks

Typical display filters

ip.src == 192.168.1.31 and ip.addr == 203.252.150.28 and http

tcpdump

tcpdump [ -AbdDefhHIJKlLnNOpqStuUvxX# ] [ -B buffer_size ] 
 
         [ -c count ] [ -C file_size ] 
         [ -E spi@ipaddr algo:secret,... ] 
         [ -F file ] [ -G rotate_seconds ] [ -i interface ] 
         [ --immediate-mode ] [ -j tstamp_type ] [ -m module ] 
         [ -M secret ] [ --number ] [ --print ] [ -Q in|out|inout ] 
         [ -r file ] [ -s snaplen ] [ -T type ] [ --version ] 
         [ -V file ] [ -w file ] [ -W filecount ] [ -y datalinktype ] 
         [ -z postrotate-command ] [ -Z user ] 
         [ --time-stamp-precision=tstamp_precision ] 
         [ expression ]

Options

Option Description Remarks
-n Don't convert addresses
-i interface Listen on interface
-A Print each packet (minus its link level header) in ASCII Handy for capturing web pages
-s n Snarf n bytes of data from each packet rather than the default of 262144 bytes Setting n to 0 sets it to the default of 262144
-x When parsing and printing, in addition to printing the headers of each packet, print the data of each packet (minus its link level header) in hex

Filter Expression

Primitive Description Remarks
host host True if either the IPv4/v6 source or destination of the packet is host
dst host host True if the IPv4/v6 destination field of the packet is host
src host host True if the IPv4/v6 source field of the packet is host
port port True if either the source or destination port of the packet is port.
dst port port True if the packet is ip/tcp, ip/udp, ip6/tcp or ip6/udp and has a destination port value of port.
src port port True if the packet has a source port value of port.
ip proto protocol True if the packet is an IPv4 packet of protocol type protocol.
tcp = ip proto tcp