(4 intermediate revisions by the same user not shown) | |||
Line 563: | Line 563: | ||
| || Not || <tt>not</tt> || || |
| || Not || <tt>not</tt> || || |
||
|- |
|- |
||
− | | Misc || In || <tt>in</tt> || || |
+ | | Misc || In || <tt>in</tt> || || <tt><nowiki>if _port.protocol in ['http', 'https']</nowiki></tt> |
− | |- |
+ | |- |
| || Is || <tt>is</tt> || Perform a test || |
| || Is || <tt>is</tt> || Perform a test || |
||
|- |
|- |
||
Line 639: | Line 639: | ||
|- |
|- |
||
| [http://jmespath.org/specification.html#or-expressions Or Expression] || <tt>''expr1'' <nowiki>||</nowiki> ''expr2''</tt> |
| [http://jmespath.org/specification.html#or-expressions Or Expression] || <tt>''expr1'' <nowiki>||</nowiki> ''expr2''</tt> |
||
− | | evaluate to either the left expression or the right expression || |
+ | | evaluate to either the left expression or the right expression || <tt><nowiki>[validators, trackers][] || `[]`</nowiki></tt> |
|- |
|- |
||
| [http://jmespath.org/specification.html#and-expressions And Expression] || <tt>''expr1'' && ''expr2''</tt> |
| [http://jmespath.org/specification.html#and-expressions And Expression] || <tt>''expr1'' && ''expr2''</tt> |
||
Line 677: | Line 677: | ||
"{:,d}".format(node|json_query('uptime || `0`')) |
"{:,d}".format(node|json_query('uptime || `0`')) |
||
+ | |||
+ | for node in ripple|d({})|json_query('[validators, trackers][] || `[]`') |
||
hostvars|json_query('*.prepare_validator_token.results[].stdout_lines[]') |
hostvars|json_query('*.prepare_validator_token.results[].stdout_lines[]') |
||
Line 683: | Line 685: | ||
rippled|json_query('config.ports[?name==`port_rpc`]|[0]') |
rippled|json_query('config.ports[?name==`port_rpc`]|[0]') |
||
+ | |||
+ | host|json_query('haproxy.rippleProxies[?backends[?@.servers[?@.name==`' ~ rippled.name ~ '`]]]')|first or {} |
||
</syntaxhighlight> |
</syntaxhighlight> |
||
Revision as of 07:34, 24 June 2019
- https://www.ansible.com/
- Desc. : can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates.
- License :
- Written in :
- Sources : https://github.com/ansible/ansible
References
Ansible 2.7 "In the Light"
- Inventory Parameters
- ansible_connection, ansible_host, ansible_port, ansible_user, ansible_ssh_pass, ansible_ssh_private_key_file, ...
- ansible_become_pass (ansible_sudo_pass, ansible_su_pass)
Ansible 2.6 "Heartbreaker"
Ansible 2.4
- Ansible 2.4 Documentation
- Inventory : Hosts and Groups
- Inventory Parameters
- Configuration File :
ansible.cfg
- Module Index
- Directives Glossary
- Playbooks
- Variables
- Magic Variables(Built-in Variables)
hostvars, group_names, groups, ansible_play_hosts, ansible_play_batch, inventory_dir, inventory_file, playbook_dir, ansible_check_mode
- Templating (Jinja2)
- Filters
- Loops
- Return Values
- Lookups
- Vault
- Fact Caching
- Asynchronous Actions and Polling
Commands
ansible-playbook
Option | Description | Remarks |
---|---|---|
--list-hosts | ||
--list-tasks | ||
--list-tags | ||
--syntax-check | ||
--connection=CONNECTION, -c CONNECTION | ||
--limit=SUBSET, -l SUBSET | ||
--tags=TAGS, -t TAGS | ||
--ask-pass, -k |
Ansible Container
Concepts
playbook = play+
play = (host+, task+)
task = (name, module, ...)
Directives
Scope | Directive | Description | Remarks |
---|---|---|---|
Task | environment | A dictionary that gets converted into environment vars to be provided for the task upon execution. | |
when | |||
become | |||
with_items | |||
with_subelements | Subelements walks a list of hashes (aka dictionaries) and then traverses a list with a given (nested sub-)key inside of those records. | ||
run_once | Boolean that will bypass the host loop, forcing the task to execute on the first host available and will also apply any facts to all active hosts. | ||
delegate_to | |||
register | |||
changed_when | |||
failed_when |
Modules
Module | Description | Remarks |
---|---|---|
debug
|
Print statements during execution | |
assert
|
Asserts given expressions are true | |
fail
|
Fail with custom message | |
pause
|
Pause playbook execution | |
stat
|
Retrieve file or file system status | |
file
|
Sets attributes of files | |
copy
|
Copies files to remote locations | |
archive
|
Creates a compressed archive of one or more files or trees | zip, tar cvf, tar cvzf |
unarchive
|
Unpacks an archive after (optionally) copying it from the local machine | unzip, tar xvf, tar xvzf |
lineinfile
|
Ensure a particular line is in a file, or replace an existing line using a back-referenced regular expression | |
replace
|
Replace all instances of a particular string in a file using a back-referenced regular expression | |
template
|
Templates a file out to a remote server | |
shell
|
Execute commands in nodes | /bin/sh , /bin/bash
|
command
|
Executes a command on a remote node | |
expect
|
Executes a command and responds to prompts | |
get_url
|
Downloads files from HTTP, HTTPS, or FTP to node | curl, wget |
uri
|
Interacts with HTTP and HTTPS web services and supports Digest, Basic and WSSE HTTP authentication mechanisms | curl, weg |
apt |
Manages apt-packages | |
apt_repository |
Add or remove an APT repositories in Ubuntu and Debian | |
apt_key |
Add or remove an apt key, optionally downloading it | |
debconf |
Configure a .deb package using debconf-set-selections. | |
yum |
Manages packages with the yum package manager | |
yum_repository |
Add or remove YUM repositories | |
rpm_key |
Adds or removes a gpg key from the rpm db | |
pip
|
Manages Python library dependencies | |
systemd
|
Controls systemd services on remote hosts | systemd provides a system and service manager that runs as PID 1 and starts the rest of the system.
|
cron
|
Manage cron.d and crontab entries | |
docker_container
|
Manage the life cycle of docker containers | |
docker_image
|
Manage docker images | |
git
|
Deploy software (or files) from git checkouts | |
mysql_variables
|
Manage MySQL global variables (Query / Set MySQL variables) | |
mysql_db
|
Add or remove MySQL databases from a remote host | |
mysql_user
|
Adds or removes a user from a MySQL database |
Variables
- List of Behavioral Inventory Parameters
- Magic Variables, and How To Access Information About Other Hosts
Variable | Description | Remarks |
---|---|---|
ansible_connection | Connection type to the host. | ssh, smart, local, ... |
ansible_host | The name of the host to connect to | |
ansible_user | The default ssh user name to use. |
Magic Variables
Variable | Description | Remarks |
---|---|---|
hostvars | ||
group_names | a list (array) of all the groups the current host is in | |
groups | a list of all the groups (and hosts) in the inventory | |
inventory_hostname | the name of the hostname as configured in Ansible’s inventory host file | |
ansible_play_hosts | the full list of all hosts still active in the current play | |
ansible_play_batch | a list of hostnames that are in scope for the current ‘batch’ of the play | |
playbook_dir | the playbook base directory | |
inventory_dir | the pathname of the directory holding Ansible’s inventory host file | |
inventory_file | the pathname and the filename pointing to the Ansible’s inventory host file |
Facts
Category | Fact | Description | Values | Remarks |
---|---|---|---|---|
Common | ansible_architecture
|
"x86_64" | ||
ansible_distribution
|
"Ubuntu" | |||
ansible_distribution_version
|
"16.04" | |||
ansible_distribution_major_version
|
"16" | |||
ansible_distribution_release
|
"xenial" | |||
ansible_hostname
|
hostname | |||
ansible_env
|
ansible_env.HOME, ansible_env.PATH, ansible_env.PWD
| |||
ansible_interfaces
|
["lo", "eth0", "eth1", "eth2", "eth3", "docker0", ...] | Array | ||
ansible_os_family
|
"Debian" | |||
ansible_pkg_mgr
|
"apt" | |||
ansible_check_mode
|
false
|
|||
inventory_hostname
|
name of the host as configured in Ansible's inventory host file (usually hosts.yml )
|
"m001", "m002" | not always same with ansible_hostname
|
Filters
Filter | Description | Applied to | Syntax | Remarks |
---|---|---|---|---|
json_query
|
query a complex JSON structure and iterate over it using a loop structure | an object | JMESPath Specification | |
regex_replace
|
replace text in a string with regex | string | Regex in Python 3 | |
regex_search
|
search a string with a regex | string | ||
select
|
Filters a sequence of objects by applying a test to each object, and only selecting the objects with the test succeeding. | a sequence of objects | ||
selectattr
|
Filters a sequence of objects by applying a test to the specified attribute of each object, and only selecting the objects with the test succeeding. | a sequence of objects | selectattr(attrName, test, value) (test : equalto, ne, gt, ge, number, odd, ...) |
|
reject
|
Filters a sequence of objects by applying a test to each object, and rejecting the objects with the test succeeding. | a sequence of objects | ||
rejectattr
|
Filters a sequence of objects by applying a test to the specified attribute of each object, and rejecting the objects with the test succeeding. | a sequence of objects | ||
map
|
Applies a filter on a sequence of objects or looks up an attribute. This is useful when dealing with lists of objects but you are really only interested in a certain value of it. | a sequence of objects | map(attribute="attrName") map(filterName, [arg1, arg2, arg3, ...)
|
|
attr
|
Get an attribute of an object. | an object | ||
list
|
Convert the value into a list. | |||
flatten | Flatten a list | a sequence of objects | flatten([levels=n])
|
|
sort
|
Sort an iterable | iterable | sort(reverse=False, case_sensitive=False, attribute=None)
|
|
default , d
|
Return predefined value for a undefined variable or a variable whose value is evaluated to false | default(default_value=u'', boolean=False) | ||
combine | allows hashes to be merged | hash, dictionary | dict1|combine(combine(dict2, recursive=True) | |
to_json, to_nice_json | ||||
to_yaml, to_nice_yaml | ||||
mandatory |
common.quorum.download.constellation|regex_replace('^.*/', '')
configDefault.outputs|selectattr("type", "eq", "logstash")|first
['ripple']|map('extract', common, ['tls', 'dn', 'c'])|list|first|default('KO')
Tests
Test | Description | Applied to | Syntax | Remarks |
---|---|---|---|---|
directory, file, link, abs, mount | provide information about a path on the controller | NOT for remote path | ||
failed, succeeded, changed, skipped | check the status of tasks | |||
match, search | match strings against a substring or a regex | string | ||
version | compare a version number | string | version(compared_version, operator) | <, lt, <=, le, >, gt, >=, ge, ==, =, eq, !=, <>, ne |
Plugins
Category | Plugin | Description | Remarks |
---|---|---|---|
Lookup | fileglob | Matches all files in a single directory, non-recursively, that match a pattern |
Jinja2
- Core Concepts
- Variable
- Filter (var|filter1|filter2|...|filtern)
- Test
- Expression
- Statement
Initialization Parameters
- Jinja2 High Level API
- lists initialization parameters
Parameter | Description | Default | Remarks |
---|---|---|---|
trim_blocks | If this is set to True the first newline after a block is removed (block, not variable tag!) | False | |
lstrip_blocks | If this is set to True leading spaces and tabs are stripped from the start of a line to a block. | False | |
keep_trailing_newline | Preserve the trailing newline when rendering templates. | False | |
newline_sequence | The sequence that starts a newline. Must be one of '\r', '\n' or '\r\n'. | '\n' |
Operators
Category | Name | Operator | Description | Remarks |
---|---|---|---|---|
Math | Addition | + | ||
Subtraction | - | |||
Multiplication | * | |||
Division | / | |||
Division to truncated integer | // | 20 // 7 == 2 | ||
Comparison | Equal to | == | ||
Inequal to | != | |||
Greater than | > | |||
Lower than | < | |||
Logic | And | and | ||
Or | or | |||
Not | not | |||
Misc | In | in | if _port.protocol in ['http', 'https'] | |
Is | is | Perform a test | ||
Filter | | | Apply a filter | ||
String concatenation | ~ |
Readings
- inline
if
expressions - Jinja2
selectattr( )
Filterselectattr('email', 'undefined'), selectattr("type", "equalto", "floating")
- Enable jinja2 line statements (2012-09-29)
- How do you sort a list in Jinja2? (Dec 24 '09)
- list_of_tuples|sort(attribute='0')
- Ansible template adds 'u' to array in template (Jan 7 '17)
- Use {{ value | to_json }}
- Jinja2 template variable if None Object set a default value (Oct 27 '13)
- For Jinja, undefined is different from none.
var|default('', true)
var or ''
var or {}
var or []
JMESPath
Expressions
Expression | Syntax | Description | Remarks |
---|---|---|---|
Sub Expression | expr1.expr2, expr1.* | definitions.simpleTypes | |
Index Expression | expr[n] | properties[0], properties[3], properties[-3] | |
Slice Expression | expr[i:j:k] | properties[0:-1:2] | |
Hash Wildcard Expression | * | Return a list of the hash element’s values. Any subsequent expression will be evaluated against each individual element in the list | |
List Wildcard Expression | [*] | Return all the elements in a list. Any subsequent expressions will be evaluated against each individual element. | |
Multi-Select List Expression | [ expr1, expr2, expr3 ... ] | Extract a subset of elements as a list from a JSON hash | |
Multi-Select Hash Expression | [ key1: expr1, key2: expr2, ...] | Extract a subset of elements as a hash from a JSON hash | |
Filter Expression | [? boolean expr ] | Select JSON elements based on a comparison to another expression for each element in an array | |
Flatten Operator | [] | Merge sublists in the current result into a single list. | Filter first and flatten last |
Pipe Expressions | expr1 | expr2 | combines two expressions, separated by the | character | |
Or Expression | expr1 || expr2 | evaluate to either the left expression or the right expression | [validators, trackers][] || `[]` |
And Expression | expr1 && expr2 | evaluate to either the left expression or the right expression | |
Not Expression | ! expr | negates the result of an expression | |
Literal Expressions | `json-value` | allows arbitrary JSON objects to be specified | escape ` with leading \ : \` |
Operators
Operator | Name | Remarks |
---|---|---|
== | Equality | |
!= | Inequality | |
< | Less than | |
<= | Less than or equal to | |
> | Greater than | |
>= | Greater than or equal to |
Examples
hostvars.*.ripple.[validators, trackers][][].name
common|json_query('haproxy.systemd.LimitNOFILE' || `20000`)
"{:,d}".format(node|json_query('uptime || `0`'))
for node in ripple|d({})|json_query('[validators, trackers][] || `[]`')
hostvars|json_query('*.prepare_validator_token.results[].stdout_lines[]')
network[*].{group: `nodes`, data: {id: name, name: name}}
rippled|json_query('config.ports[?name==`port_rpc`]|[0]')
host|json_query('haproxy.rippleProxies[?backends[?@.servers[?@.name==`' ~ rippled.name ~ '`]]]')|first or {}
Roles
Role | Location | Remarks |
---|---|---|
apparmor | https://github.com/Oefenweb/ansible-apparmor | Remove apparmor in Debian-like systems. |
Readings
- Ansible Directory Layout Best Practice
- Ansible doesn't load .profile and .bashrc files (21 Apr 2016)
- This is expected and by design, Ansible is a batch system and avoid interactive logins if possible, which means it won't source certain user/shell specific files.
- Using Ansible's command and shell modules properly (21 Sep 2016)
changed_when
,when
- File lookup() relative to playbook (Aug 27 '14)
- List of Ansible default variables
- Running Python script via ansible (Feb 1 '16)
- Ansible: copy template only when destination file does not exist (Jul 5 '14)
force: false
- Working with date and timestamp in Ansible (July 27, 2017)
- When command/shell outputs JSON, its result should be immediately parseable :
from_json
filter - Examples of using Blocks in Ansible 2 (March 8, 2016)
- "failed" attribute of a task result works strangely in conditionals (9 Dec 2015)
when: result|failed, when: result.failed, or when: result.failed == True
- ansible run command on remote host in background (Sep 6 '16)
- Using
async
andpoll
directives
- Using
- register is executed on skipped tasks (11 Sep 2016)
- This is expected...
- Ansible Jinja Warrior - Mastering "Loop Variable Scope" (16 Feb 2017)
{% do colour.update({'people_count':colour_count}) %}
- How to add a new entry into a dictionary object while using jinja2? (Apr 27 '16)
- {% do data.update({'location': node.location}) %}
- ansible run command on remote host in background (Sep 6 '16)
- When
nohup
is not working at all with Ansible - Use
async
andpoll
directives
- When
- Install .deb Packages in Ansible (Mar 4, 2015)
- get_url - counterintuitive that it re-downloads if "dest" is a directory
- Copy local file if exists, using ansible (Mar 4 '15)
- Use 'is_file' filter or 'fileglob' for local files, but use proceeding 'stat' task for remote files
Templating
Filters / Tests
- How to filter, join and map lists in Ansible (31 December 2017)
- Advanced list operations in Ansible (16 March 2018)
- Ansible: filter a list by its attributes (Aug 8 '15)
network.addresses.private_man | selectattr("type", "match", "^fixed$") | map(attribute='addr') | list
- How to convert a dictionary of dictionaries into a list of dictionaries in a Ansible vars file? (May 17 '16)
hostvars.values()|list
- Looping through dictionaries in jinja2 templates (5 Nov 2015)
- Ansible/Jinja2 - Map nested key in list (Aug 14 '17)
sum(start=[])
- Cannot use to_json on hostvars (18 Feb 2016)
Collection data-type
- List and Dict Comprehensions in Python (21 February 2017)
- Python Mapping Types —
dict
- Python Sequence Types —
list
,tuple
,range
String
Formatting
- Python 2.7 Format Specification Mini-Language : {:[[fill]align][sign][#][0][width][,][.precision][type]}
- Python 3.7 Format Specification Mini-Language
Security
- How Ansible Vault Works (NOVEMBER 24, 2017)
- Ansible: How to encrypt some variables in an inventory file in a separate vault file? (May 13 '15)
ansible-vault
- Securing playbooks with ansible-vault (15 januari 2017)
- Encrypting the Ansible Vault passphrase using GPG (Dec 14, 2016)
- Generating self-signed OpenSSL certs with Ansible 2.4's crypto modules (October 31, 2017)
Modules
- copy module contents parameter does not allow for newline chars (20 Feb 2014)
content: " {{ item | to_nice_json(indent=2) }}"
- ansible extract without first directory (Feb 20 '17)
extra_opts: [--strip-components=1]
- How to use ansible 'expect' module for multiple different responses? (Jul 15 '16)
Examples
Tips and Tricks
Print out variables and expressions using Ansible console and debug module
Get into console using ansible-console
command and then use debug
task
$ ansible-console -l m001
...
sshuser@all (1)[f:5]$ debug msg="{{ hostvars }}"
...
sshuser@all (1)[f:5]$ exit
$
Executing simple modules using ansible command without any playbook
$ ansible m001 -m ping
$ ansible m001 -m command -a 'cat /etc/environment'
$ ansible m003 -i inventories/product2/hosts.yml -c local -m debug -a 'msg="{{ ansible_connection }}"'
Executing play on localhost instead of remote machine
- Need more comparison with dry run using -C or --check option
- You can change ansible_user into local user when executing playbook with '-c local' option.
$ ansible-playbook -i inventories/prd/hosts.yml -t setupRippleNode -l p001 \
-c local --flush-cache -e "ansible_user=tom" plays/prepare-hosts.yml
Using Jinja statements inside play
- name: Start constellation nodes
vars:
tls: "{{ item.constellation.tls|default(common.quorum.constellation.tls) }}"
protocol: >-2
{%- if tls == 'strict' -%}'http'{%- else -%}'http'{%- endif -%}
shell: |
...
To build more complex object as an task scope variable, build script variable using statements then print out the script variable using expression at the last line
- name: Generate validators file ('validators.txt')
vars:
pubKeys: |
{%- set pubKeys = [] -%}
{%- for lines in hostvars|json_query('*.extract_validator_public_key.results[].stdout_lines')|sort(attribute='0') -%}
{%- if lines[0] in item.peers -%}
{%- do pubKeys.append(lines[1]) -%}
{%- endif -%}
{%- endfor -%}
{{ pubKeys }}
template:
...
Safe escaping undefined variable
Use default
filter.
when: not fabric.generate.crypto.skip|default(false)
with_items: favorites.movies|default([])
{% for member in members|default([]) %}
{%- for message in (member.mailbox|default({})).messages|default([]) -%}
...
{%- endfor -%}
{% endfor %}
Safe drill-down using json_query
filter
If you want access deep descendant node from a safe node without escaping every step using default
filter, you can use json_query
filter which utilizes JMESPath internally.
- name: Generate CSR for TLS certificate if not exists
openssl_csr:
path: "{{ ansible_env.HOME }}/ripple/nodes/{{ item.name }}/tls/tls-server.csr"
privatekey_path: "{{ ansible_env.HOME }}/ripple/nodes/{{ item.name }}/tls/tls-server.key"
state: present
force: false
mode: 0600
country_name: "{{ item|json_query('tls.dn.countryName')|default(common.ripple.tls.baseName.countryName, true) }}"
state_or_province_name: "{{ item|json_query('tls.dn.stateOrProvinceName')|default(common.ripple.tls.baseName.stateOrProvinceName, true) }}"
locality_name: "{{ item|json_query('tls.dn.localityName')|default(common.ripple.tls.baseName.localityName, true) }}"
organization_name: "{{ item|json_query('tls.dn.organizationName')|default(common.ripple.tls.baseName.organizationName, true) }}"
organizational_unit_name: "{{ item|json_query('tls.dn.organizationalUnitName')|default(common.ripple.tls.baseName.organizationalUnitName, true) }}"
email_address: "{{ item|json_query('tls.dn.emailAddress')|default(common.ripple.tls.baseName.emailAddress, true) }}"
common_name: "{{ item|json_query('tls.dn.commonName')|default(item.name, true) }}"
when: true
become: false
with_items: "{{ (ripple|default({})).validators|default([]) }}"
register: generate_validator_tls_csr
tags: ['tls']
When using multi-select-list
expression of JMESPath, an item in the resulted list can be null or nested. So, it should be filtered using select
and flattened
.
- name: Backup and clean artifacts including data, configuration, and logs
var:
timestamp: "{{ lookup('pipe', 'date date +%Y%m%d'T'%H%M%Z -u') }}"
file:
src: "{{ ansible_env.HOME }}/ripple/nodes/{{ item.name }}/"
dest: "{{ ansible_env.HOME }}/ripple/nodes/{{ item.name }}_{{ timestamp }}/"
when: common.ripple.flags.cleans
with_items: "{{ ripple|default({})|json_query('[validators, trackers]')|select|flatten }}"
- References
Filtering and drilling down the entire hostvars
using selectattr
and map
filters
With 4 host variable files under host_vars
directory
host_vars/m001.yml
quorum:
nodes:
- name: node1
host: "{{ inventory_hostname }}"
type: permissioned
- name: node2
host: "{{ inventory_hostname }}"
type: permissioned
host_vars/m002yml
quorum:
nodes:
- name: node3
host: "{{ inventory_hostname }}"
type: permissioned
host_vars/m003yml
quorum:
portal:
host_vars/m004yml
grafana:
The following task would print out only 3 quorum nodes
- name: Print out Quorum nodes
debug:
msg: "{{ item }}"
with_items: "{{ hostvars.values()|selectattr('quorum', 'defined')|map(attribute='quorum')|selectattr('nodes', 'defined')|map(attribute='nodes')|sum(start=[])|list }}"
# when: (hostvars[item].quorum|default({})).nodes
when: true
Filtering and drilling down the entire hostvars
using json_query
filter
You can navigate the entire hostvars in null safe way more conveniently using json_query filter.
- name: Print out Quorum nodes
debug:
msg: "{{ hostvars|json_query('*.quorum.nodes')|flatten|sort(attribute='name')|reverse|list }}"
when: true
Or you can use json_query more aggressively
- name: Print out Quorum nodes
debug:
msg: "{{ hostvars|json_query('*.quorum.nodes[]')|sort(attribute='name')|reverse|list }}"
when: true
Even more aggressively (may not work)
- name: Print out Quorum nodes
debug:
msg: "{{ hostvars|json_query('*.quorum.nodes[] | sort_by([], &name)')|reverse|list }}"
when: true
Another example
- name: Print out Quorum nodes
debug:
msg: "{{ hostvars|json_query('*.ripple.validators[?name!=`v0`][]')|sort(attribute='name')|reverse|list|json_query('[*].config.ports[?name==`port_peer`][]') }}"
when: true
Filtering and drilling down current host variable using json_query
filter
{% for node in ripple|default({})|json_query('[validators, trackers][]') -%}
{%- set rpcPort = node|json_query('config.ports[?name==`port_rpc`][]')|first -%}
[[inputs.exec]]
name_override = "rippled"
commands = ["bash /etc/telegraf/telegraf_exec_cmd_rippled.sh {{ rpcPort.ip }} {{ rpcPort.port }}"]
interval = "10s"
timeout = "5s"
data_format = "json"
json_string_fields = ["build_ver", "complete_ledgers", "server_state", "base_log_level"]
[inputs.exec.tags]
type="ripple"
service = "{{ node.name }}"
{% endfor -%}
Fine tuning line breaks and indentations for Jinja2 templates
Default setting for line breaks and trimming should be specified at the top of the template file
#jinja2:line_statement_prefix: '%', trim_blocks: False, lstrip_blocks: True, keep_trailing_newline: True
For simple loop without other nested conditionals or nested loops, use {% for ... -%} ... {% endfor %}
#jinja2:line_statement_prefix: '%', trim_blocks: False, lstrip_blocks: True, keep_trailing_newline: True
[validators]
{% for key in pubKeys -%}
{{ key }}
{% endfor %}
Generating well indented and lined files
Generate files with trim_blocks
disabled.
Sample
#jinja2:line_statement_prefix: '%', trim_blocks: False, lstrip_blocks: True, keep_trailing_newline: True
{#
For, detailed help, refer https://github.com/ripple/rippled/blob/1.0.0/cfg/rippled-example.cfg
#}
# Configuration for Ripple node named {{ item.name }}
{# 1. Server #}
[server]
{% for port in item.config.ports -%}
{{ port.name }}
{% endfor -%}
{% for port in item.config.ports -%}
[{{ port.name }}]
port = {{ port.port }}
ip = {{ port.ip }}
{% if port.admin is defined %}{{ 'admin = ' ~ port.admin }}{% endif %}
protocol = {{ port.protocol }}
{% if port.ssl_key is defined %}{{ 'ssl_key = ' ~ port.ssl_key }}{% endif %}
{% if port.ssl_cert is defined %}{{ 'ssl_cert = ' ~ port.ssl_cert }}{% endif %}
{% endfor -%}
{%- if item.config.rpc_startup is defined %}
[rpc_startup]
{% for cmd in item.config.rpc_startup -%}
{{ cmd|to_json }}
{% endfor %}
{%- endif %}
{# [websocket_ping_frequency] #}
{# 2. Peer Protocol #}
{# [ips] #}
{# [ips_fixed] #}
[peer_private]
1
[peers_max]
{{ common.ripple.configDefault.peers_max }}
[node_size]
medium
# https://github.com/ripple/rippled/blob/1.0.0/cfg/rippled-example.cfg
- name: Generate validator configuration from template
template:
src: "{{ playbook_dir }}/../templates/rippled.cfg.j2"
dest: "{{ ansible_env.HOME }}/ripple/nodes/{{ item.name }}/rippled.cfg"
newline_sequence: '\n'
mode: "u=rw,g=r,o=r"
become: flase
with_items: "{{ (ripple|default({})).validators|default([]) }}"
tags: [genConfig]
Tips
Non-nested for without included if
- Use {% for ... -%}...{% endfor -%}.
- Add blank line before ending {% endfor -%} line, to add a single blank line after each iterated item.
[server]
{% for port in item.config.ports -%}
{{ port.name }}
{% endfor -%}
{% for node in ripple|default({})|json_query('[validators, trackers][]') -%}
{%- set rpcPort = node|json_query('config.ports[?name==`port_rpc`][]')|first -%}
[[inputs.exec]]
name_override = "rippled"
commands = ["bash /etc/telegraf/telegraf_exec_cmd_rippled.sh {{ rpcPort.ip }} {{ rpcPort.port }}"]
interval = "10s"
timeout = "5s"
data_format = "json"
json_string_fields = ["build_ver", "complete_ledgers", "server_state", "base_log_level"]
[inputs.exec.tags]
type="ripple"
service = "{{ node.name }}"
{% endfor -%}
Executing complex interactive only command using expect module
- name: Make MariaDB more secure using 'mysql_secure_installation'
expect:
command: mysql_secure_installation --defaults-file="/etc/mysql/conf.d/my_{{ item.name }}.cnf"
responses:
'Enter current password for root \(enter for none\): ': "\n"
'Set root password\? \[Y/n\]': 'n'
'Remove anonymous users\? \[Y/n\]': 'Y'
'Disallow root login remotely\? \[Y/n\]': 'Y'
'Remove test database and access to it\? \[Y/n\]': 'Y'
'Reload privilege tables now\? \[Y/n\]': 'Y'
timeout: 10
no_log: true
when: true
become: true
with_items: "{{ mariadb|json_query('[*]')|select|flatten }}"
tags: ['initMariaDB']
Using multiple vault passwords
To use multiple vault passwords to encrypt strings
- Define all the passwords with vault_identity_list in ansible.cfg
- Specify the label of the password to use with --encrypt-vault-id option when calling ansible-vault encrypt_string command.
Example ansible.cfg
[defaults]
inventory = hosts.yml
gathering = smart
fact_caching = jsonfile
fact_caching_connection = ./fact_cache
fact_caching_timeout = 60
retry_files_save_path = plays/retry
ask_sudo_pass = True
ask_vault_pass = False
vault_identity_list = dev@vault-pass-test.sh, test@vault-pass-test.sh
jinja2_extensions = jinja2.ext.do,jinja2.ext.i18n
; https://docs.ansible.com/ansible/latest/plugins/vars/host_group_vars.html
[yaml_valid_extensions]
defaults = [u'.yml', u'.yaml']
Example vault-pass-test.sh
#!/bin/sh
echo 5678
The commandline to use dev password (enclosing single quotation marks are not part of password)
$ ansible-vault encrypt_string --encrypt-vault-id dev 'abcd'
Nontrivial template samples using both Jinja and JMESPath extensively
#jinja2:line_statement_prefix: '%', trim_blocks: False, lstrip_blocks: False, keep_trailing_newline: True
{
{% set nodes = [] -%}
{%- for host in hostvars|json_query('*') -%}
{%- set rippleds = host|json_query('ripple.[validators, trackers][]')|d([], true) -%}
{%- for rippled in rippleds -%}
{%- set proxy = host|json_query('haproxy.rippleProxies[?backends[?@.servers[?@.name==`' ~ rippled.name ~ '`]]]')|first or {} -%}
{%- do rippled.update({'proxy': proxy}) -%}
{%- do nodes.append(rippled) -%}
{%- endfor -%}
{%- endfor -%}
"rippleds" : {{ nodes|to_nice_json(indent=2) }}
}