Working with Dynamic Inventory

1. Understanding Dynamic Inventory

Concept	Description	Usage
Dynamic Inventory	Inventory sources that are generated at runtime by external scripts or APIs	Used to dynamically fetch host information from cloud providers, databases, or service discovery tools
Static Inventory	Fixed list of hosts defined in an inventory file (e.g., `hosts.ini`)	Used for environments with stable host configurations

Example: Dynamic Inventory Script

#!/usr/bin/env python3
import json

# Simulated dynamic inventory from a cloud provider API
def get_inventory():
    return {
        "webservers": {
            "hosts": ["web1.example.com", "web2.example.com"],
            "vars": {
                "ansible_user": "ec2-user"
            }
        },
        "dbservers": {
            "hosts": ["db1.example.com"],
            "vars": {
                "ansible_user": "postgres"
            }
        }
    }

print(json.dumps(get_inventory()))

Note: Dynamic inventory scripts must be executable and output valid JSON.

Key Benefits of Dynamic Inventory

Automatically updates host lists based on external sources
Reduces manual maintenance of inventory files
Supports integration with cloud providers (AWS, GCP, Azure)

2. Using AWS EC2 Plugin

Option	Values	Description
plugin	`aws_ec2`	Specifies the AWS EC2 inventory plugin to use
regions	`['us-east-1', 'us-west-2']`	List of AWS regions to query for instances
filters	`{'tag:Environment': 'production'}`	EC2 instance filters to apply
hostnames	`['tag:Name', 'private-ip-address']`	List of hostname sources to use for inventory
vpc_destination	`true`	Use VPC destination addresses for instances

Example: Basic AWS EC2 Inventory Configuration

plugin: aws_ec2
regions:
  - us-east-1
  - us-west-2
filters:
  tag:Environment: production
hostnames:
  - tag:Name
  - private-ip-address

Example: Using AWS EC2 with Custom Tags

plugin: aws_ec2
regions:
  - eu-west-1
filters:
  tag:Application: web-server
  instance-state-name: running
hostnames:
  - tag:Name
  - private-ip-address

Note: Ensure that the AWS credentials are configured via AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables or via IAM roles if running on EC2.

Warning: The aws_ec2 plugin can be slow when querying large numbers of instances. Consider using filters to limit the scope.

3. Using Azure Plugin

Option	Values	Notes
plugin	`azure_rm`	Specifies the Azure Resource Manager inventory plugin
auth_source	`auto`, `cli`, `msi`, `env`, `credential_file`	Determines how to authenticate with Azure
use_azure_rm_cache	`true`, `false`	Enables caching of Azure resource data to improve performance
cache_path	`string`	Path to the cache file (default: `/tmp/azure_rm.cache`)
cache_timeout	`integer`	Cache timeout in seconds (default: 3600)

Example: Basic Azure Inventory Configuration

plugin: azure_rm
auth_source: cli
include_vm_resource_groups:
  - my-vm-resource-group

Example: Using Environment Variables for Authentication

plugin: azure_rm
auth_source: env

Note: Ensure that the Azure CLI or appropriate environment variables are configured for authentication before using this plugin.

4. Using GCP Plugin

Option	Values	Description
gcp_config_file	`~/.gcp/credentials.json`	Path to the GCP service account credentials file
project	`my-gcp-project`	GCP project ID to query for resources
filters	`["name:my-instance"]`	List of filters to apply when querying GCP resources
auth_kind	`serviceaccount`	Authentication method (e.g., `serviceaccount`, `application`)

Example: GCP Inventory Configuration

plugin: gcp_compute
projects:
  - my-gcp-project
filters:
  - name: my-instance
auth_kind: serviceaccount
gcp_config_file: ~/.gcp/credentials.json

Note: Ensure the GCP service account has compute.instances.list and compute.networks.list permissions.

Warning: The gcp_compute plugin requires the google-cloud-compute Python library to be installed on the Ansible control node.

5. Configuring Inventory Plugins

Property	Description	Default	Notes
plugin	Name of the inventory plugin to use	None	Required field
cache	Enable caching of inventory data	false	Improves performance for large inventories
cache_plugin	Cache plugin to use (e.g., 'redis', 'memcached')	None	Requires external cache backend
cache_timeout	Cache expiration time in seconds	3600	Default is 1 hour

Example: Using the AWS EC2 Inventory Plugin

plugin: amazon.aws.aws_ec2
regions:
  - us-east-1
  - us-west-2
filters:
  instance-state-name: running
  tag:Environment: production
cache: true
cache_plugin: redis
cache_timeout: 300

Example: Using the Azure Inventory Plugin

plugin: azure.azcollection.azure
auth_source: auto
group_by_resource_group: true
group_by_location: true
cache: true
cache_plugin: memcached
cache_timeout: 600

Note: Inventory plugins can significantly improve performance by caching results. Ensure that cache backends like Redis or Memcached are properly configured in your environment.

6. Setting Plugin Options

Option	Values	Description
plugin	`aws_ec2`, `gcp_compute`, `azure_rm`, `vmware`, `openstack`	Specifies the dynamic inventory plugin to use for fetching inventory data.
regions	`us-east-1`, `eu-west-1`	List of AWS regions to query. Defaults to all regions if not specified.
filters	`{"tag:Environment": "production"}`	Dictionary of filters to apply when querying cloud resources.
hostnames	`["tag:Name", "private-ip-address"]`	List of hostname types to use for resolving hostnames.
strict	`true`, `false`	Whether to fail if any plugin configuration is invalid.

Example: Configuring AWS EC2 Plugin with Filters

plugin: aws_ec2
regions:
  - us-east-1
  - eu-west-1
filters:
  tag:Environment: production
  instance-state-name: running
hostnames:
  - tag:Name
  - private-ip-address
strict: true

7. Caching Inventory Data

Option	Values	Notes
cache_plugin	`jsonfile`	Plugin used to cache inventory data. Default is `jsonfile`.
cache_connection	`~/.ansible/cache`	Directory where cache files are stored. Default is `~/.ansible/cache`.
cache_timeout	`3600`	Cache timeout in seconds. Default is 3600 (1 hour).

Example: Configure caching in `ansible.cfg`

[inventory]
cache = yes
cache_plugin = jsonfile
cache_connection = /tmp/ansible_cache
cache_timeout = 7200

Note: Caching improves performance by reducing the number of calls to inventory sources. It is especially useful for large inventories or when using external dynamic inventory plugins.

8. Creating Custom Inventory Scripts

Custom inventory scripts allow you to integrate external data sources into Ansible's inventory system. These scripts must output JSON that conforms to Ansible's expected structure.

Field	Description	Required
_meta	Metadata about the inventory, including hostvars.	Optional
all	Group containing all hosts.	Required
groups	Dictionary of group names to lists of hostnames.	Required

Example: Simple Custom Inventory Script (Python)

#!/usr/bin/env python3
import json

# Sample static inventory data
inventory = {
    "all": {
        "hosts": ["web01", "web02", "db01"],
        "children": ["web", "db"]
    },
    "web": {
        "hosts": ["web01", "web02"]
    },
    "db": {
        "hosts": ["db01"]
    },
    "_meta": {
        "hostvars": {
            "web01": {"ansible_host": "192.168.1.10"},
            "web02": {"ansible_host": "192.168.1.11"},
            "db01": {"ansible_host": "192.168.1.20"}
        }
    }
}

print(json.dumps(inventory))

Note: The script must be executable and output valid JSON to stdout.

Warning: Ensure the script has proper permissions to execute (e.g., chmod +x inventory.py) and is discoverable by Ansible.

Key Points

Inventory scripts must output JSON.
Script must be executable and in the correct path or referenced via inventory parameter.
Use _meta.hostvars to define host-specific variables.

9. Using Constructed Inventory

Constructed inventory allows you to dynamically build inventories based on external data sources or logic, rather than static files.

Property	Description	Example
constructed	Enables constructed inventory plugin	`true`
strict	Fail if any group or host is not found	`false`
groups	Define groups based on criteria	`{"web_servers": {"host_filter": "tags.env == 'prod' and tags.role == 'web'"}}`
host_filters	Filter hosts based on variables or facts	`{"all_hosts": "ansible_facts.os_family == 'RedHat'"}`

Example: Constructed Inventory Configuration

plugin: constructed
strict: false
groups:
  web_servers:
    host_filter: "tags.env == 'prod' and tags.role == 'web'"
  database_servers:
    host_filter: "tags.env == 'prod' and tags.role == 'db'"
host_filters:
  all_hosts: "ansible_facts.os_family == 'RedHat'"

Note: Constructed inventory requires the ansible.builtin.constructed plugin to be installed.

Warning: Complex filters can impact performance; test thoroughly in staging environments.

Filter Type	Description	Usage Example
host_filter	Filter hosts based on Jinja2 expressions	`"ansible_facts.cpu_count > 4"`
group_by	Group hosts by a specific variable or fact	`"ansible_facts.os_family"`
compose	Compose new variables from existing ones	`{"fqdn": "{{ ansible_facts.hostname }}.{{ ansible_facts.domain }}"}`

10. Testing Dynamic Inventory

Example: Test dynamic inventory script

#!/bin/bash
# test_inventory.sh - Test script for dynamic inventory

# Output static inventory JSON (for testing)
cat <<EOF
{
  "all": {
    "hosts": {
      "web01": {
        "ansible_host": "192.168.1.10",
        "ansible_user": "admin"
      },
      "db01": {
        "ansible_host": "192.168.1.20",
        "ansible_user": "admin"
      }
    },
    "vars": {
      "ansible_connection": "ssh"
    }
  }
}
EOF

Command	Description	Usage
ansible-inventory -i inventory_script.py --list	Lists inventory from a dynamic script	`ansible-inventory -i test_inventory.sh --list`
ansible-inventory -i inventory_script.py --host <hostname>	Shows host-specific variables	`ansible-inventory -i test_inventory.sh --host web01`

Note: Ensure the script is executable and outputs valid JSON.

Warning: Dynamic inventory scripts must be executable and return valid JSON.