Skip to main content
Mapagam

Implementing Federation

1. Understanding Federation Use Cases

Common Federation Scenarios

  • Geographic Distribution: Connect RabbitMQ instances across data centers or regions to maintain message flow despite network partitions.
  • Legacy System Integration: Bridge modern systems with older messaging infrastructure without full migration.
  • Load Distribution: Distribute message load across multiple RabbitMQ clusters to improve throughput and reduce latency.
  • High Availability: Ensure message delivery even if one RabbitMQ instance becomes unavailable.
  • Multi-Tenant Environments: Isolate tenant messages while allowing cross-tenant communication through federated exchanges.
Use Case Description Benefits Implementation
Cross-Cluster Messaging Messages flow between clusters in different regions or data centers. Improved resilience, reduced latency for global services. Configure federation upstreams and exchanges to route messages across clusters.
Legacy Integration Connect to older systems that use different messaging protocols or formats. Enables gradual migration without downtime or complex re-architecture. Use federation to bridge AMQP 0.9.1 and newer versions or other protocols.
Scalability Distribute message load across multiple RabbitMQ nodes or clusters. Enhanced performance and reduced bottlenecks in high-volume environments. Set up federation links to balance traffic across clusters.
Disaster Recovery Ensure message availability during outages or failures in one cluster. Provides redundancy and maintains data integrity across systems. Configure federation to replicate messages to backup clusters automatically.
Note: Federation is ideal for scenarios where full clustering isn't feasible or desired, offering a lightweight alternative to mirroring.

2. Enabling Federation Plugin

Command Description Notes
rabbitmq-plugins enable rabbitmq_federation Enables the federation plugin Required for federation features
rabbitmq-plugins enable rabbitmq_federation_management Enables the federation management plugin Provides UI and API for federation configuration

Example: Enable Federation Plugins

rabbitmq-plugins enable rabbitmq_federation
rabbitmq-plugins enable rabbitmq_federation_management
Note: After enabling the plugin, restart RabbitMQ for changes to take effect.

3. Defining Upstream

Property Description Values
federation.upstream Defines the upstream RabbitMQ instance to federate with string
federation.upstream.set Defines a set of upstreams to use for federation string
federation.upstream.set.name Name of the upstream set string
federation.upstream.set.uri URI of the upstream RabbitMQ instance string

Example: Defining an upstream

federation:
  upstream:
    name: "upstream-rabbit"
    uri: "amqp://user:pass@upstream-host:5672/vhost"

Example: Defining an upstream set

federation:
  upstream:
    set:
      name: "my-upstream-set"
      uri: "amqp://user:pass@upstream-host:5672/vhost"

4. Setting Upstream Parameters

Parameter Description Default Value Notes
uri URI of the upstream RabbitMQ server None Required for federation setup
vhost Virtual host to federate from / Can be overridden per federation link
username Username for upstream authentication guest Must have appropriate permissions
password Password for upstream authentication guest Should be secured in production
heartbeat Heartbeat interval in seconds 60 Used to detect broken connections
reconnect_delay Delay before reconnecting in seconds 5 Used to avoid overwhelming the upstream server

Example: Federation Upstream Configuration

{
  "uri": "amqp://upstream-server:5672",
  "vhost": "/federated-vhost",
  "username": "federation-user",
  "password": "secure-password",
  "heartbeat": 30,
  "reconnect_delay": 10
}
Note: The upstream parameters must match the configuration of the target RabbitMQ server to ensure successful federation.

5. Applying Federation Policy

Example: Define federation policy

rabbitmq-policy set --vhost / my-federation-policy ^amq\.fifo$ federation
Property Value Description
pattern ^amq\.fifo$ Regular expression matching queue names to apply policy to
definition federation Policy name to apply (e.g., federation, replication)
priority 0 Priority of policy (higher values override lower)
Note: Policies are applied per virtual host. Ensure the policy is set on the correct vhost.
Warning: Federation policies can cause message duplication if not configured correctly.

Policy Application Steps

  • Define the pattern to match queues
  • Set the federation policy using rabbitmq-policy
  • Verify policy is applied to matching queues

6. Using Exchange Federation

Exchange federation allows you to federate exchanges across different RabbitMQ nodes, enabling message routing and distribution across clusters.

Property Description Default Value Notes
exchange Name of the exchange to federate None Required field
local Local exchange name (used for federation) Same as exchange Optional field
remote Remote exchange name (used for federation) Same as exchange Optional field
prefetch_count Number of messages to prefetch from the remote exchange 100 Optional field
reconnect_delay Time (in seconds) to wait before reconnecting to the remote exchange 5 Optional field

Example: Configuring Exchange Federation

{
  "federation": {
    "exchange": "my-exchange",
    "local": "local-exchange",
    "remote": "remote-exchange",
    "prefetch_count": 50,
    "reconnect_delay": 10
  }
}
Note: Ensure that the local and remote exchanges are configured to handle the same message types and routing keys for proper federation.

7. Using Queue Federation

Queue Federation allows you to federate queues across different RabbitMQ nodes or clusters, enabling message routing and distribution.

Property Description Default Value
federation.upstream Name of the upstream exchange or queue to federate None
federation.upstream.set Name of the upstream set to federate None
federation.reconnect-delay Time in seconds to wait before reconnecting after a failure 5
federation.max-reconnects Maximum number of reconnect attempts before giving up 10

Example: Configuring Queue Federation

{
  "federation": {
    "upstream": "my-upstream-queue",
    "reconnect-delay": 10,
    "max-reconnects": 5
  }
}
Note: Queue Federation requires the federation plugin to be enabled on the RabbitMQ node.
Warning: Ensure that the upstream queue or exchange exists and is accessible before enabling federation.
Property Value Description
federation.upstream string Name of the upstream exchange or queue to federate
federation.upstream.set string Name of the upstream set to federate
federation.retry_interval integer Retry interval in seconds (default: 5)
federation.max_hops integer Maximum number of hops (default: 1)
federation.prefetch_count integer Number of messages to prefetch (default: 100) 
{
  "federation": {
    "upstream": "upstream_exchange",
    "retry_interval": 10,
    "max_hops": 2,
    "prefetch_count": 50
  }
}
Note: Ensure the upstream exchange or queue exists in the federated broker before configuring the link.

9. Setting Max-Hops

Property Values Description
federation.max-hops 1 to 255 Maximum number of federation hops allowed before a message is rejected. Defaults to 1.

Example: Configuring max-hops in rabbitmq.conf

federation.max-hops = 3
Note: Setting federation.max-hops to a higher value allows messages to traverse more federation links, but increases the risk of message loops or long processing times.

10. Understanding Federation Topology

Note: Federation in RabbitMQ allows you to connect multiple RabbitMQ brokers and route messages between them, creating a federated topology that can span across different networks or data centers.

Federation enables you to:

  • Replicate queues and exchanges across brokers
  • Route messages between federated brokers
  • Ensure high availability and scalability

Federation vs. Mirroring

  • Federation: Connects separate RabbitMQ instances to route messages between them
  • Mirroring: Duplicates queues within a single cluster for redundancy
Topology Type Description Use Case
Federated Upstream Connects to an upstream broker to consume messages from it Replicate messages from a remote broker to local queues
Federated Downstream Routes messages to a downstream broker Forward messages from local queues to remote brokers
Federated Exchange Routes messages between exchanges in different brokers Enable exchange-level federation for routing flexibility

Example: Federated Exchange Configuration

{
  "federation-upstreams": {
    "upstream-1": {
      "uri": "amqp://user:pass@remote-broker:5672/vhost",
      "reconnect-delay": 5
    }
  },
  "federation-exchanges": {
    "exchange-name": {
      "upstream": "upstream-1",
      "prefetch-count": 100
    }
  }
}

Topology Components

  • Broker: A RabbitMQ instance that can be part of a federation
  • Upstream: A remote broker that provides messages to be federated
  • Downstream: A local broker that receives messages from a federated source
  • Exchange: A routing mechanism that can be federated across brokers

Setting Up Federation Topology

  1. Enable the federation plugin on both brokers
  2. Define upstream connections to remote brokers
  3. Configure exchange or queue federation rules
  4. Start the federation agents to begin message routing

11. Monitoring Federation Status

Property Description Values
federation.upstream Defines the upstream exchange or queue to federate exchange.name or queue.name
federation.retry_interval Time in seconds to wait before retrying failed federation 30
federation.max_hops Maximum number of hops for federation messages 1

Example: Federation Status Check

rabbitmqctl list_federation_upstreams

Example: Federation Health Check

rabbitmqctl list_federation_links
Note: Federation status can be monitored via RabbitMQ Management UI or CLI commands.
Warning: Ensure upstream connections are stable to avoid message loss.

12. Troubleshooting Federation Issues

Issue Possible Cause Solution
Federation link fails to establish Network connectivity issues or incorrect broker URLs Verify network access to the upstream broker and check federation configuration for correct host/port settings
Messages not being federated Incorrect federation policy or missing permissions Ensure the federation policy is correctly applied and user has necessary permissions to read from upstream
High latency in message delivery Network congestion or broker overload Monitor network and broker performance; consider increasing federation link prefetch count
Message duplication observed Unreliable message handling or duplicate detection misconfiguration Enable idempotent publishing and review federation settings for duplicate handling 
rabbitmqctl eval 'federation:status().'
Note: Use rabbitmqctl eval 'federation:status().' to get detailed status of all federation links.
Warning: Ensure that the upstream broker is accessible and that all required plugins (e.g., rabbitmq_federation) are enabled.

Was this article helpful?

Related Cheat Sheets