Agents
Flux Agents
Agents are lightweight Flux add-ons that allow the remote execution of any of the four following tasks:
- Process Action (invoke command-line programs and scripts)
- File Copies
- File Moves
- File Polling (trigger a workflow when a file appears)
Agents are not a replacement for the Flux engine – rather, they augment the functionality of the engine by allowing it to send these specified tasks to be executed remotely on the network. For this reason, at least one engine is necessary in order to use Flux agents.
Some of the major benefits of agents:
- Lightweight: Agents have a lighter footprint than engines and are ideal for machines where only a small subset of the overall process must be run.
- Scalable: Agents and engines communicate efficiently and can be spread across a wide network.
- Remote Execution: Agents run tasks remotely from the Flux engine, letting you access your processes wherever they may need to be run.
- Targeted Execution: Tasks can be targeted to specific agent pools, ensuring your tasks only run on the machines with the necessary resources.
- Reporting: Agents send their results back to Flux, allowing them to be handled and reported using the traditional Flux mechanisms.
- DMZ and Firewall Friendly: Agents do not need to open any incoming ports, allowing them to safely receive work behind a firewall or inside your DMZ.
Flux Engine-Agent relationship
The Flux engine (or cluster) coordinates all tasks that are to be executed on the remote agents. Engine-agent communication has two simple requirements:
- The Flux engine’s machine must allow incoming traffic on the engine port (by default, TCP/IP port 7520).
- The Flux agent must allow outgoing traffic on the same port.
That’s it! As long as these requirements are met, the Flux engine and agent can communicate. You do not need to open any incoming ports on the agent side.
List of Configuration Properties
The following table contains a list of configuration properties that can be set in your agent configuration, along with a description of each, and the default value (if any) that is used if you do not set the configuration property yourself.
| Property | Description | Default Value |
|---|---|---|
| CONCURRENCY_THROTTLE | The number of processes that can execute simultaneously on this agent. | 10 |
| ENGINE_ENCRYPTED_PASSWORD | The encrypted password that the Agent uses to log in to a remote Flux engine. Encrypted passwords can be generated using the Engine’s command line interface. This property can only be used if ENGINE_PASSWORD is null or empty. | none (this property is not set by default) |
| ENGINE_HOST | The hostname of the Flux engine with which an agent intends to register. When an agent registers with an engine, the agent effectively registers with all engines in that cluster. This property must be set to allow the agent to look up the engine. | none (this property is not set by default and must be explicitly configured) |
| ENGINE_PASSWORD | The password this agent uses to login to a remote Flux engine. This property can only be set if ENGINE_ENCRYPTED_PASSWORD is null or empty. | none (this property is not set by default) |
| ENGINE_PORT | The TCP/IP port that the remote Flux engine is bound to. The agent will use this port to look up the remote engine. | 7520 |
| ENGINE_USERNAME | The username that this agent will use to log in to the remote Flux engine. | none (this property is not set by default) |
| ID_DESCRIPTION | A description of the agent instance intended for human consumption. | none |
| ID_FULL_NAME | A long, descriptive name of the agent instance, intended for display purposes. | none |
| ID_NAME | The case-insensitive name of the agent instance. Used to differentiate agent instances in the logs, the audit trail, and Flux security login modules. If the name is not set explicitly in a configuration, the configuration will generate a name using: 1.The hostname of the local computer but not “localhost”. For example, “foo” or “foo.bar.com”. 2. Failing that, the IP address of the local computer but not “127.0.0.1”. For example, “12.34.56.78”. 3. Failing that, “unknown”. If there is more than one active agent in a JVM, each generated name is made unique. To make a generated name unique, a “:” symbol is first appended to the name that is generated as described above. A number, starting from one, is then appended to ensure that the generated name is unique within the JVM. | determined on agent startup (see description column) |
| LOG_FILENAME | The path (relative or absolute) and filename of the log file that this agent should use for its logging data.Relative paths are translated relative to the directory where Flux was started – usually, the Flux installation directory. | flux-agent.log |
| POOL | The (case-insensitive) name of this pool that the agent will belong to. Agent pools allow you to dictate where Process Actions send their work to be executed – a Process Action may run its work on any available agent, or it can use a pool to specify a specific agent or group of agents to execute the work. If an agent’s POOL configuration setting matches the value that a Process Action uses, that agent will be eligible to run the work for the Process Action.This property can be set to any value except ‘*’ is a special character used in Process Actions to indicate that the work can be run on any available agent, regardless of the pool. | none (agents do not belong to any pool by default) |
As with engine configurations, agent configurations use the standard Java properties file format <Property name>=<value>. A sample agent configuration file might look like this:
Agent Configuration
ENGINE_HOST=myhost
ENGINE_PORT=7520
ENGINE_USERNAME=admin
ENGINE_PASSWORD=
Creating a Flux Agent
For your consideration, Flux includes a script for launching a Flux agent. This script, start-agent, is included in the installation directory of your Flux package.
If you installed Flux using the Windows installer, an Agent is automatically installed as a Windows service as well.
If you run your agent using the default script or service, you can find its configuration settings in config-agent-config.properties. If you’re running Flux on a remote machine for the first time, you’ll probably need to specify your ENGINE_HOST and ENGINE_PORT to match the name and location of the Flux agent you want this agent to receive instructions from.
Agents can also be created and started from the Command Line Interface, like so:
Creating a Flux Agent from the command line
java flux.Main agent-server -configurationfile path/to/agent-config.properties start
Where path/to/agent-config.properties is the path to an agent configuration file. The configuration properties supplied in the configuration will determine how the agent is created and how it connects to the Flux engine.
You can also specify configuration settings directly on the command line. When specifying a configuration setting, the name of the property should be applied in lowercase, with all ‘_’ characters removed, followed by a space, then the value of the configuration property. For example:
Start a Flux Agent from the command line with specific options
java flux.Main agent-server -port 7520 -enginehost localhost -engineusername admin -enginepassword admin start
States of a Flux Agent
The diagram below outlines the various states of the Flux agent:
Once an agent is created, it enters its initial state, registering. From this state, it will either register successfully and move to the STOPPED state, or it will continue trying every 3 minutes until it is able to successfully register.
From the STOPPED state, the agent can be started, at which point it is FREE. A FREE agent can accept work from a Flux engine.
Once some work arrives from the engine, the agent is BUSY. A BUSY agent is an agent that is actively executing a task. If an agent is BUSY and its CONCURRENCY_THROTTLE is full, the agent will not accept any new work until at least one existing item is finished.
From any state, the agent can be DISPOSED, where it will stop executing running until it is started again, where it will repeat this cycle.