Engine Configuration
The engine configuration defines both how your engine runs, and how it stores and maintains data from the Flux engine (including workflow and user data).
The engine configuration settings can be configured using the Operations Console, in a .properties file, or, if you are starting the engine using Java code, a flux.Configuration object. These configuration settings cannot be changed after the engine has started. For a comprehensive list of static configuration options, see the Javadoc for the flux.Configuration option (Javadocs are located in the /doc/javadoc directory under your Flux installation directory).
Engine behavior is also controlled by the Runtime Configuration. The runtime configuration and the engine configuration are used together to define how the engine operates.
List of Configuration Properties
The following table contains a list of configuration properties that can be set in your engine configuration, along with a description of each, and the default value (if any) that is used if you do not set the configuration property yourself.
| Property | Description | Default Valve |
|---|---|---|
| AGENT_FAILOVER_TIME_WINDOW | A time expression that specifies the duration of time between agent connectivity tests. If an agent is busy and cannot be reached during this time period, the process assigned to the unreachable agent will be reassigned to the next available agent in the pool. | +15m |
| AUDIT_TRAIL_EXPIRATION | A time expression indicates how old individual audit trail, run average, and run history entries must be before they are automatically pruned. Expired entries are automatically deleted from the database. A null audit trail expiration indicates that these entries are never deleted from the database. If your workflows use the Audit Trail Trigger, take caution when setting this property to ensure that your triggers will have sufficient time to record events before they are deleted from the database. | +1H (when using the H2 in-memory database), OR +w (for all other database types) |
| AUDIT_TRAIL_FILTER | A list of audit trail events to record to the audit trail. If this audit trail filter is not set or empty, all audit trail events are recorded to the audit trail. For usage instructions and examples, see Logging and Audit Trail. | none (by default, all audit trail events are recorded) |
| AUDIT_TRAIL_LOGGER | The name of the logger used when logging audit trail information. | audit_trail |
| AUDIT_TRAIL_ROLLBACK_RECORDINGS_ENABLED | Indicates whether actions that execute in a workflow that are later rolled back in the current database transaction are still recorded to the audit trail. Useful for knowing what actions led up to an error before the error caused the current database transaction to be rolled back. | false |
| CACHE_SIZE | The initial size of the workflow cache. The cache size determines the number of workflows that can be stored in the cache. Flux will dynamically increase the size of the cache as needed to accommodate larger loads of workflows that arrive. | 200 |
| CACHE_TYPE | This configuration property controls how workflows are cached. There are two possible settings for this property: 1. LOCAL: Workflows are only cached locally, to increase performance without sharing cache information across the network. 2. NONE: Workflows are not cached at all and are pulled directly from the database. Depending upon the nature of your workflows, there are instances where LOCAL cache performs more slowly than NONE. This is especially true with very large workflows involving many actions and variables. | LOCAL |
| CLIENT_LOGGER | The name of the logger used when logging client-side information. | client |
| CLUSTER_NETWORKING_ENABLED | Indicates whether networking is enabled in the engine instance when it participates in a cluster. Networking is not required for clustering, but networking increases responsiveness among engine instances across the cluster. By disabling cluster networking, the engine instance still participates in a cluster, but all its communication with other engine instances is performed through the database server. The workflow real-time monitor, file transfer monitor, and agents depend on cluster networking functionality. These features will be disabled when cluster networking is turned off. | true |
| CONCURRENCY_LEVEL | The concurrency throttle limit in the root node of the runtime configuration tree. This limit controls the number of flow contexts that may execute at one time across the entire engine. In essence, this configuration property provides a shortcut to the concurrency throttle limit in the root node of the runtime configuration tree, for the sake of convenience. This property is only used if no runtime configuration is available. If a runtime configuration is available, the root throttle of the runtime tree will be used instead. | 10 |
| DAILY_MAINTENANCE_TIME | A time expression specifying the time at which the engine performs once-a-day maintenance on the Flux database tables. For example, to run daily maintenance each morning at 04:30, set this configuration property to the Cron-style time expression “0 0 30 4”. The time zone is always the local time zone. Regardless of the exact time expression used, maintenance is performed exactly once per day. | 0 0 0 2 (By default, maintenance is done at 02:00 each morning) |
| DATA_SOURCE | The JNDI name used to lookup an application server’s database connection pool. | none (you must explicitly set this property in order to use a data source) |
| DATA_SOURCE_AUTO_COMMIT_CHECK | Indicates whether to disable auto-commit on database connections obtained through a data source. If this property is enabled, the engine will check every connection retrieved from the data source for an auto-commit value of “false”. If the connection has an auto-commit value of “true”, the engine will log a warning and automatically attempt to disable auto-commit on the connection. If this property is disabled, the engine will not attempt to check data source connections for the auto-commit value. | true |
| DATA_SOURCE_CACHING | Indicates whether data source objects are saved and reused later. Such caching of data source objects reduces warning messages emitted by WebSphere 5.0. Unfortunately, this caching may also reduce the ability of an application server to perform clustering. If possible, it is preferred to keep this configuration property at its default setting of false, which means that data source objects are not cached. | false |
| DATA_SOURCE_PASSWORD | The password used to retrieve a database connection from an application server data source. This property is only used if it is not null and not the empty string, the data source username is not null and is not the empty string, and a data source has been set. | ”” (empty string) |
| DATA_SOURCE_USER_TRANSACTION_CLIENT_JNDI_NAME | The JNDI name of a javax.transaction.UserTransaction object that should be used for client calls into the Engine. User transactions are used to guarantee transaction integrity when the engine’s transactions need to be integrated with transactions from other software systems. This property is only used when DATA_SOURCE_USER_TRANSACTIONS is enabled. | java:comp/UserTransaction |
| DATA_SOURCE_USER_TRANSACTION_SERVER_JNDI_NAME | The JNDI name of a javax.transaction.UserTransaction object that should be used for workflows and background tasks running on the engine. User transactions are used to guarantee transaction integrity when the engine’s transactions need to be integrated with transactions from other software systems. This property is only used when DATA_SOURCE_USER_TRANSACTIONS is enabled. | java:comp/UserTransaction |
| DATA_SOURCE_USER_TRANSACTION_TIMEOUT | A time expression specifying the timeout for all user transactions that the engine uses. If this property is null, Flux will not set timeouts for user transactions. This property is only used when DATA_SOURCE_USER_TRANSACTIONS is enabled. | +d (one day) |
| DATA_SOURCE_USER_TRANSACTIONS | Indicates whether a User Transaction object manages database transactions that are retrieved from a data source. If true, a javax.transaction.UserTransaction object is used to manage transactions. If false, Flux manages transactions. | FALSE |
| DATA_SOURCE_USERNAME | The username used to retrieve a database connection from an application server data source. This property is only used if it is not null and not the empty string and a data source has been set. | ”” (empty string) |
| DATABASE_PROPERTIES | The full path to a database properties file. Database properties allow you to specify how Flux should connect to the database, rename database tables and columns, reassign a standard SQL type, and provide initialization SQL statements to database connections that are created to directly access the database. For complete documentation on this property, see Databases. | none |
| DATABASE_TYPE | The type of database that the engine should use. NOTE: If the DATABASE_TYPE is set to Derby or H2, you will not be able to use clustering or failover with this engine. | DERBY |
| DRIVER | The class name of a JDBC driver. If DATABASE_TYPE is DERBY or H2, this property is ignored (as Flux automatically manages drivers for those database types). | none |
| ENGINE_CONTENTS_SUMMARY_FREQUENCY | A time expression that specifies how often engine contents status audit trail events are generated. The engine contents summary audit trail event provides information on all of the workflows, messages, and message publishers that exist on the engine. Although this property allows you to adjust the frequency of these events, it is not possible to disable the engine contents summary completely. The minimum frequency is 5 minutes. | +1d (one day) |
| FAILOVER_TIME_WINDOW | A time expression that specifies how often the engine checks to see whether other engines in the cluster have failed. This is used for failover purposes to determine whether any workflows have prematurely stopped running and must be restarted on another engine. | +3m (three minutes) |
| FAIRNESS_TIME_WINDOW | A time expression that specifies how often Flux should temporarily increase workflow priorities to prevent lower-priority workflows from being starved (not allowed to run) by higher-priority workflows. A non-running workflow’s priority will be temporarily updated each time Flux performs the fairness check. The workflow’s priority may be temporarily increased several times before it is able to run. Once the workflow does run, its priority will be reset to its original value and it will begin waiting again for the next opportunity to execute. A workflow’s permanent priority is called its “priority”. If the priority is temporarily elevated by fairness, that elevated priority is called its “effective priority”. For additional information, see the runtime configuration priority setting. | none (fairness checks are not performed by default — setting a value for this property will automatically enable them) |
| FILE_TRANSFER_DEBUG | Indicates whether debugging is enabled for file transfers. If this property is enabled, all interaction between Flux and any remote file hosts will be logged to the Flux system log. When enabled, this property requires the INTERNAL_LOGGER_LEVEL to be set at FINE logging level or greater. | false |
| FLOW_CHART_DEADLINES_ENABLED | Indicates whether workflow deadlines (SLAs or Service Level Agreements) are enabled; disabling deadlines reduces database activity. When this property is enabled, you can set deadlines on individual workflows. If a workflow exceeds its deadline, it will publish an audit trail event indicating that the deadline has passed. The Operations Console will also graphically indicate any workflows that are approaching (or have exceeded) their deadlines. | true |
| FLOW_CHART_LOGGER | The name of the logger used when logging information during workflow execution. | flow_chart |
| HEARTBEAT_FREQUENCY | A time expression that specifies the frequency with which heartbeat audit trail events are generated. A heartbeat audit trail event provides information on the health of the engine. Although this property allows you to adjust the frequency of these events, it is not possible to disable heartbeats completely. The minimum frequency is 5 seconds. | +1H (one hour) |
| HOST | The hostname or IP address that the engine should bind to. If there are multiple hostnames / IP addresses available on the server where the engine is running, this property allows you to specify which one Flux should use for network requests. If you are using this configuration to look up an existing engine, this property allows you to specify the hostname where the engine is running. | Flux will automatically select a hostname from the available names on the system |
| ID_DESCRIPTION | A description of the engine instance intended for human consumption. | none |
| ID_NAME | The case-insensitive name of the engine instance. Used to differentiate Flux engine instances in the logs and audit trail. If the name is not set explicitly in a configuration, the configuration will generate a name using: 1. The hostname of the local computer but not “localhost”. For example, “foo” or “foo.bar.com”. 2. Failing that, the IP address of the local computer but not “127.0.0.1”. For example, “12.34.56.78”. 3. Failing that, “unknown”. If there is more than one active Flux engine in a JVM, each generated name is made unique. To make a generated name unique, a “:” symbol is first appended to the name that is generated as described above. A number, starting from one, is then appended to ensure that the generated name is unique within the JVM. | determined on engine startup (see description column) |
| INITIAL_CONTEXT_FACTORY | The JNDI name of an application server’s initial context factory. | none |
| INITIAL_CONTEXT_PASSWORD | The password used to create an application server’s initial context when looking up objects in an application server’s JNDI tree (like data source, EJB, or JMS information). This property is only used if it is not null and not the empty string and the INITIAL_CONTEXT_USERNAME configuration property is not null and is not the empty string. | ”” (empty string) |
| INITIAL_CONTEXT_USERNAME | The username used to create an application server’s initial context when looking up objects in an application server’s JNDI tree (like data source, EJB, or JMS information). This property is only used if it is not null and not the empty string. | ”” (empty string) |
| INTERNAL_LOGGER_FILE_DIRECTORY | The directory where log files are placed when using the internal logger. | . (equivalent to the system property “user.dir”, typically the directory where the engine or JVM was started) |
| INTERNAL_LOGGER_FILE_ROTATION_SIZE | The maximum size to which a log file for the internal asynchronous and internal synchronous loggers can grow before it is rotated. At most, there are seven log files, which include the current log file and up to six previously rotated log files. The rotation size is specified in bytes. | 10485760 (ten megabytes, 10 * 1024 * 1024) |
| INTERNAL_LOGGER_LEVEL | The logging level used for the internal asynchronous and internal synchronous loggers. This property does not affect the logging level for any other loggers. | INFO |
| JDBC_CONNECTION_RECYCLE_FREQUENCY | When Flux is configured to obtain database connections directly through JDBC (rather than a data source), it creates a connection pool and periodically recycles connections in the pool. This property is a time expression that indicates how often Flux should recycle database connections. The minimum recycle frequency is 15 minutes, and the maximum is 90 days. | +1H (one hour) |
| JDBC_ENCRYPTED_PASSWORD | The encrypted password used to obtain database connections when Flux is configured to obtain database connections directly through JDBC (rather than a data source). This property is only used if it is not null and not the empty string, the JDBC_USERNAME configuration property is not null and is not the empty string, and the JDBC_PASSWORD configuration property is null or the empty string. Passwords can be encrypted using the encrypt password command-line utility. | ”” (empty string) |
| JDBC_PASSWORD | The password used to obtain database connections when Flux is configured to obtain database connections directly through JDBC (rather than a data source). This property is only used if it is not null and not the empty string and the JDBC_USERNAME configuration property is not null and is not the empty string. Must be supplied in cleartext format. | none |
| JDBC_USERNAME | The username used to obtain database connections when Flux is configured to obtain database connections directly through JDBC (rather than a data source). This property is only used if it is not null and not the empty string. | sa |
| LOG_EXPIRATION | A time expression indicates how old log entries in the database must be before they are pruned (deleted) from the database. At the DAILY_MAINTENANCE_TIME each day, Flux will find and prune all entries older than this time expression. If this property is null, Flux will not attempt to prune log entries in the database. | +w (one week). If the DATABASE_TYPE is H2, however, the default value is ?+1H (one hour). |
| LOGGER_TYPE | Indicates which logging system Flux should use. This property should only be set when a single logging system is used on the engine; for multiple loggers, use the LOGGER_TYPES property instead. | INTERNAL_ASYNCHRONOUS |
| LOGGER_TYPES | Indicates which logging systems Flux should use. This property should only be set when multiple logging systems are to be used on the engine (for example, this could be configured to use both log4j and the internal asynchronous logger to log messages). For usage instructions, see Logging and Audit Trail. | INTERNAL_ASYNCHRONOUS |
| MAX_CONNECTIONS | The maximum number of database connections that the engine should be allowed to have open at once. For more information on the MAX_CONNECTIONS, including recommended settings, see Max Connections and Concurrency Level. | 15 |
| ORACLE_LARGE_OBJECT_ADAPTER | By default, Flux uses standard JDBC APIs for persisting large binary data and characters to the database. Due to an Oracle limitation, however, standard JDBC cannot persist more than 4k bytes of data when communicating with an Oracle database. The Oracle large object adapter works around this limitation using an Oracle-specific API. This configuration property accepts the name of an adapter class that uses database- and driver-specific APIs to overcome this Oracle limitation. The adapter class specified must implement the flux.OracleLargeObjectAdapter interface. For your convenience, Flux provides several such adapter classes by default. They can be used either when the Oracle JDBC driver is used directly, or when an application server driver wraps the Oracle driver (an Oracle driver supplied by Oracle Corporation must be used, however, either directly or indirectly). The class names for these drivers are listed as constants in the Javadoc documentation for the flux.OracleLargeObjectAdapter interface. If you prefer, you can provide your own adapter. It must implement the flux.OracleLargeObjectAdapter interface. Your custom adapter can be used to support non-Oracle drivers or other application servers that provide JDBC drivers that wrap the Oracle JDBC drivers. An Oracle Large Object Adapter must be defined when using an Oracle data source. | none |
| PORT | The TCP/IP port to which this engine will listen. This should be set to a port number that is available on the server. | 7520 |
| PROVIDER_URL | The URL to an application server. Used when creating an initial context. | none |
| RUN_HISTORY_ENABLED | Indicates whether workflow run history information is recorded to the database; disabling run history reduces database activity. Run histories include execution times for workflows and actions as well as workflow metrics. These metrics include counts of successful and unsuccessful workflow runs. | true |
| RUNTIME_CONFIGURATION | Used to populate the workflow namespace with configuration properties that apply to one engine only. These configuration properties can change on the fly, that is, dynamically. A union of the cluster-wide runtime configuration properties and the per-engine runtime configuration properties is formed to govern engine behavior. If the union creates a conflict, the per-engine properties take precedence. | A default runtime configuration that allows at most one workflow to run at the same time and defines a default error handler that retries failed actions up to five times, with one-minute delays after a failure. This default error handler gives up after the fifth failure and places the workflow in the ERROR super-state and the PAUSED sub-state. This default runtime configuration is immutable. You cannot make any changes to it. |
| RUNTIME_CONFIGURATION_FILE | The path to a runtime configuration file. Used to load a runtime configuration from a file instead of using the appropriate APIs. See Runtime Configuration for usage instructions. | none |
| RUNTIME_CONFIGURATION_FILE_REFRESH_FREQUENCY | A time expression that specifies the frequency with which the runtime configuration file is reloaded automatically. Using this configuration property, if changes are made to the runtime configuration file, they can be imported into the engine automatically. | +1m (one minute) |
| SCRIPTING_LANGUAGES | Specifies the scripting languages that are registered with the Flux engine. Scripting languages are used for pre-and post-processing in triggers and actions. If this configuration property is null, BeanShell is used as the default scripting language. See Prescripts and Postscripts (And Scripting in General) for more information and usage. | null (BeanShell is assumed to be the default language in this case) |
| SECURITY_ENABLED | Indicates whether security is enabled. An engine that is created with security enabled can accept remote connections from clients and users. Either this property or SERVER must be set to true (see below) in order to connect to the engine remotely (using the Operations, the Java APIs, or any other client). | false |
| SERVER | Indicates whether this engine will be created as a server that can be contacted over the network. | false |
| SSL | Indicates whether SSL is enabled on the server. This allows secure communication between the engine and all clients, including the Operations Console. | true |
| SYSTEM_DELAY | The maximum amount of time the engine sleeps when it has nothing to do. | +3m (three minutes) |
| SYSTEM_DELAY_MINIMUM | The minimum amount of time the engine sleeps before polling the database to determine if more work is available. This option is a temporary workaround to reduce database activity until a better solution can be implemented. | +1s (one second) |
| SYSTEM_LOGGER | The name of the logger used when logging system information. | system |
| SYSTEM_RESOURCE_COMMAND | Command to obtain system resource information such as CPU and memory usage. Standard output generated by the command is used to update system resource values. The system resource command is executed at the intervals defined by the system resource interval configuration property. This should either be the full path to a command, script, or executable on the server, or the name of a command, script, or executable that is available from the system PATH. See below for more information on the type and syntax of output that is expected to be returned from this command. | none (the system resource command is not used by default) |
| SYSTEM_RESOURCE_INTERVAL | A time expression specifying how often the SYSTEM_RESOURCE_COMMAND should be executed. | ”” (not set by default) |
| TABLE_PREFIX | The prefix for all of the Flux database tables. | FLUX_ |
| URL | A JDBC database URL. If DATABASE_TYPE is DERBY or H2, this property is ignored. | none |
| WORK_MANAGER_RESOURCE_NAME | The name used to locate a Work Manager resource, to which the engine delegates background processing instead of using threads created directly by the engine. If this property is not defined or does not refer to a valid Work Manager resource, the engine delegates background processing to threads that it creates directly. For WebSphere and WebLogic, this name refers to a CommonJ WorkManager JNDI name configured in WebSphere or WebLogic. For JBoss, this name refers to a work manager MBean (for example: “jboss.jca:service=WorkManager”) configured in JBoss. For GlassFish, this name refers to a thread pool (for example: “thread-pool-1”) configured in GlassFish. | none |
Disabling SERVER
Disable the SERVER option by setting:
SERVER=false
Your engine configuration allows you to prevent Flux from accepting any remote connections to the engine. This can be useful when the engine needs to be “locked down” and unavailable to end-users, or if there are restrictions on the network that require Flux to avoid opening ports for communication.
Note, however, that this property can only be set if your engine is created programmatically (using the Flux Java APIs). If your engine is started from the command line, from a service on the Operating System, or from a batch or shell script, the SERVER property will always be enabled even if you have explicitly disabled it in the configuration.
The reason for this is that the engine, if created through code, can still be interacted with locally (using the Java API). If the engine is created from a command line, service, or script, however, it will not be able to accept any work or input from a user (since the only way to send work to an engine is to access the engine over the network). Because of this, if you have a need to disable the SERVER option, you will need to be sure that the engine is created using the Java APIs.
Note also that to completely disable network communications on the engine, you will need to disable cluster networking as well, by setting:
CLUSTER_NETWORKING_ENABLED=false
in your engine configuration.
SERVER and SECURITY_ENABLED
If you have enabled security on your engine by setting:
SECURITY_ENABLED=true
The engine will automatically be accessible over the network. It is not necessary to enable the SERVER property; if you do attempt to set SERVER=true while security is enabled, the engine will encounter an error on startup and will fail to start until you remove the SERVER property.
If you set SERVER=false, but security is enabled, then the SERVER property is ignored and the engine will automatically enable remote communication. It is not possible to disable remote communication with security enabled.
This also means that a secured server will always accept remote communication (provided that valid credentials are specified) and remote communication does not need to be explicitly enabled.
For more information on remote communication and security, see Security.
Configuring Flux to Use Your Database
See Configuring Flux to run with Databases for instructions on setting up the Flux engine to point to your database.
Configuring the System Resource Command
If you have the SYSTEM_RESOURCE_COMMAND property set, Flux can use the output of the command invoked to gather information about the current resources on the system (including memory and CPU statistics) and make scheduling decisions based on that information.
When you invoke the system resource command, it must display the following values to standard output, on a single line, separated by tab characters, in this order:
- Percentage of CPU usage
- Total physical memory
- Free physical memory
- Total virtual memory
- Free virtual memory
The output of the monitor command, therefore, might look something like:
10.000000 1051120 306356 2097024 2065060
To see a sample script you can adapt for your own system resource command, see the examples/software_developers/system_resource_monitoring directory under your Flux installation directory. This directory contains example scripts for both Windows and Unix environments that you can use to quickly configure the command on your own system.
Once you’ve enabled the system resource monitor, Flux will automatically use the CPU information it retrieves to make scheduling decisions across the cluster. As long as you have two or more engines running in the cluster and all of the engines have the system resource command enabled, Flux can automatically assign workflows to the engine with the lowest CPU usage (and therefore the engine with the most available resources for running a new workflow).