Core Triggers and Actions - Process Action
The documents in this section discuss the core triggers and actions in Flux.
The Process Action, also known as the “Command Line Action”, can execute command-line programs, batch files, scripts, and, more generally, any native program.
You can run processes in the foreground (meaning that the workflow will wait for the Process Action to finish running before moving on), or asynchronously in the background (the workflow will move on immediately while the Process Action runs).
If the process prints output to standard in (stdin), standard out (stdout), or standard error (stderr), you can configure the Process Action to store that output in a file, or you can simply direct that output to the engine’s standard out / in / error.
You can set the process’s working directory and its environment, allowing you to control when and how the process is executed.
You can indicate whether the process should be terminated if a user interrupts the workflow, if a signal is raised, or if the action times out.
You can also configure if (and when) the Process Action should invoke Flux’s Error Handling mechanism by setting an exit code or codes that indicate that an error has occurred.
Properties
Asynchronous
Indicates whether this process action waits synchronously — or does not wait (asynchronous) — for the newly spun-off process to finish running before this action completes. By default, the newly spun-off process runs synchronously, until it completes, before this process action finishes.
Command and Working Directory
As its name indicates, the command property specifies the command line for the native process. The Process Action can run any command that you are able to run from the command line.
The working directory is the directory where the command will run. By default, Flux will use the Java working directory as the working directory. Usually, this means that the command will run from the directory where Flux (or, more specifically, the JVM) was started. If you set the working directory property, however, it will override the Java working directory, allowing you to specify exactly which directory the Process Action should use. You can also change the default Java working directory by setting the “user.dir” system property.
Flux searches for executables before setting the working directory. This means that Flux will first attempt to locate the specified executable from the path (that is, the system’s PATH environment variable), then, once it is found, execute it as though it were running in the specified working directory.
In other words, this means that to invoke an executable using the Process action, you must either a) specify an executable that is available from the path (the system’s PATH environment variable), b) specify the full path to the command, or c) invoke the command shell or command prompt, then pass in the name of the executable that should be invoked.
Examples (Unix / Linux environments):
| Command | Working Directory | Description |
|---|---|---|
| myProcess.sh | /home/myWorkingDirectory | Execute the script myProcess.sh (which must be available on the path) from the directory /home/myWorkingDirectory. |
| /home/processes/myProcess.sh | /home/myWorkingDirectory | Execute the script located at /home/processes/myProcess.sh from the directory /home/myWorkingDirectory. |
| myProcess.sh | <null> | Execute the script myProcess.sh (which must be available on the path) from the directory where the JVM was started. |
| sh myProcess.sh | /home/processes | Invoke the command shell to execute the script located at /home/processes/myProcess.sh (this would be the same as running the command “sh myProcess.sh” from the directory /home/processes). |
| sh -c | <null> | Creates a shell in which you can run your command. This is useful when using utilities such as “tar” with wildcards in the arguments. The shell is what resolves the wildcards, not the actual utility. When using a shell you would then need to add your actual process commands (for example “tar -cf test.tar.gz *.txt”) as a single argument in the Process Action’s Command Arguments property since that is the argument to the shell command. |
Examples (Windows environments)
| Command | Working Directory | Description |
|---|---|---|
| myProcess.bat | C:\home\myWorkingDirectory | Execute the script myProcess.bat (which must be available on the path) from the directory C:\home\myWorkingDirectory. |
| C:\home\processes\myProcess.bat | C:\home\myWorkingDirectory | Execute the script located at C:\home\processes\myProcess.bat in the directory C:\home\myWorkingDirectory. |
| myProcess.bat | <null> | Execute the script myProcess.bat (which must be available on the path) from the directory where the JVM was started. |
| cmd /c myProcess.bat | C:\home\processes | Invoke the command prompt to execute the script located at C:\home\processes\myProcess.bat (this would be the as running the command “cmd /c myProcess.bat” from the directory C:\home\processes). |
Example workflows that demonstrate the Process Action’s usage can be found in /examples/end_users/process_action (for an example using the Flux Designer) and /examples/software_developers/process (using Java code).
Passing Arguments to the Command
To pass an argument to the command that Flux is invoking, you can simply supply it directly on the command line (exactly as you would if you were invoking the command outside of Flux). Flux supports both static arguments and, using variable substitution, arguments that are gathered dynamically from the workflow at runtime. This allows you, for example, to invoke your command using a file name that was gathered from a previous step in the workflow.
Examples:
| Command | Description |
|---|---|
| /home/processes/myProcess.sh myFile.txt | Invoke the script located at /home/processes/myProcess.sh with the argument “myFile.txt”. |
| /home/processes/myProcess.sh ${filename} | Invoke the script located at /home/processes/myProcess.sh, passing in the value of the flow context variable “filename” as an argument (dynamically substituted at workflow runtime). |
| /home/processes/myProcess.sh ${runtime filename} | Invoke the script located at /home/processes/myProcess.sh, passing in the value of the runtime variable “filename” as an argument (dynamically substituted at workflow runtime). |
For cases where you must use runtime data mapping to supply an argument to the command (for example, if you are using a complex return value from a previous step as an argument), you must Command Properties, as runtime data mapping would overwrite the command value if you attempted to map the argument directly to the command. For most cases, however, variable substitution is the most suitable method for passing an argument to the command.
To see an example of passing arguments to a command using the Flux Designer, refer to the /examples/end_users/process_action directory under the Flux installation directory. For an example using Java code, refer to /examples/software_developers/individual_processing.
Running the Command as a Different Operating System User
By default, Process Actions execute commands as the user who started the Flux engine or agent where the Process Action runs. Often, though, a Process Action must execute a command as a different user. In those cases, you can modify the Process Action’s command property to take advantage of native commands on the Operating System and run using the rights of the desired user, while still maintaining the integrity of your existing security restrictions.
On Windows systems, you can use “runas” to execute a command as a different user. For example, if wanted to run the program notepad.exe as the user “myuser” in the domain “mydomain”, you would set the Process Action’s command property like so:
runas /savecred /user:mydomain\myuser notepad.exe
User names can be entered in the form USER@DOMAIN, or DOMAN\USER as above. As a security measure, Windows requires that the user’s password be entered each time you use the “runas” command. By adding the “/savecred” argument to the command, you allow Windows to save the user credentials, so the user will only have to provide the password the first time the command is run (this means that you will need to supply the user credentials by running this command once outside of Flux, before Flux can invoke the “runas” command from a Process Action).
On Unix / Linux systems, “sudo” allows you to execute a command as a different user. Before you can use “sudo” in a Process Action, however, you must configure it with the necessary restrictions. To do so, edit the user privileges section of the file /etc/sudoers to provide appropriate permissions for the user who will run the Process Action (that is, the user account that is used to start the engine or agent).
For example, adding the following line to the sudoers file would allow the user “flux” to run a command as the user “batch” without being prompted for a password:
flux ALL=(batch) NOPASSWD:ALL
Once the sudoers file has been edited, any Process Action that is run under the flux user will be able to execute commands as the batch user with the “sudo” command. Including the following in a Process Action’s command property would execute the /opt/batch/report script as the user batch:sudo -u batch /opt/batch/reportIt is also possible to edit the sudoers file to only provide permission for certain commands under a certain user account. For example, including the following line in the sudoers file would give the user flux permission to only execute the /opt/batch/report script as the user batch:flux ALL=(batch) NOPASSWD:/opt/batch/reportUsing the techniques described above, Process Actions can be set up to run commands as different users while still maintaining the integrity of existing security restrictions.
Environment
The environment properties used by the native process. This allows you to specify environment properties and values that can be accessed from within the native process at runtime.
If this property is not set, the native process will (by default) inherit the environment settings of Flux itself (that is to say – whatever environment properties were used to start the Flux process on the Operating System). If this property is set, however, the Flux environment will not be passed to the native process – if you do set the environment properties here, be sure to set all environment properties required by your native process.
This property is useful when a native process depends on an environment variable being set in order to run. For example – the Windows command processor and other Windows processes typically need the “SystemRoot” property set in order to run. You can use this property to specify example what value the property should take when the native process runs.
This property is specified as a map of environment property names and their values. As an example, if you wanted to set the “SystemRoot” property to “C:\windows”, you would use the following map:
SystemRoot C:\windows
In some environments, it may not be possible for Flux to automatically populate the environment properties using its own settings. For this reason, if you plan to use the automatic environment inheritance feature of Flux, we recommend testing in your own environment to be sure it will work.
For example, the Windows command processor and other Windows programs need the “SystemRoot” property to be set in order to run correctly. Typically, this property is set to “c:\windows”
Error Condition
An expression that Flux can use to check whether the process action’s result code indicates that the process has failed. If the condition is satisfied, the action will throw an exception and invoke the standard error handling mechanisms.
In the error condition, the string “RESULT” is automatically substituted with the exit code of the invoked process (the value of the Process Action Result’s result field). So, for example, if the error condition was “result = 1” and the process returned an exit code of 1, the error condition would be satisfied and the action would throw an error.
The “RESULT” string is case-sensitive and must be entered in all-caps as shown in this document.
The syntax for the error condition is similar to that used for condition flows - however, the error condition can only be used to compare the result value (exit code) of the invoked process. The error condition cannot compare the result value against the value of a flow context variable, so if this is necessary you must use a conditional flow instead of the error condition.
By default, the error condition is undefined and the process is always deemed to have succeeded, regardless of the exit code.
See the table below for a complete list of supported operators.
Examples
| Operator | Description | Example | Result |
|---|---|---|---|
| = | Equals | RESULT = 1 | Throw an exception if the exit code equals 1. |
| > | Greater Than | RESULT > 0 | Throw an exception if the exit code is greater than 0. |
| < | Less Than | RESULT < 1 | Throw an exception if the exit code is less than 1. |
| >= | Greater Than or Equal To | RESULT >= 1 | Throw an exception if the exit code is greater than or equal to 1. |
| <= | Less Than or Equal To | RESULT <= 1 | Throw an exception if the exit code is less than or equal to 1. |
| <> | Not Equal To | RESULT <> 0 | Throw an exception if the exit code is any value other than 0. |
| + | Addition | RESULT + 1 = 2 | Throw an exception if the exit code plus 1 equals 2 (the exit code is 1). |
| - | Subtraction | RESULT - 1 = 1 | Throw an exception if the exit code minus 1 equals 1 (the exit code is 1). |
| * | Multiplication | RESULT * 2 = 2 | Throw an exception if the exit code times 2 equals 2 (the exit code is 1). |
| / | Division | RESULT / 2 = 1 | Throw an exception if the exit code divided by 2 equals 1 (the exit code is 2). |
| % | Wildcard character for use with LIKE | (see below) | (see below) |
| LIKE | Search for a pattern | RESULT LIKE ‘%9’ | Throw an exception if the exit code is a number whose last digit is 2 (2, 62, 112, etc.) |
| NOT | TRUE if the conditional expression evaluates to FALSE | NOT RESULT = 1 | Throw an exception if the exit code is not equal to 1. |
| AND | TRUE if all conditional expressions evaluate to TRUE | RESULT > 0 AND RESULT < 10 | Throw an exception if the exit code is a value between 0 and 10. |
| OR | TRUE if any conditional expression evaluates to TRUE | RESULT = 1 OR RESULT = 99 | Throw an exception if the exit code equals 1 or 99. |
Destroy On Signal
Process Actions have the ability to be aborted upon receiving a Flux signal. The Process Action’s action property Destroy On Signal controls this functionality. If the “Destroy On Signal” property is set to true, a native process being run by a Process Action will be aborted upon the reception of any Flux signal. The mechanism of this feature is very similar to the Destroy On Timeout feature, also available on Process Actions.
For example, suppose Process Action A in a workflow runs longer than Process Action B in a parallel flow within the same workflow. A signal can be raised in the outgoing flow of Process Action B that terminates the native process being run by Process Action A. After Process Action A’s native process is terminated, Process Action A will conditionally branch into a predefined signal flow where any further action may be taken. The screenshot below shows the workflow layout for this functionality.
Results
The Process Action returns its results in the flow context variable “result”. These results include the exit code of the invoked process, plus the stdout and stderr output:
| Flow Context Variable | Field | Java Type | Description | Prescript / Postscript Example |
|---|---|---|---|---|
| RESULT | result | int | The process’s exit code, as returned by the operating system, if the process is executed synchronously. | int exitCode = flowContext.get("RESULT").result; System.out.println("Exit code: " + exitCode); |
| RESULT | stderr | String | The standard error (stderr) from the completed process, if the process is executed synchronously. | String stderr = flowContext.get("RESULT").stderr; System.out.println("Standard error: " + stderr); |
| RESULT | stdout | String | The standard output (stdout) from the completed process, if the process is executed synchronously. | String stdout = flowContext.get("RESULT").stdout; System.out.println("Standard out: " + stdout); |
Passing Results with a Runtime Data Map
You can use a Runtime Data Map to copy one of the result fields into a new variable (for future reference or to reuse the data later in the workflow).
To copy a result field, you can use a data map like:
Process Actions and Agents
In the event that agents are enabled, a process action passes off its work to any one of the running free agents. To specify which agent the process action should run on, the “Agent Pool” action property must be specified. If no value is found for this property, the process runs on the engine’s host. If a value is defined for that property, the process will run on the first free agent running in the specified pool. To run the process on any free agent in any pool, use the value “*” for the “Agent Pool” action property.
Running a Process on a Remote Machine without Agents (Agentless Scheduling)
In some cases, you can use SSH as an alternative to Flux Agents. Like Agents, SSH allows you to remotely log in and execute a script or a command on a remote machine, but unlike Agents, SSH does not require you to install any additional Flux software on the remote machine.
This technique of using SSH instead of agents is called agentless scheduling.
To use SSH, you must first have the public key of the Flux machine (the server where the Flux engine runs) runs appended to the authorized keys of the target host (a Unix or Linux where you will run your command or process). A quick and simple way to set this up is to create a public key for the Flux machine (as described in this IBM article). The command to create the public key looks like this:
ssh-keygen -t rsa
After entering this command, follow the on-screen instructions, but do not set a password when prompted (if you set a password here, you would need to enter a password each time you wanted to use the key). This will create a private and a public key for your server.
Now, you will need to copy the contents of the public key file (.ssh/id_rsa.pub, located under your user’s home directory), and append those contents to the .ssh/authorized_keys file on the remote host for the user you want to use when logging in.
You will need to repeat this process to append your public key file for each machine that you want to be able to access using SSH. Once you have done this, Flux will be able to run remote commands using SSH without requiring interactive logins (basically, this makes the Flux server a trusted host on each target host and allowed the Flux server to remotely perform operations).
Once the SSH keys are set up, it is simple to set up the Process Action to invoke a remote command. You only need to specify two properties: Command, which should be set to “ssh <user>@<target host>”, and Command Arguments, which will look like “<command>;<command>”. For example:
- Command: ssh myuser@remotehost
- Command Arguments: df;uptime
Likewise, to invoke a script, you use the same command but give the Command Argument “sh <script name/path>” instead, like so:
- Command: ssh
@ - Command Arguments: sh test.sh
Running PowerShell Scripts
Some versions of Windows contain a known bug that prevents PowerShell from returning control to the JVM after executing a script.
If you are running a PowerShell script and you find that your workflow is stuck in the FIRING state, you can use the following simple workaround to allow Flux to continue executing after your script runs:
- Set the “Command” property on your process action to cmd \C.
- Set the “Working Directory” property to the directory where your PS1 script file is stored.
-
dd a “Command Argument” and enter **“echo . powershell C:\path\to\my\script.ps1”** (include the double-quotes in the argument).
This should allow powershell to return control to Flux after running the script.
PowerShell Scripts with Spaces in the Path
If a powershell script has a space in its path, you can escape the space using the ` character. For example:
"echo . | C:\Documents` and` Settings\script.ps1"
Permissions
Flux inherits its file system and script/process execution permissions from the user who starts the Flux process on the OS (the user who starts the Flux service, engine startup script, or Java process containing Flux).
If you are attempting to execute a script and an error like the following appears:
java.io.IOException: error=13, Permission denied
This indicates that the user who started Flux is missing either read or execute permissions for the script.
If you are launching the script through an absolute path, like so:
/path/to/my.bat
Make sure that the user who starts Flux has all of the following permissions:
- Read access to all folders in the path.
- Read access to the script itself.
- Execute permission on the script.
If you are using a relative path, like the following:
../path/to/my.bat
In addition to the permissions listed above, check that the relative path is available starting from the directory where the JVM is created (usually, the Flux installation folder).