Data Prep Runtime Variables can be created in Pipelines and Mapping Groups as needed to support more versatile transformations. A Pipeline Runtime Variable can be passed a value at the point of execution of the Pipeline. Transformations defined within the Pipeline can leverage the value of the Variable in the execution. A Mapping Group Runtime Variable can also be passed a value but requires a related Pipeline Runtime Variable to be defined.
Creating Runtime Variables
To create a Runtime Variable, select the Variables icon (<>) in the Columns pane of the Pipeline or Mapping Group and then select Create Variables.
In the Edit Variables screen, select the Data Type of the Variable. Specify the Display Name of the Variable. Optionally, the Variable can be flagged as Required, and a Default Value can be defined. Click Save to create the Variable.
❗ The Variable data type must match the column type in which the Variable will be used. If the variable needs to be used across multiple fields of different data types, create additional Variables with the correct data type(s).
📚 The Default Value allows a value to be specified that will be used in any Transformation or Mapping Group if the Variable value is not specified at execution time.
⚠️ When a Variable is flagged as Required, a value must be specified for the Variable when the Pipeline is executed. Failure to specify a Value will result in the Pipeline erroring when executed even if a Default Value is specified.
To add additional Variables, click the blue plus sign in the Edit Variables pane.
Editing Runtime Variables
Data Prep Runtime Variables can be modified anytime after having been defined. To modify a Data Prep Runtime Variable, select the Variables icon (<>) in the Columns pane of the Pipeline or Mapping Group and then select Edit Variables.
Existing Variables can be edited as needed, including updating the Data Type, Display Name, Required flag, and Default Value. If modifying the Data Type of a Runtime Variable, please update any Transformations or Mapping Groups that utilize the Variable. After updating a Variable, be sure to save the changes.
Deleting Runtime Variables
Data Prep Runtime Variables can be deleted at any time. To delete a Data Prep Runtime Variable, select the Variables icon (<>) in the Columns pane of the Pipeline or Mapping Group and then select Edit Variables. Click the trash can icon next to the Variable to be deleted. Click Save, and the Variable deleted is removed.
Be sure to update any Transformations or Mapping Groups using the deleted Runtime Variable.
⚠️ Please exercise caution when deleting Data Prep Runtime Variables. There is not a confirmation dialogue when deleting Variables. Any Transformations or Mapping Groups that use a deleted Variable and have not been updated will utilize the internal Variable ID (GUID) as the value for the Variable when executed. Creating a new Variable with the same name as the deleted Variable will not resolve this as the new Variable has a different internal ID than the deleted Variable.