Data Prep Variables can be utilized in Pipeline Transformations when defining a Transformation. When a Transformation has a field that enables input such as the Value field of the Insert Column Transformation, any of the Data Prep Variable types can be used to specify the Value. If applicable to the Transformation, multiple Variables can be utilized or combined with a specific text or numeric value.

To utilize a Data Prep Variable, click in the input field of the Transformation or click the Variable selector icon (<>) in the input field. The Variable selector will appear providing access to each of the Variables available in the Pipeline. The Variable Type (System, Column, Runtime) is indicated for each of the available Variables. You can also search for a Variable using the search box in the selector.

Variable Selector

Click the Variable name to add the Variable to the Transformation. To hide the Variable selector, click the Variable selector icon (<>) and the selector will be collapsed.

Variable in use

Multiple Variable values can be assigned, combined with a static value, or a combination thereof.

Stacked Variables

Previewing Transformations using Runtime Variables

The data preview pane of Pipelines will exhibit different behaviors depending on how a Runtime Pipeline Variable is defined.

Runtime Variables defined as Required

When a Runtime Variable is defined as required, then a preview value (which should not be confused with a default value) must be set in order for the Transformation preview to function. Failure to set a preview value will result in an error.

Preview error

To set a preview value, open the Variables editor in the Columns pane. Enter a preview value for the Variable(s) specified as required and click Set Values.

Set preview value

The preview pane will display the results of the Transformation using the preview value specified.

Preview Value

Runtime Variables defined as not Required

When a Runtime Variable is not flagged as required, then a preview value is not required in order for the preview pane to display the result of a Transformation using the Variable. If a preview value is not specified for the Variable, the value null will display in the preview pane for the Column impacted by the Transformation using the Variable.

Preview: Runtime Variables flagged as not Required with no preview value
Did this answer your question?