Download OpenAPI specification:Download
To enable programmatic control over its objects, the Trifacta Platform supports a range of REST API endpoints across its objects. This section provides an overview of the API design, methods, and supported use cases.
Most of the endpoints accept JSON
as input and return JSON
responses.
This means that you must usually add the following hearders to your request:
Content-type: application/json
Accept: application/json
The term resource
refers to a single type of object in the Trifacta Platform metadata. An API is broken up by its endpoint's corresponding resource.
The name of a resource is typically plural, and expressed in camelCase. Example: jobGroups
.
Resource names are used as part of endpoint URLs, as well as in API parameters and responses.
The platform supports Create, Read, Update, and Delete operations on most resources. You can review the standards for these operations and their standard parameters below.
Some endpoints have special behavior as exceptions.
To create a resource, you typically submit an HTTP POST
request with the resource's required metadata in the request body.
The response returns a 201 Created
response code upon success with the resource's metadata, including its internal id
, in the response body.
An HTTP GET
request can be used to read a resource or to list a number of resources.
A resource's id
can be submitted in the request parameters to read a specific resource.
The response usually returns a 200 OK
response code upon success, with the resource's metadata in the response body.
If a GET
request does not include a specific resource id
, it is treated as a list request.
The response usually returns a 200 OK
response code upon success, with an object containing a list of resources' metadata in the response body.
When reading resources, some common query parameters are usually available. e.g.:
/v4/jobGroups?limit=100&includeDeleted=true&embed=jobs
Query Parameter | Type | Description |
---|---|---|
embed | string | Comma-separated list of objects to include part of the response. See Embedding resources. |
includeDeleted | string | If set to true , response includes deleted objects. |
limit | integer | Maximum number of objects to fetch. Usually 25 by default |
offset | integer | Offset after which to start returning objects. For use with limit query parameter. |
Updating a resource requires the resource id
, and is typically done using an HTTP PUT
or PATCH
request, with the fields to modify in the request body.
The response usually returns a 200 OK
response code upon success, with minimal information about the modified resource in the response body.
Deleting a resource requires the resource id
and is typically executing via an HTTP DELETE
request. The response usually returns a 204 No Content
response code upon success.
Resource names are plural and expressed in camelCase.
Resource names are consistent between main URL and URL parameter.
Parameter lists are consistently enveloped in the following manner:
{ "data": [{ ... }] }
Field names are in camelCase and are consistent with the resource name in the URL or with the embed URL parameter.
"creator": { "id": 1 },
"updater": { "id": 2 },
When reading a resource, the platform supports an embed
query parameter for most resources, which allows the caller to ask for associated resources in the response.
Use of this parameter requires knowledge of how different resources are related to each other and is suggested for advanced users only.
In the following example, the sub-jobs of a JobGroup are embedded in the response for jobGroup=1:
http://example.com:3005/v4/jobGroups/1?embed=jobs
The Trifacta Platform uses HTTP response codes to indicate the success or failure of an API request.
HTTP Status Code (client errors) | Notes |
---|---|
400 Bad Request | Potential reasons:
|
403 Forbidden | Incorrect permissions to access the Resource. |
404 Not Found | Resource cannot be found. |
410 Gone | Resource has been previously deleted. |
415 Unsupported Media Type | Incorrect Accept or Content-type header |
Each request has a request identifier, which can be found in the response headers, in the following form:
x-trifacta-request-id: <myRequestId>
ℹ️ NOTE: If you have an issue with a specific request, please include the
x-trifacta-request-id
value when you contact support
✅ TIP: You can use the request identifier value to scan the logs to identify technical details for an issue with a specific request.
You can use a third party client, such as curl, HTTPie, Postman or the Insomnia rest client to test the Trifacta API.
⚠️ When testing the API, bear in mind that you are working with your live production data, not sample data or test data.
Note that you will need to pass an API token with each request.
For e.g., here is how to run a job with curl:
curl -X POST 'http://example.com:3005/v4/jobGroups' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <token>' \
-d '{ "wrangledDataset": { "id": "<recipe-id>" } }'
Using a graphical tool such as Postman or Insomnia, it is possible to import the API specifications directly:
Note that with Postman, you can also generate code snippets by selecting a request and clicking on the Code button.
ℹ️ NOTE: Each request to the Trifacta Platform must include authentication credentials.
API access tokens can be acquired and applied to your requests to obscure sensitive Personally Identifiable Information (PII) and are compliant with common privacy and security standards. These tokens last for a preconfigured time period and can be renewed as needed.
You can create and delete access tokens through the Settings area of the application. With each request, you submit the token as part of the Authorization header.
Authorization: Bearer <tokenValue>
As needed, you can create and use additional tokens. There is no limit to the number of tokens you can create. See API Access Token API for more information.
Security Scheme Type | HTTP |
---|---|
HTTP Authorization Scheme | bearer |
An object used to provide a simpler and more secure way of accessing the REST API endpoints of the Trifacta Platform. Access tokens limit exposure of clear-text authentication values and provide an easy method of managing authentication outside of the browser. See the Authentication section for more information.
Create an API Access Token. See the Authentication section for more information about API Access Token.
⚠️ API tokens inherit the API access of the user who creates them. Treat tokens as passwords and keep them in a secure place.
This request requires you to be authenticated.
lifetimeSeconds required | integer Lifetime in seconds for the access token. Set this value to -1 to create a non-expiring token. |
description | string User-friendly description for the access token |
{- "lifetimeSeconds": -1,
- "description": "API access token description"
}
{- "tokenValue": "eyJ0b2tlbklkIjoiYmFiOTA4ZjctZGNjMi00OTYyLTg1YmQtYzFlOTZkMGNhY2JkIiwic2VjcmV0IjoiOWIyNjQ5MWJiODM4ZWY0OWE1NzdhYzYxOWEwYTFkNjc4ZmE4NmE5MzBhZWFiZDk3OGRlOTY0ZWI0MDUyODhiOCJ9",
- "tokenInfo": {
- "tokenId": "0bc1d49f-5475-4c62-a0ba-6ad269389ada",
- "description": "API access token description",
- "expiredAt": "2019-08-24T14:15:22Z",
- "createdAt": "2019-08-24T14:15:22Z",
- "lastUsed": null
}
}
{- "data": [
- {
- "tokenId": "0bc1d49f-5475-4c62-a0ba-6ad269389ada",
- "description": "API access token description",
- "expiredAt": "2019-08-24T14:15:22Z",
- "createdAt": "2019-08-24T14:15:22Z",
- "lastUsed": null
}
]
}
Get an existing api access token
tokenId required | string Example: 0bc1d49f-5475-4c62-a0ba-6ad269389ada |
{- "tokenId": "0bc1d49f-5475-4c62-a0ba-6ad269389ada",
- "description": "API access token description",
- "expiredAt": "2019-08-24T14:15:22Z",
- "createdAt": "2019-08-24T14:15:22Z",
- "lastUsed": null
}
An object containing information for accessing AWS S3 storage, including details like defaultBucket, credentials, etc.
Create a new AWS config
credentialProvider required | string Enum: "default" "temporary" "instance"
|
defaultBucket | string Default S3 bucket where user can upload and write results |
extraBuckets | Array of strings |
role | string AWS IAM Role, required when credential provider is set to temporary |
key | string AWS key string, required when credential provider is set to default |
secret | string AWS secret string, required when credential provider is set to default |
personId | integer When creating an AWS configuration, an administrator can insert the personId parameter to assign the configuration to the internal identifier for the user. If this parameter is not included, the AWS configuration is assigned to the user who created it. |
workspaceId | integer When creating an AWS configuration, an administrator can insert the workspaceId parameter to assign the configuration to the internal identifier for the workspace. |
{- "defaultBucket": "bucketName",
- "extraBuckets": [
- "bucket1"
], - "credentialProvider": "default",
- "role": "arn:aws:iam::xxxxxxxxxxxxx:role/sample-role",
- "key": "string",
- "secret": "string",
- "personId": 1,
- "workspaceId": 1
}
{- "defaultBucket": "bucketName",
- "extraBuckets": [
- "bucket1"
], - "credentialProvider": "default",
- "role": "arn:aws:iam::xxxxxxxxxxxxx:role/sample-role",
- "credential": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "activeRoleId": 1
}
List existing aws configs
ℹ️ NOTE: Admin role is required to use this endpoint.
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "data": [
- {
- "defaultBucket": "bucketName",
- "extraBuckets": [
- "bucket1"
], - "credentialProvider": "default",
- "role": "arn:aws:iam::xxxxxxxxxxxxx:role/sample-role",
- "credential": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "activeRoleId": 1
}
]
}
The request body contains the parameters of the awsConfigs object that you wish to modify. You do not have to include parameters that are not being modified.
The following changes the default bucket for the AWS configuration object.
{ "defaultBucket": "testing2" }
id required | integer |
credentialProvider required | string Enum: "default" "temporary" "instance"
|
id | integer unique identifier for this object. |
defaultBucket | string Default S3 bucket where user can upload and write results |
extraBuckets | Array of strings |
role | string AWS IAM Role, required when credential provider is set to temporary |
key | string AWS key string, required when credential provider is set to default |
secret | string AWS secret string, required when credential provider is set to default |
personId | integer When creating an AWS configuration, an administrator can insert the personId parameter to assign the configuration to the internal identifier for the user. If this parameter is not included, the AWS configuration is assigned to the user who created it. |
workspaceId | integer When creating an AWS configuration, an administrator can insert the workspaceId parameter to assign the configuration to the internal identifier for the workspace. |
{- "id": 1,
- "defaultBucket": "bucketName",
- "extraBuckets": [
- "bucket1"
], - "credentialProvider": "default",
- "role": "arn:aws:iam::xxxxxxxxxxxxx:role/sample-role",
- "key": "string",
- "secret": "string",
- "personId": 1,
- "workspaceId": 1
}
{- "defaultBucket": "bucketName",
- "extraBuckets": [
- "bucket1"
], - "credentialProvider": "default",
- "role": "arn:aws:iam::xxxxxxxxxxxxx:role/sample-role",
- "credential": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "activeRoleId": 1
}
An object containing the AWS IAM Role ARN for authenticating aws resources when using role-base authentication, this object belongs to an AwsConfig.
Create an aws role. If neither personId nor workspaceId is provided, create the role for the request user
ℹ️ NOTE: Admin role is required to use this endpoint.
role required | string |
personId | integer |
workspaceId | integer When creating an AWS role, an administrator can insert the workspaceId parameter to assign the configuration to the internal identifier for the workspace. |
{- "role": "string",
- "personId": 1,
- "workspaceId": 1
}
{- "id": 1,
- "awsConfigId": 1,
- "role": "string",
- "createdFrom": "api",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deletedAt": "2019-08-24T14:15:22Z"
}
List AWS roles for a user
or workspace. If neither personId
nor workspaceId
is provided, list the roles associated with the request user.
personId | integer person id |
workspaceId | integer workspace id |
{- "data": {
- "id": 1,
- "awsConfig": {
- "id": 1
}, - "role": "string",
- "createdFrom": "api",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deletedAt": "2019-08-24T14:15:22Z"
}
}
Update an existing aws role
ℹ️ NOTE: Admin role is required to use this endpoint.
id required | integer |
personId | integer |
workspaceId | integer When creating an AWS role, an administrator can insert the workspaceId parameter to assign the configuration to the internal identifier for the workspace. |
role | string |
createdFrom | string Enum: "api" "idp" shows which means created the role |
createdAt | string <date-time> The time this object was first created. |
updatedAt | string <date-time> The time this object was last updated. |
deletedAt | string <date-time> The time this object was deleted. |
{- "personId": 1,
- "workspaceId": 1,
- "role": "string",
- "createdFrom": "api",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deletedAt": "2019-08-24T14:15:22Z"
}
{- "id": 1,
- "awsConfigId": 1,
- "role": "string",
- "createdFrom": "api",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deletedAt": "2019-08-24T14:15:22Z"
}
An object representing Trifacta's connection to an external data source. Connections can be used for import, publishing, or both, depending on type.
Create a new connection
vendor required | string String identifying the connection`s vendor |
vendorName required | string Name of the vendor of the connection |
type required | string Enum: "jdbc" "rest" Type of connection |
credentialType required | string Enum: "basic" "custom" "conf" "kerberosDelegate" "azureTokenSso" "kerberosImpersonation" "sshKey" "securityToken"
|
name required | string Display name of the connection. |
params required | object This setting is populated with any parameters that are passed to the source duringconnection and operations. For relational sources, this setting may include thedefault database and extra load parameters. |
ssl | boolean When |
description | string User-friendly description for the connection. |
disableTypeInference | boolean If set to false, type inference has been disabled for this connection. The default is true. When type inference has been disabled, the Trifacta Platform does not apply Trifacta types to data when it is imported. |
isGlobal | boolean If NOTE: After a connection has been made public, it cannot be made private again. It must be deleted and recreated. |
credentialsShared | boolean If |
host | string Host of the source |
port | integer Port number for the source |
Array of basic (object) or custom (object) or conf (object) or kerberosDelegate (object) or azureTokenSso (object) or kerberosImpersonation (object) or sshKey (object) or securityToken (object) (credentialsInfo) [ items ] If present, these values are the credentials used to connect to the database. |
{- "vendor": "oracle",
- "vendorName": "oracle",
- "type": "jdbc",
- "name": "example_oracle_connection",
- "description": "This is an oracle connection",
- "disableTypeInference": false,
- "isGlobal": false,
- "credentialsShared": false,
- "host": "my_oracle_host",
- "port": 1521,
- "params": {
- "service": "my_oracle_service"
}, - "credentialType": "basic",
- "credentials": [
- {
- "username": "my_oracle_username",
- "password": "my_oracle_password"
}
]
}
{- "vendor": "oracle",
- "vendorName": "oracle",
- "type": "jdbc",
- "credentialType": "basic",
- "ssl": true,
- "name": "example_oracle_connection",
- "description": "string",
- "disableTypeInference": true,
- "isGlobal": true,
- "credentialsShared": true,
- "host": "example.oracle.test",
- "port": 1521,
- "id": "55",
- "uuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "credentials": [
- {
- "username": "string",
- "password": "string"
}
], - "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "params": {
- "database": "dev"
}
}
List existing connections
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
sharedRole | string Which type of role to list the connections |
{- "data": [
- {
- "vendor": "oracle",
- "vendorName": "oracle",
- "type": "jdbc",
- "credentialType": "basic",
- "ssl": true,
- "name": "example_oracle_connection",
- "description": "string",
- "disableTypeInference": true,
- "isGlobal": true,
- "credentialsShared": true,
- "host": "example.oracle.test",
- "port": 1521,
- "id": "55",
- "uuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "credentials": [
- {
- "username": "string",
- "password": "string"
}
], - "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "params": {
- "database": "dev"
}
}
]
}
Count existing connections
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
sharedRole | string Which type of role to count the connections |
{- "count": 1
}
Get an existing connection
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "vendor": "oracle",
- "vendorName": "oracle",
- "type": "jdbc",
- "credentialType": "basic",
- "ssl": true,
- "name": "example_oracle_connection",
- "description": "string",
- "disableTypeInference": true,
- "isGlobal": true,
- "credentialsShared": true,
- "host": "example.oracle.test",
- "port": 1521,
- "id": "55",
- "uuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "credentials": [
- {
- "username": "string",
- "password": "string"
}
], - "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "params": {
- "database": "dev"
}
}
Update an existing connection
id required | integer |
host | string Host of the source |
port | integer Port number for the source |
ssl | boolean When |
description | string User-friendly description for the connection. |
disableTypeInference | boolean If set to false, type inference has been disabled for this connection. The default is true. When type inference has been disabled, the Trifacta Platform does not apply Trifacta types to data when it is imported. |
name | string Display name of the connection. |
params | object This setting is populated with any parameters that are passed to the source duringconnection and operations. For relational sources, this setting may include thedefault database and extra load parameters. |
isGlobal | boolean If NOTE: After a connection has been made public, it cannot be made private again. It must be deleted and recreated. |
credentialsShared | boolean If |
Array of basic (object) or custom (object) or conf (object) or kerberosDelegate (object) or azureTokenSso (object) or kerberosImpersonation (object) or sshKey (object) or securityToken (object) (credentialsInfo) [ items ] If present, these values are the credentials used to connect to the database. | |
credentialType | string Enum: "basic" "custom" "conf" "kerberosDelegate" "azureTokenSso" "kerberosImpersonation" "sshKey" "securityToken"
|
vendor | string String identifying the connection`s vendor |
{- "host": "example.oracle.test",
- "port": 1521,
- "ssl": true,
- "description": "string",
- "disableTypeInference": true,
- "name": "example_oracle_connection",
- "params": {
- "database": "dev"
}, - "isGlobal": true,
- "credentialsShared": true,
- "credentials": [
- {
- "username": "string",
- "password": "string"
}
], - "credentialType": "basic",
- "vendor": "oracle"
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Get the list of all vendors of connections that are supported in the instance of the platform.
ℹ️ NOTE: Additional vendors can become available based on the deployment of a set of configuration files for the vendor. Adding vendors that are not on this list may require Trifacta Customer Success Services. For more information on the available connections, see Connection Types.
[- {
- "name": "string",
- "displayName": "string",
- "type": "string",
- "category": "relational",
- "credentialTypes": [
- "basic"
], - "operation": "import",
- "connectionParameters": [
- {
- "name": "string",
- "displayName": "string",
- "type": "string",
- "required": true,
- "category": "string",
- "default": "string"
}
]
}
]
Get the list of all credential types that are supported in the instance of the platform.
[- {
- "name": "string",
- "displayName": "string",
- "credentialParameters": [
- {
- "name": "string",
- "displayName": "string",
- "type": "string",
- "required": true,
- "obfuscated": true,
- "accessor": "email"
}
]
}
]
An internal object representing the relationship between a Connection and any Person objects with which it is shared.
Create a new connection permission
id required | integer |
required | Array of personObjectWithRole (object) or personIdWithRole (object)[ items ] |
{- "data": [
- {
- "person": {
- "id": 1
}, - "role": "owner",
- "policy": "string"
}
]
}
{- "data": [
- {
- "role": "owner",
- "person": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
]
}
Get existing connection permissions
id required | integer |
{- "data": [
- {
- "name": "string",
- "email": "string",
- "id": 1,
- "connectionPermission": {
- "role": "owner",
- "person": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}, - "isCreatedBy": true
}
]
}
Get an existing connection permission
id required | integer |
aid required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "role": "owner",
- "person": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
An internal object representing the relationship between a Person and an Azure Databricks cluster.
Update databricks access token for current user.
databricksAccessToken required | string |
{- "databricksAccessToken": "string"
}
{- "message": "string"
}
Admin can update databricks access token for user with id=personId.
ℹ️ NOTE: Admin role is required to use this endpoint.
personId required | integer |
databricksAccessToken required | string |
{- "personId": 1,
- "databricksAccessToken": "string"
}
{- "message": "string"
}
A versioned set of Releases.
A Deployment allows you to create a separation between your development and production environments. You can for e.g. develop Flows in a development instance and then import them to a deployment instance where they will be read-only.
You can override file paths or tables when importing Flow packages to a deployment instance using updateObjectImportRules and updateValueImportRules.
The Deployment Manager includes the tools to migrate your software between environments, manage releases of it, and separately control access to development and production flows. See the documentation for more details.
Create a new deployment
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
name required | string Display name of the deployment. |
{- "name": "Test Deployment"
}
{- "name": "Test Deployment",
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
List all deployments, including information about the latest Release in each deployment.
You can get all Releases for a Deployment by using embed
:
/v4/deployments/{id}?embed=releases
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "data": [
- {
- "name": "Test Deployment",
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "numReleases": 1,
- "latestRelease": {
- "notes": "string",
- "packageUuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "active": true,
- "deployment": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
}
]
}
Run the primary Flow in the active Release of the given Deployment.
The request body can stay empty. You can optionally pass parameters:
{
"runParameters": {
"overrides": {
"data": [{"key": "varRegion", "value": "02"}]
}
}
}
You can also pass Spark Options that will be used for the Job run.
{
"sparkOptions": [
{"key": "spark.executor.memory", "value": "4GB"}
]
}
You can also override each outputs in the Flow using the recipe name.
{
"overrides": {
"my recipe": {
"profiler": true,
"writesettings": [
{
"path": "<path_to_output_file>",
"action": "create",
"format": "csv",
"compression": "none",
"header": false,
"asSingleFile": false
}
]
}
}
}
An array of JobGroup results is returned. Use the flowRunId
if you want to track the status of the Deployment run. See Get Flow Run Status for more information.
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
x-execution-id | string Example: f9cab740-50b7-11e9-ba15-93c82271a00b Optional header to safely retry the request without accidentally performing the same operation twice. If a FlowRun with the same |
object (runParameterOverrides) Allows to override parameters that are defined in the flow on datasets or outputs for e.g. | |
Array of objects (outputObjectSparkOptionUpdateRequest) [ items ] | |
overrides | object Overrides for each of the output object. Use the recipe name to specify the overrides. |
{ }
{- "data": [
- {
- "id": 1,
- "flowRun": {
- "id": 1
}, - "jobs": {
- "data": [
- {
- "id": 1
}
]
}, - "jobGraph": {
- "edges": [
- {
- "source": 1,
- "target": 1
}
], - "vertices": [
- 1
]
}, - "reason": "Job started",
- "sessionId": "f9cab740-50b7-11e9-ba15-93c82271a00b"
}
]
}
Create a release for the specified deployment.
Release is created from a local ZIP
containing the package of the flow exported from the source system.
When importing a release, import-mapping rules are executed. These import rules allow you to replace the file location or the table names of different objects during the import for a deployment. See updateObjectImportRules and updateValueImportRules if you need to update the import rules.
This endpoint accept a multipart/form
content type.
Here is how to send the ZIP
package using curl.
curl -X POST http://example.com:3005/v4/deployments/:id/releases \
-H 'authorization: Bearer <api-token>' \
-H 'content-type: multipart/form-data' \
-F 'data=@path/to/flow-package.zip'
The response lists the objects that have been created.
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
folderId | integer |
{- "primaryFlowIds": [
- 1
], - "flows": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
], - "deletedObjects": { },
- "datasources": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
], - "flownodes": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}
], - "outputobjects": [
- {
- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "importRuleChanges": {
- "object": [
- { }
], - "value": [
- { }
]
}, - "release": { }
}
Get the list of releases for the specified deployment
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "data": [
- {
- "notes": "string",
- "packageUuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "active": true,
- "deployment": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}
Test importing flow package, applying all import rules that apply to this deployment, and return information about what objects would be created.
The same payload as for Import Deployment package is expected.
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
folderId | integer |
{- "primaryFlowIds": [
- 1
], - "flows": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
], - "deletedObjects": { },
- "datasources": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
], - "flownodes": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}
], - "outputobjects": [
- {
- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "importRuleChanges": {
- "object": [
- { }
], - "value": [
- { }
]
}, - "release": { }
}
Count existing deployments
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "count": 1
}
Get the specified deployment.
You can get all Releases for a Deployment by using embed
:
/v4/deployments/:id?embed=releases
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "id": 1,
- "name": "2013 POS",
- "createdAt": "2019-03-27T17:45:14.837Z",
- "updatedAt": "2019-03-27T17:45:14.837Z",
- "releases": {
- "data": [
- {
- "id": 1,
- "notes": "v01",
- "packageUuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "active": null,
- "createdAt": "2019-03-27T17:45:48.345Z",
- "updatedAt": "2019-03-27T17:46:24.675Z",
- "deployment": {
- "id": 1
}, - "creator": {
- "id": 2
}, - "updater": {
- "id": 2
}
}, - {
- "id": 2,
- "notes": "v02",
- "packageUuid": "ff8738c0-50b7-11e9-ba15-93c82271a00b",
- "active": true,
- "createdAt": "2019-03-27T17:46:24.671Z",
- "updatedAt": "2019-03-27T17:46:24.671Z",
- "deployment": {
- "id": 1
}, - "creator": {
- "id": 2
}, - "updater": {
- "id": 2
}
}
]
}, - "creator": {
- "id": 2
}, - "updater": {
- "id": 2
}
}
Update an existing deployment
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
name | string Display name of the deployment. |
{- "name": "Test Deployment"
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Patch an existing deployment
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
name | string Display name of the deployment. |
{- "name": "Test Deployment"
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Delete the specified deployment.
⚠️ Deleting a deployment removes all releases, packages, and flows underneath it. This step cannot be undone.
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
Get active outputs of the specified deployment. When the deployment is run, the listed outputs are generated.
This endpoint is useful if you only want to run a specific job in a deployment, or pass overrides.
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
{- "data": [
- {
- "outputObjectId": 1,
- "flowNodeId": 1,
- "recipeName": "string"
}
]
}
Create a list of object-based import rules for the specified deployment. Delete all previous rules applied to the same object.
ℹ️ NOTE: Import rules must be applied to individual deployments.
The generated rules apply to all Flows that are imported into the Deployment after they has been created.
The response contains any previously created rules that have been deleted as a result of this change.
You can also make replacements in the import package based on value mappings. See updateValueImportRules.
The following JSON array describes replacing the Connection specified by the UUID, which is a field on the connection object exported from the original platform instance. This Connection reference is replaced by a reference to connection ID 1 in the local platform instance and is applied to any release uploaded into the deployment after the rule has been created:
[
{
"tableName": "connections",
"onCondition": {
"uuid": "d75255f0-a245-11e7-8618-adc1dbb4bed0"
},
"withCondition": {"id": 1}
}
]
This example request includes replacements for multiple connection references.
ℹ️ NOTE: Rules are applied in the listed order. If you are applying multiple rules to the same object in the import package, the second rule must reference the expected changes applied by the first rule.
This type of replacement applies if the imported packages contain sources that are imported through two separate connections:
[
{
"tableName": "connections",
"onCondition": {
"uuid": "d75255f0-a245-11e7-8618-adc1dbb4bed0"
},
"withCondition": {"id": 1}
},
{
"tableName": "connections",
"onCondition": {
"uuid": "d552045e0-c314-22b5-9410-acd1bcd8eea2"
},
"withCondition": {"id": 2}
}
]
The response body contains any previously created rules that have been deleted as a result of this update.
If the update does not overwrite any previous rules, then no rules are deleted. So, the response looks like the following:
{"deleted": {"data": []}}
If you submit the request again, the response contains the rule definition of the previous update, which has been deleted. This example applies to the one-rule change listed previously:
{
"deleted": {
"data": [
{
"onCondition": {
"uuid": "d75255f0-a245-11e7-8618-adc1dbb4bed0"
},
"withCondition": {"id": 1},
"id": 1,
"tableName": "connections",
"createdAt": "2019-02-13T23:07:51.720Z",
"updatedAt": "2019-02-13T23:07:51.720Z",
"creator": {"id": 7},
"updater": {"id": 7},
"deployment": {"id": 4}
}
]
}
}
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
tableName required | string Name of the table to which the mapping is applied. |
onCondition required | object The matching object identifier and the specified literal or pattern to match. |
withCondition required | object The identifier for the object type,
as specified in by the |
[- {
- "tableName": "connections",
- "onCondition": {
- "uuid": "d75255f0-a245-11e7-8618-adc1dbb4bed0"
}, - "withCondition": {
- "id": 1
}
}
]
{- "deleted": {
- "data": [ ]
}
}
Create a list of value-based import rules for the specified deployment. Delete any previous rules applied to the same values.
ℹ️ NOTE: Import rules must be applied to individual deployments.
The generated rules apply to all flows that are imported into the Production instance after they have been created.
The response contains any previously created rules that have been deleted as a result of this change.
You can also make replacements in the import package based on object references. See updateObjectImportRules
The following JSON array describes a single replacement rule for the S3 bucket name. In this case, the wrangle-dev bucket name has been replaced by the wrangle-prod bucket name, which means data is pulled in the Production deployment from the appropriate S3 bucket.
ℹ️ NOTE: The executing user of any job must have access to any data source that is remapped in the new instance.
[
{
"type": "s3Bucket",
"on": "wrangle-dev",
"with": "wrangle-prod"
}
]
The following JSON array describes two replacements for the fileLocation values. In this case, rules are applied in succession.
ℹ️ NOTE: Rules are applied in the listed order. If you are applying multiple rules to the same object in the import package, the second rule must reference the expected changes applied by the first rule.
[
{
"type": "fileLocation",
"on": "klamath",
"with": "klondike"
},
{
"type": "fileLocation",
"on": "//dev//",
"with": "/prod/"
}
]
In the above:
klamath
in the path to the source with the following value: klondike
.
The second rule performs a regular expression match on the string /dev/
. Since the match is described using the regular expression syntax, the backslashes must be escaped. The replacement value is the following literal: /prod/
.Match Type | Example Syntax |
---|---|
string literal | {"on":"d75255f0-a245-11e7-8618-adc1dbb4bed0"} |
regular expression | {"on":"/[0-9a-zA-z]{8}-a245-11e7-8618-adc1dbb4bed0/"} |
This example request includes replacements for a database table and its path (database name) in a relational publication.
ℹ️ NOTE: Rules are applied in the listed order. If you are applying multiple rules to the same object in the import package, the second rule must reference the expected changes applied by the first rule.
This type of replacement applies if the imported packages contain sources that are imported through two separate connections:
[
{
"type": "dbTableName",
"on": "from_table_name",
"with": "to_table_name"
},
{
"type": "dbPath",
"on": "from_path_element",
"with": "to_path_element"
}
]
Type | Description |
---|---|
dbTableName | Replaces the name of the table in the source (on value) with the new table name to use (with value). |
dbPath | Replaces the path to the database in the source (on value) with the new path to use (with value). The value of dbPath is an array. So, the replacement rule is applied to each element of the array. in most cases, the number of elements is 1. If your path contains multiple elements, you should be careful in your use of regular expressions for remapping dbPath values. |
✅ TIP: The
on
parameter values can be provided as a regular expression.
The response body contains any previously created rules that have been deleted as a result of this update.
{"deleted": {"data": []}}
If you submit the request again, the response contains the rule definition of the previous update, which has been deleted.
{
"deleted": {
"data": [
{
"id": 1,
"type": "s3Bucket",
"on": "wrangle-dev",
"with": "wrangle-prod",
"createdAt": "2019-02-13T23:27:13.351Z",
"updatedAt": "2019-02-13T23:27:13.351Z",
"creator": {"id": 7},
"updater": {"id": 7},
"deployment": {"id": 2}
}
]
}
}
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
type required | string Enum: "fileLocation" "s3Bucket" "dbTableName" "dbPath" "host" "userinfo" The type of value import rule:
|
on required | string The specified literal or pattern to match. |
with required | string The replacement value or pattern. |
[- {
- "type": "s3Bucket",
- "on": "wrangle-dev",
- "with": "wrangle-prod"
}
]
{- "deleted": {
- "data": [ ]
}
}
An internal object representing the state of a Recipe at a given point in time.
Gets a summary of the history of given recipe edit. This includes information about the changes involved in each edit along the way, as well as the person who made the edit.
You can obtain the recipe for a given wrangledDataset by using:
GET v4/wrangledDatasets/:id?embed=editablescript
It is then possible to know the current edit id of a recipe by looking at the currentEditId
field of the recipe.
id required | integer |
{- "nextEditId": 1,
- "history": [
- {
- "owner": {
- "id": 1,
- "email": "joe@example.com",
- "name": "Joe Guy"
}, - "date": "2019-08-24T14:15:22Z",
- "editId": 1,
- "changes": [
- {
- "type": "inserted",
- "task": { },
- "portId": 1,
- "id": 1
}
], - "tableNameMap": { }
}
]
}
An internal object representing the AWS Elastic MapReduce (EMR) cluster configured to run trifacta jobs.
Create a new emr cluster
ℹ️ NOTE: Admin role is required to use this endpoint.
emrClusterId required | string The identifier for the EMR Cluster |
resourceBucket required | string S3 bucket to store Trifacta's libraries, external libraries, and any other resources for Spark execution |
resourcePath | string Path on S3 bucket to store resources for execution on EMR |
region | string The region where the EMR Cluster runs at |
{- "emrClusterId": "j-XXXXXXXXXXXXX",
- "resourceBucket": "bucketName",
- "resourcePath": "",
- "region": "us-west-2"
}
{- "emrClusterId": "j-XXXXXXXXXXXXX",
- "resourceBucket": "bucketName",
- "resourcePath": "",
- "region": "us-west-2",
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
List existing emr clusters
ℹ️ NOTE: Admin role is required to use this endpoint.
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "data": [
- {
- "emrClusterId": "j-XXXXXXXXXXXXX",
- "resourceBucket": "bucketName",
- "resourcePath": "",
- "region": "us-west-2",
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
]
}
Count existing emr clusters
ℹ️ NOTE: Admin role is required to use this endpoint.
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "count": 1
}
Get an existing emr cluster
ℹ️ NOTE: Admin role is required to use this endpoint.
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "emrClusterId": "j-XXXXXXXXXXXXX",
- "resourceBucket": "bucketName",
- "resourcePath": "",
- "region": "us-west-2",
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Update an existing emr cluster
ℹ️ NOTE: Admin role is required to use this endpoint.
id required | integer |
emrClusterId | string The identifier for the EMR Cluster |
resourceBucket | string S3 bucket to store Trifacta's libraries, external libraries, and any other resources for Spark execution |
resourcePath | string Path on S3 bucket to store resources for execution on EMR |
region | string The region where the EMR Cluster runs at |
{- "emrClusterId": "j-XXXXXXXXXXXXX",
- "resourceBucket": "bucketName",
- "resourcePath": "",
- "region": "us-west-2"
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
A container for wrangling logic. Contains Imported Datasets, Recipe, Output Objects, and References.
Create a new flow with specified name and optional description and target folder.
ℹ️ NOTE: You cannot add datasets to the flow through this endpoint. Moving pre-existing datasets into a flow is not supported in this release. Create the flow first and then when you create the datasets, associate them with the flow at the time of creation.
name | string Display name of the flow. |
description | string User-friendly description for the flow. |
folderId | integer Internal identifier for a Flow Folder. |
{- "name": "string",
- "description": "string",
- "folderId": 1
}
{- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
List existing flows
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
flowsFilter | string Which types of flows to list. One of ['all', 'shared', 'owned'] |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
]
}
Import all flows from the given package.
A ZIP
file as exported by the export Flow endpoint is accepted.
Before you import, you can perform a dry-run to check for errors. See Import Flow package - Dry run.
This endpoint accept a multipart/form
content type.
Here is how to send the ZIP
package using curl.
curl -X POST http://example.com:3005/v4/flows/package \
-H 'authorization: Bearer <api-token>' \
-H 'content-type: multipart/form-data' \
-F 'data=@path/to/flow-package.zip'
The response lists the objects that have been created.
folderId | integer |
{- "primaryFlowIds": [
- 1
], - "flows": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
], - "deletedObjects": { },
- "datasources": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
], - "flownodes": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}
], - "outputobjects": [
- {
- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "importRuleChanges": {
- "object": [
- { }
], - "value": [
- { }
]
}, - "release": { }
}
Test importing flow package and return information about what objects would be created.
The same payload as for Import Flow package is expected.
folderId | integer |
{- "primaryFlowIds": [
- 1
], - "flows": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
], - "deletedObjects": { },
- "datasources": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
], - "flownodes": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}
], - "outputobjects": [
- {
- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "importRuleChanges": {
- "object": [
- { }
], - "value": [
- { }
]
}, - "release": { }
}
Create a copy of this flow, as well as all contained recipes.
id required | integer |
name | string name of the new copied flow. |
description | string description of the new copied flow. |
copyDatasources | boolean Default: false If true, Data sources will be copied (i.e. new imported datasets will be created, no data is copied on the file system). Otherwise, the existing imported datasets are reused. |
{- "name": "string",
- "description": "string",
- "copyDatasources": false
}
{- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
Run all adhoc destinations in a flow. If a scheduleExecutionId
is provided, run all scheduled destinations in the flow.
id required | integer |
x-execution-id | string Example: f9cab740-50b7-11e9-ba15-93c82271a00b Optional header to safely retry the request without accidentally performing the same operation twice. If a FlowRun with the same |
object (runParameterOverrides) Allows to override parameters that are defined in the flow on datasets or outputs for e.g. | |
scheduleExecutionId | integer |
Array of objects (outputObjectSparkOptionUpdateRequest) [ items ] | |
overrides | object Overrides for each of the output object. Use the recipe name to specify the overrides. |
{- "runParameters": {
- "referenceDate": "2019-08-24T14:15:22Z",
- "overrides": {
- "data": [
- {
- "key": "string",
- "value": "string"
}
]
}
}, - "scheduleExecutionId": 1,
- "sparkOptions": [
- {
- "key": "string",
- "value": "string"
}
], - "overrides": {
- "property1": {
- "execution": "photon",
- "profiler": true,
- "writesettings": [
- {
- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "outputObjectId": 1,
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
]
}
], - "publications": [
- {
- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputObjectId": 1,
- "connectionId": "55",
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
], - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
], - "sparkOptions": [
- {
- "key": "string",
- "value": "string"
}
], - "webhooks": [
- {
- "name": "string",
- "url": "string",
- "method": "post",
- "triggerEvent": "onJobFailure",
- "body": "string",
- "headers": {
- "property1": "string",
- "property2": "string"
}, - "secretKey": "string",
- "sslVerification": true,
- "retryOnFailure": true
}
], - "datasources": {
- "property1": [
- "protocol://bucket/path-to-file.csv"
], - "property2": [
- "protocol://bucket/path-to-file.csv"
]
}
}, - "property2": {
- "execution": "photon",
- "profiler": true,
- "writesettings": [
- {
- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "outputObjectId": 1,
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
]
}
], - "publications": [
- {
- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputObjectId": 1,
- "connectionId": "55",
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
], - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
], - "sparkOptions": [
- {
- "key": "string",
- "value": "string"
}
], - "webhooks": [
- {
- "name": "string",
- "url": "string",
- "method": "post",
- "triggerEvent": "onJobFailure",
- "body": "string",
- "headers": {
- "property1": "string",
- "property2": "string"
}, - "secretKey": "string",
- "sslVerification": true,
- "retryOnFailure": true
}
], - "datasources": {
- "property1": [
- "protocol://bucket/path-to-file.csv"
], - "property2": [
- "protocol://bucket/path-to-file.csv"
]
}
}
}
}
{- "data": [
- {
- "id": 1,
- "flowRun": {
- "id": 1
}, - "jobs": {
- "data": [
- {
- "id": 1
}
]
}, - "jobGraph": {
- "edges": [
- {
- "source": 1,
- "target": 1
}
], - "vertices": [
- 1
]
}, - "reason": "Job started",
- "sessionId": "f9cab740-50b7-11e9-ba15-93c82271a00b"
}
]
}
Count existing flows
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
flowsFilter | string Which types of flows to count. One of ['all', 'shared', 'owned'] |
{- "count": 1
}
Get an existing flow
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
Update an existing flow based on the specified identifier.
ℹ️ NOTE: You cannot add datasets to the flow through this endpoint. Moving pre-existing datasets into a flow is not supported in this release. Create the flow first and then when you create the datasets, associate them with the flow at the time of creation.
id required | integer |
name | string Display name of the flow. |
description | string User-friendly description for the flow. |
folderId | integer Internal identifier for a Flow Folder. |
{- "name": "string",
- "description": "string",
- "folderId": 1
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Update an existing flow based on the specified identifier.
ℹ️ NOTE: You cannot add datasets to the flow through this endpoint. Moving pre-existing datasets into a flow is not supported in this release. Create the flow first and then when you create the datasets, associate them with the flow at the time of creation.
id required | integer |
name | string Display name of the flow. |
description | string User-friendly description for the flow. |
folderId | integer Internal identifier for a Flow Folder. |
{- "name": "string",
- "description": "string",
- "folderId": 1
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Retrieve a package containing the definition of the specified flow.
Response body is the contents of the package. Package contents are a ZIPped version of the flow definition.
The flow package can be used to import the flow in another environment. See the Import Flow Package for more information.
id required | integer |
comment | string comment to be displayed when flow is imported in a deployment package |
List flows, with special filtering behaviour
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
flowsFilter | string Which types of flows to list. One of ['all', 'shared', 'owned'] |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
]
}
Count flows, with special filtering behaviour
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
flowsFilter | string Which types of flows to count. One of ['all', 'shared', 'owned'] |
{- "count": {
- "flow": 1,
- "folder": 1,
- "all": 1
}
}
List all the inputs of a Flow. Also include data sources that are present in referenced flows.
id required | integer |
{- "data": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
]
}
List all the outputs of a Flow.
id required | integer |
{- "data": [
- {
- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}
Get all flows contained in this folder.
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
flowsFilter | string Which types of flows to list. One of ['all', 'shared', 'owned'] |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
]
}
Get the count of flows contained in this folder.
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
flowsFilter | string Which types of flows to count. One of ['all', 'shared', 'owned'] |
{- "count": 1
}
Update the list of flow watchers
id required | integer |
required | Array of personIdProperty (object) or emailProperty (object) (watcher) [ items ] |
{- "watchers": [
- {
- "personId": 1
}
]
}
{- "data": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "email": "user@example.com",
- "subscriptionId": "string"
}
]
}
Set the default output directory for the current flow
id required | integer |
defaultOutputDir required | string |
{- "defaultOutputDir": "string"
}
{- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
Notification settings for a Flow.
Create a new flow notification settings
onJobFailure required | string Enum: "all" "scheduled" "adhoc" "never" on job failure trigger condition |
onJobSuccess required | string Enum: "all" "scheduled" "adhoc" "never" on job success trigger condition |
flowId required | integer |
{- "onJobFailure": "all",
- "onJobSuccess": "all",
- "flowId": 1
}
{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "onJobFailure": "all",
- "onJobSuccess": "all",
- "flow": {
- "id": 1
}
}
Get an existing flow notification settings
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "onJobFailure": "all",
- "onJobSuccess": "all",
- "flow": {
- "id": 1
}
}
Patch an existing flow notification settings
id required | integer |
onJobFailure | string Enum: "all" "scheduled" "adhoc" "never" on job failure trigger condition |
onJobSuccess | string Enum: "all" "scheduled" "adhoc" "never" on job success trigger condition |
{- "onJobFailure": "all",
- "onJobSuccess": "all"
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Get a list of users and groups with which a Flow is shared. Collaborators can add and edit recipes and datasets in this Flow.
id required | integer |
{- "data": [
- {
- "id": 1,
- "email": "joe@example.com",
- "name": "Joe Guy",
- "flowPermission": {
- "role": "owner",
- "person": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
}
]
}
Get an existing flow permission
id required | integer |
aid required | integer |
{- "role": "owner",
- "person": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
An object representing a Flow run.
Get an existing flow run
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "scheduleExecutionId": 1,
- "requestId": "string",
- "flow": {
- "id": 1
}
}
Get the status of a Flow Run. It combines the status of the underlying Job Groups.
id required | integer |
"Complete"
Get the list of JobGroups.
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "ranfrom": "ui",
- "ranfor": "recipe",
- "status": "Complete",
- "profilingEnabled": true,
- "runParameterReferenceDate": "2019-08-24T14:15:22Z",
- "snapshot": {
- "id": 1
}, - "wrangledDataset": {
- "id": 1
}, - "flowrun": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}
Used to override the default value of RunParameter in a Flow
Create a new flow run parameter override
flowId required | number |
overrideKey required | string key/name used when overriding the value of the variable |
required | object |
{- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "myValue"
}
}, - "flowId": 0
}
{- "id": 1
}
Get an existing flow run parameter override
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "myValue"
}
}, - "flow": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Patch an existing flow run parameter override
id required | integer |
overrideKey | string key/name used when overriding the value of the variable |
object |
{- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "myValue"
}
}
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
List of users receiving notifications for a Flow.
Create a new flow watcher
flowId required | integer |
email required | string <email> <= 255 characters |
{- "flowId": 1,
- "email": "user@example.com"
}
{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "email": "user@example.com",
- "subscriptionId": "string"
}
Get an existing flow watcher
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "email": "user@example.com",
- "subscriptionId": "string"
}
A collection of Flows, useful for organization.
Create a new folder
name | string Display name of the folder. |
description | string User-friendly description for the folder. |
{- "name": "string",
- "description": "string"
}
{- "name": "string",
- "description": "string",
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
List existing folders
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
]
}
Count existing folders
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "count": 1
}
Get an existing folder
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "name": "string",
- "description": "string",
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
Update an existing folder
id required | integer |
name | string Display name of the folder. |
description | string User-friendly description for the folder. |
{- "name": "string",
- "description": "string"
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Patch an existing folder
id required | integer |
name | string Display name of the folder. |
description | string User-friendly description for the folder. |
{- "name": "string",
- "description": "string"
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Get all flows contained in this folder.
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
flowsFilter | string Which types of flows to list. One of ['all', 'shared', 'owned'] |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
]
}
Get the count of flows contained in this folder.
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
flowsFilter | string Which types of flows to count. One of ['all', 'shared', 'owned'] |
{- "count": 1
}
An object representing data loaded into Trifacta, as well as any structuring that has been applied to it. Imported Datasets are the starting point for wrangling, and can be used in multiple Flows.
Create an imported dataset from an available resource. Created dataset is owned by the authenticated user.
In general, importing a file is done using the following payload:
{
"uri": "protocol://path-to-file",
"name": "my dataset",
"detectStructure": true
}
See more examples in the Request Samples section.
✅ TIP: When an imported dataset is created via API, it is always imported as an unstructured dataset by default. To import a dataset with the inferred recipe, add
detectStructure: true
in the payload.
ℹ️ NOTE: Do not create an imported dataset from a file that is being used by another imported dataset. If you delete the newly created imported dataset, the file is removed, and the other dataset is corrupted. Use a new file or make a copy of the first file first.
ℹ️ NOTE: Importing a Microsoft Excel file or a file that need to be converted using the API is not supported yet.
name required | string Display name of the imported dataset. |
uri required | string Dataset URI |
description | string User-friendly description for the imported dataset. |
type | string Indicate the type of dataset. If not specified, the default storage protocol is used. |
isDynamic | boolean Default: false indicate if the datasource is parameterized. In that case, a |
detectStructure | boolean Default: false Indicate if a parsing script should be inferred when importing the dataset. By default, the dataset is imported |
dynamicPath | string Path used when resolving the parameters. It is used when running a job or collecting a sample. It is different from the one used as a storage location which corresponds to the first match. The latter is used when doing a fast preview in the UI. |
encoding | string Default: "UTF-8" Optional dataset encoding. |
sanitizeColumnNames | boolean Default: false Indicate whether the column names in the imported file should be sanitized |
Array of objects (runParameterFileBasedInfo) [ items ] Description of the dataset parameters if the dataset is parameterized. |
{- "uri": "protocol://path-to-file",
- "name": "my dataset",
- "detectStructure": true
}
{- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
Deprecated. Use listDatasetLibrary instead.
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "data": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
]
}
Add the specified imported dataset to a flow based on its internal identifier.
ℹ️ NOTE: Datasets can be added to flows based on the permissions of the access token used on this endpoint. Datasets can be added to flows that are shared by the user.
id required | integer |
required | object The flow to add this dataset to. |
{- "flow": {
- "id": 1
}
}
{- "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}
Create a copy of an imported dataset
id required | integer |
name | string name of the copied dataset |
{- "name": "string"
}
{- "dynamicPath": "string",
- "isSchematized": true,
- "isDynamic": true,
- "isConverted": true,
- "disableTypeInference": true,
- "hasStructuring": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
List all the inputs of a Flow. Also include data sources that are present in referenced flows.
id required | integer |
{- "data": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
]
}
Deprecated. Use countDatasetLibrary instead.
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "data": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
]
}
Get the specified imported dataset.
Use the following embedded reference to embed in the response data about the connection used to acquire the source dataset if it was created from a custom connection. See embedding resources for more information.
/v4/importedDatasets/{id}?embed=connection
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
Modify the specified imported dataset. Only the name and description properties should be modified.
id required | integer |
name | string Display name of the imported dataset. |
description | string User-friendly description for the imported dataset. |
{- "name": "My Dataset",
- "description": "string"
}
{- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
List all the inputs that are linked to this output object. Also include data sources that are present in referenced flows.
id required | integer |
{- "data": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
]
}
List Trifacta datasets.
This can be used to list both imported and reference datasets throughout the system,
as well as recipes in a given flow.
required | string or Array of strings Which types of datasets to list.
Valid choices are: [ |
ownershipFilter | string Which set of datasets to list.
One of [ |
schematized | boolean If included, filter to only show schematized imported datasets. |
currentFlowId | integer Required for including |
datasourceFlowId | integer When included, filter included datasets to only include those associated to the given flow. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
flowId | integer When provided, list datasets associated with this flow before other datasets. |
{- "data": [
- {
- "type": "datasource",
- "referenceCount": 1,
- "importedDataset": {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- null
], - "last": {
- "unit": null,
- "number": null,
- "dow": null
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
}
]
}
Count Trifacta datasets. Gives counts for various types of datasets matching the provided filters.
ownershipFilter | string Which set of datasets to count.
One of [ |
schematized | boolean If included, filter to only show schematized imported datasets. |
currentFlowId | integer Required for including |
datasourceFlowId | integer When included, filter included datasets to only include those associated to the given flow. |
flowId | integer When provided, count datasets associated with this flow before other datasets. |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "count": {
- "imported": 1,
- "reference": 1,
- "recipe": 1,
- "all": 1
}
}
An internal object encoding the information necessary to run a part of a Trifacta JobGroup.
This is called a "Stage" on the Job Results page in the UI.
Get information about the batch jobs within a Trifacta job.
id required | integer |
{- "data": [
- {
- "id": 1,
- "status": "Complete",
- "jobType": "wrangle",
- "sampleSize": 1,
- "percentComplete": 1,
- "jobGroup": {
- "id": 1
}, - "errorMessage": {
- "id": 1
}, - "lastHeartbeatAt": "2019-08-24T14:15:22Z",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "executionLanguage": "photon",
- "cpJobId": "string",
- "wranglescript": {
- "id": 1
}, - "emrcluster": {
- "id": 1
}
}
]
}
Create a jobGroup, which launches the specified job as the authenticated user. This performs the same action as clicking on the Run Job button in the application.
The request specification depends on one of the following conditions:
In the last case, you must specify some overrides when running the job. See the example with overrides
for more information.
ℹ️ NOTE: Override values applied to a job are not validated. Invalid overrides may cause your job to fail.
To run a job, you just specify the recipe identifier (wrangledDataset.id). If the job is successful, all defined outputs are generated, as defined in the outputobject, publications, and writeSettings objects associated with the recipe.
✅ TIP: To identify the wrangledDataset Id, select the recipe icon in the flow view and take the id shown in the URL. e.g. if the URL is
/flows/10?recipe=7
, the wrangledDataset Id is7
.
{"wrangledDataset": {"id": 7}}
If you must change some outputs or other settings for the specific job, you can insert these changes in the overrides section of the request. In the example below, the running environment, profiling option, and writeSettings for the job are modified for this execution.
{
"wrangledDataset": {"id": 1},
"overrides": {
"execution": "spark",
"profiler": false,
"writesettings": [
{
"path": "<path_to_output_file>",
"action": "create",
"format": "csv",
"compression": "none",
"header": false,
"asSingleFile": false
}
]
}
}
You can also override the spark options that will be used for the job run
{
"wrangledDataset": {"id": 1},
"overrides": {
"execution": "spark",
"profiler": true,
"sparkOptions": [
{"key": "spark.executor.cores", "value": "2"},
{"key": "spark.executor.memory", "value": "4GB"}
]
}
}
If you have created a dataset with parameters, you can specify overrides for parameter values during execution through the APIs. Through this method, you can iterate job executions across all matching sources of a parameterized dataset.
In the example below, the runParameters override has been specified for the country
. In this case, the value "Germany" is inserted for the specified variable as part of the job execution.
{
"wrangledDataset": {"id": 33},
"runParameters": {
"overrides": {
"data": [{"key": "country", "value": "Germany"}]
}
}
}
The response contains a list of jobs which can be used to get a granular status of the JobGroup completion.
The jobGraph
indicates the dependency between each of the jobs.
{
"sessionId": "79276c31-c58c-4e79-ae5e-fed1a25ebca1",
"reason": "JobStarted",
"jobGraph": {
"vertices": [21, 22],
"edges": [{"source": 21, "target": 22}]
},
"id": 9,
"jobs": {"data": [{"id": 21}, {"id": 22}]}
}
x-execution-id | string Example: f9cab740-50b7-11e9-ba15-93c82271a00b Optional header to safely retry the request without accidentally performing the same operation twice. If a JobGroup with the same |
required | object The identifier for the recipe you would like to run. |
forceCacheUpdate | boolean Setting this flag to true will invalidate any cached datasources. This only applies to SQL datasets. |
testMode | boolean Setting this flag to true will not run the job but just perform some validations. |
object (runParameterOverrides) Allows to override parameters that are defined in the flow on datasets or outputs for e.g. | |
workspaceId | integer Internal. Does not need to be specified |
object Allows to override execution settings that are set on the output object. | |
ranfrom | string Enum: "ui" "schedule" "api" Where the job was executed from. Does not need to be specified when using the API.
|
{- "wrangledDataset": {
- "id": 7
}
}
{- "sessionId": "79276c31-c58c-4e79-ae5e-fed1a25ebca1",
- "reason": "JobStarted",
- "jobGraph": {
- "vertices": [
- 21,
- 22
], - "edges": [
- {
- "source": 21,
- "target": 22
}
]
}, - "id": 9,
- "jobs": {
- "data": [
- {
- "id": 21
}, - {
- "id": 22
}
]
}
}
Deprecated. Use listJobLibrary instead.
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
flowNodeId | integer |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "ranfrom": "ui",
- "ranfor": "recipe",
- "status": "Complete",
- "profilingEnabled": true,
- "runParameterReferenceDate": "2019-08-24T14:15:22Z",
- "snapshot": {
- "id": 1
}, - "wrangledDataset": {
- "id": 1
}, - "flowrun": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}
Cancel the execution of a running Trifacta JobGroup.
ℹ️ NOTE: If the job has completed, this endpoint does nothing.
id required | integer |
{ }
{- "jobIds": [
- 1
], - "jobgroupId": 1
}
Deprecated. Use countJobLibrary instead.
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
flowNodeId | integer |
{- "count": 1
}
Get the specified JobGroup.
A job group is a job that is executed from a specific node in a flow. The job group may contain:
It is possible to only get the current status for a JobGroup:
/v4/jobGroups/{id}/status
In that case, the response status would simply be a string:
"Complete"
If you wish to also get the related Jobs and WrangledDataset, you can use embed
. See embedding resources for more information.
/v4/jobGroups/{id}?embed=jobs,wrangledDataset
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "name": "string",
- "description": "string",
- "ranfrom": "ui",
- "ranfor": "recipe",
- "status": "Complete",
- "profilingEnabled": true,
- "runParameterReferenceDate": "2019-08-24T14:15:22Z",
- "snapshot": {
- "id": 1
}, - "wrangledDataset": {
- "id": 1
}, - "flowrun": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
Delete a JobGroup. This can help to free resources, such as cleaning the Profile Results.
ℹ️ NOTE: Deleting a JobGroup is only possible if the Job is not in progress. Pass
?forceDeletion=true
if you absolutely want to remove the JobGroup. This will not stop any background processing that might have been started with this JobGroup.
id required | integer |
forceDeletion | boolean Force the deletion of the JobGroup. Note that this might not prevent any of the background processing that could have been started with the Job. |
{- "profilerTypeCheckHistograms": {
- "property1": [
- {
- "key": "VALID",
- "count": 1
}
], - "property2": [
- {
- "key": "VALID",
- "count": 1
}
]
}, - "profilerValidValueHistograms": {
- "property1": [
- {
- "min": 0,
- "max": 0,
- "roundMin": 0,
- "roundMax": 0,
- "buckets": [
- {
- "pos": 1,
- "b": 1
}
], - "quartiles": {
- "q1": 0,
- "q2": 0,
- "q3": 0
}
}
], - "property2": [
- {
- "min": 0,
- "max": 0,
- "roundMin": 0,
- "roundMax": 0,
- "buckets": [
- {
- "pos": 1,
- "b": 1
}
], - "quartiles": {
- "q1": 0,
- "q2": 0,
- "q3": 0
}
}
]
}, - "columnTypes": {
- "property1": [
- "string"
], - "property2": [
- "string"
]
}
}
Get the job group inputs. Return the list of datasets used when running this JobGroup.
id required | integer |
{- "data": [
- {
- "name": "string",
- "inputs": [
- {
- "vendor": "string",
- "databaseConnectString": "string",
- "relationalPath": [
- "string"
], - "table": "string",
- "action": "string",
- "query": [
- "string"
]
}
]
}
]
}
Get the job group outputs. Return the list of tables and file paths used as output.
id required | integer |
{- "files": [
- {
- "uri": "string",
- "fileType": "FILE",
- "isPrimaryOutput": true
}
], - "tables": [
- {
- "vendor": "string",
- "databaseConnectString": "string",
- "relationalPath": [
- "string"
], - "table": "string",
- "action": "string",
- "query": [
- "string"
]
}
]
}
Get list of all JobGroup accessible to the authenticated user.
Note that it is possible to embed other resources while fetching the JobGroup list. e.g.:
/v4/jobLibrary/?embed=jobs,wrangledDataset
See embedding resources for more information.
It is possible to filter JobGroups based on their status.
Here is how to get all JobGroups with a Failed
status:
/v4/jobLibrary?status=Failed
It is possible to filter only scheduled JobGroups using the following request:
/v4/jobLibrary?ranfrom=schedule
It is also possible to filter the JobGroups based on the Date. Here is an example:
/v4/jobLibrary?dateFilter[createdAt][gte]=1572994800000&dateFilter[updatedAt][lt]=1581375600000
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
dateFilter | object for filtering jobgroups by start and end date |
ranfrom | string filter jobs based on how they were run |
status | string filter jobs based on their status |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "ranfrom": "ui",
- "ranfor": "recipe",
- "status": "Complete",
- "profilingEnabled": true,
- "runParameterReferenceDate": "2019-08-24T14:15:22Z",
- "snapshot": {
- "id": 1
}, - "wrangledDataset": {
- "id": 1
}, - "flowrun": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}
Count Trifacta jobs with special filter capabilities. See listJobLibrary for some examples.
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
dateFilter | object for filtering jobgroups by start and end date |
ranfrom | string filter jobs based on how they were run |
status | string filter jobs based on their status |
{- "count": 1
}
Get information about the batch jobs within a Trifacta job.
id required | integer |
{- "data": [
- {
- "id": 1,
- "status": "Complete",
- "jobType": "wrangle",
- "sampleSize": 1,
- "percentComplete": 1,
- "jobGroup": {
- "id": 1
}, - "errorMessage": {
- "id": 1
}, - "lastHeartbeatAt": "2019-08-24T14:15:22Z",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "executionLanguage": "photon",
- "cpJobId": "string",
- "wranglescript": {
- "id": 1
}, - "emrcluster": {
- "id": 1
}
}
]
}
Get list of publications for the specified jobGroup.
A Publication is an export of job results from the platform after they have been initially generated.
id required | integer |
{- "data": [
- {
- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputobject": {
- "id": 1
}, - "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
]
}
For a specified JobGroup, this endpoint performs an ad-hoc publish of the results to the designated target. Target information is based on the specified connection.
Job results to published are based on the specified jobGroup. You can specify:
Supported targets:
id required | integer |
required | object Internal identifier of the connection to use to write the results. |
path required | Array of strings path to the location of the table/datasource. |
table required | string Name of table in the database to which to write the results. |
action required | string Enum: "create" "load" "createAndLoad" "truncateAndLoad" "dropAndLoad" Type of writing action to perform with the results
|
inputFormat required | string Source format of the results. Supported values:
|
{- "connection": {
- "id": 1
}, - "path": [
- "default"
], - "table": "test_table",
- "action": "create",
- "inputFormat": "pqt"
}
{- "jobgroupId": 1,
- "reason": "Job started",
- "sessionId": "f9cab740-50b7-11e9-ba15-93c82271a00b"
}
An object containing a list of ScriptLines that can be reused across Recipes.
Performs an import of a macro package.
ℹ️ NOTE: You cannot import a macro that was exported from a later version of the product.
✅ TIP: You can paste the response of the exported macro page as the request.
ℹ️ NOTE: Modification of the macro definition is not supported outside of the Trifacta.
type required | string Type of artifact. This value is always |
kind required | string This value is |
hash required | string Hash value used to verify the internal integrity of the macro definition. |
required | object |
required | object |
{- "type": "string",
- "kind": "string",
- "hash": "string",
- "data": {
- "name": "string",
- "description": "string",
- "signature": [
- {
- "name": "Store_Nbr",
- "type": "column"
}
], - "scriptlines": [
- {
- "hash": "string",
- "task": { }
}
]
}, - "metadata": {
- "lastMigration": "20191024143300",
- "trifactaVersion": "6.8.0+4.20191104073802.8b6217a",
- "exportedAt": "2019-08-24T14:15:22Z",
- "exportedBy": 1,
- "uuid": "6b27eee0-0034-11ea-a378-9dc0586de9fb",
- "edition": "Enterprise"
}
}
{- "id": 1,
- "name": "string",
- "description": "string",
- "signature": [
- {
- "name": "Store_Nbr",
- "type": "column"
}
], - "createdBy": 1,
- "updatedBy": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "workspaceId": 1
}
Retrieve a package containing the definition of the specified macro. Response body is the contents of the package, which is an importable version of the macro definition.
✅ TIP: The response body can be pasted as the request when you import the macro into a different environment. For more information, see Import Macro Package.
ℹ️ NOTE: Modification of the macro definition is not supported outside of the Trifacta.
id required | integer |
{- "type": "string",
- "kind": "string",
- "hash": "string",
- "data": {
- "name": "string",
- "description": "string",
- "signature": [
- {
- "name": "Store_Nbr",
- "type": "column"
}
], - "scriptlines": [
- {
- "hash": "string",
- "task": { }
}
]
}, - "metadata": {
- "lastMigration": "20191024143300",
- "trifactaVersion": "6.8.0+4.20191104073802.8b6217a",
- "exportedAt": "2019-08-24T14:15:22Z",
- "exportedBy": 1,
- "uuid": "6b27eee0-0034-11ea-a378-9dc0586de9fb",
- "edition": "Enterprise"
}
}
An OutputObject is a definition of one or more types of outputs and how they are generated.
If an OutputObject already exists for the Recipe (flowNodeId
) to which you are posting, you must either modify the object instead or delete it before posting your new object.
execution required | string Enum: "photon" "spark" Execution language. Indicate on which engine the job was executed. . Can be null/missing for scheduled jobs that fail during the validation phase.
|
profiler required | |
isAdhoc | |
flowNodeId | integer FlowNode the OutputObject should be attached to. (This is also the id of the WrangledDataset). |
Array of objects (writeSettingCreateRequest) [ items ] Optionally you can include WriteSettings while creating the OutputObject | |
Array of objects (publicationCreateRequest) [ items ] Optionally you can include Publications while creating the OutputObject | |
Array of objects (outputObjectSparkOptionUpdateRequest) [ items ] |
{- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flowNodeId": 1,
- "writeSettings": [
- {
- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "outputObjectId": 1,
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
]
}
], - "publications": [
- {
- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputObjectId": 1,
- "connectionId": "55",
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
], - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
], - "outputObjectSparkOptions": [
- {
- "key": "string",
- "value": "string"
}
]
}
{- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
List existing output objects
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "data": [
- {
- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}
List all the outputs of a Flow.
id required | integer |
{- "data": [
- {
- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}
Count existing output objects
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "count": 1
}
Get the specified OutputObject.
Note that it is possible to include WriteSettings and Publications that are linked to this OutputObject. See embedding resources for more information.
/v4/outputObjects/{id}?embed=writeSettings,publications
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
Update an existing output object
id required | integer |
execution | string Enum: "photon" "spark" Execution language. Indicate on which engine the job was executed. . Can be null/missing for scheduled jobs that fail during the validation phase.
|
profiler | |
Array of objects (writeSettingCreateRequest) [ items ] | |
Array of objects (publicationCreateRequest) [ items ] | |
Array of objects (outputObjectSparkOptionUpdateRequest) [ items ] |
{- "execution": "photon",
- "profiler": true,
- "writeSettings": [
- {
- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "outputObjectId": 1,
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
]
}
], - "publications": [
- {
- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputObjectId": 1,
- "connectionId": "55",
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
], - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
], - "outputObjectSparkOptions": [
- {
- "key": "string",
- "value": "string"
}
]
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Patch an existing output object
id required | integer |
execution | string Enum: "photon" "spark" Execution language. Indicate on which engine the job was executed. . Can be null/missing for scheduled jobs that fail during the validation phase.
|
profiler | |
Array of objects (writeSettingCreateRequest) [ items ] | |
Array of objects (publicationCreateRequest) [ items ] | |
Array of objects (outputObjectSparkOptionUpdateRequest) [ items ] |
{- "execution": "photon",
- "profiler": true,
- "writeSettings": [
- {
- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "outputObjectId": 1,
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
]
}
], - "publications": [
- {
- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputObjectId": 1,
- "connectionId": "55",
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
], - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
], - "outputObjectSparkOptions": [
- {
- "key": "string",
- "value": "string"
}
]
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
List all the inputs that are linked to this output object. Also include data sources that are present in referenced flows.
id required | integer |
{- "data": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
]
}
Set the dataflow option in the person's user preferences. These options are used by default for job executions, unless overridden by some output Dataflow options.
Each project can have multiple labels, up to a maximum of 64. For more details read Requirements for labels.
ℹ️ NOTE: Issues that are shown as warnings in the UI will not be present when using the API.
ℹ️ NOTE: If VPC network mode is set to
AUTO
, do not include entries in the request fornetwork
,subnetwork
or forusePublicIps
.
ℹ️ NOTE: Subnetwork region must be the same as the regional endpoint. In case of a difference between the regions, the regional endpoint will be used.
ℹ️ NOTE: If autoscaling algorithm is set to
NONE
, do not include entry in the request formaxNumWorkers
.
type required | string
|
required | string or Array of objects |
{- "value": "us-central-1"
}
{- "region": {
- "key": "region",
- "value": "us-central-1",
- "person": {
- "id": 1
}, - "id": 1,
- "createdAt": "2020-04-20T12:49:41Z",
- "updatedAt": "2020-04-20T12:49:41Z"
}
}
Get an existing person
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
uuid | string |
workspaceId | string |
{- "email": "joe@example.com",
- "isAdmin": true,
- "isDisabled": false,
- "state": "active",
- "id": 1,
- "outputHomeDir": "/home-dir/queryResults/joe@example.com",
- "fileUploadPath": "/uploads",
- "lastLoginTime": "2019-08-24T14:15:22Z",
- "lastStateChange": "2019-08-24T14:15:22Z"
}
Update an existing person
id required | integer |
string <email> | |
isAdmin | boolean If true, the user account is an administrator account. This property can only be changed by an admin account. |
isDisabled | boolean If true, the account is disabled. This property can only be changed by an admin account. |
state | string Enum: "active" "hidden" Current state of the user account. This property can only be changed by an admin account.
|
name | string name of the user |
password | string User password |
password2 | string This value confirms the value for password. These two property values must be identical. |
outputHomeDir | string Home directory where the user's generated results are written |
{- "email": "joe@example.com",
- "isAdmin": true,
- "isDisabled": false,
- "state": "active",
- "name": "Joe Guy",
- "password": "string",
- "password2": "string",
- "outputHomeDir": "/home-dir/queryResults/joe@example.com"
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Patch an existing person
id required | integer |
string <email> | |
isAdmin | boolean If true, the user account is an administrator account. This property can only be changed by an admin account. |
isDisabled | boolean If true, the account is disabled. This property can only be changed by an admin account. |
state | string Enum: "active" "hidden" Current state of the user account. This property can only be changed by an admin account.
|
name | string name of the user |
password | string User password |
password2 | string This value confirms the value for password. These two property values must be identical. |
outputHomeDir | string Home directory where the user's generated results are written |
{- "email": "joe@example.com",
- "isAdmin": true,
- "isDisabled": false,
- "state": "active",
- "name": "Joe Guy",
- "password": "string",
- "password2": "string",
- "outputHomeDir": "/home-dir/queryResults/joe@example.com"
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
List existing people
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
string | |
workspaceId | string Filter the users using in a specific workspace. If not set, list users in the current workspace. It is ignored if the user is not an admin user. |
isDisabled | string |
{- "data": [
- {
- "email": "joe@example.com",
- "isAdmin": true,
- "isDisabled": false,
- "state": "active",
- "id": 1,
- "outputHomeDir": "/home-dir/queryResults/joe@example.com",
- "fileUploadPath": "/uploads",
- "lastLoginTime": "2019-08-24T14:15:22Z",
- "lastStateChange": "2019-08-24T14:15:22Z"
}
]
}
Create a new person
email required | string <email> |
accept required | string This property must be set to "accept" to create the user. |
isAdmin | boolean If true, the user account is an administrator account. This property can only be changed by an admin account. |
isDisabled | boolean If true, the account is disabled. This property can only be changed by an admin account. |
name | string name of the user |
password | string User password |
password2 | string This value confirms the value for password. These two property values must be identical. |
email2 | string <email> |
outputHomeDir | string Home directory where the user's generated results are written |
{- "email": "joe@example.com",
- "isAdmin": true,
- "isDisabled": false,
- "name": "Joe Guy",
- "accept": "accept",
- "password": "string",
- "password2": "string",
- "email2": "user@example.com",
- "outputHomeDir": "/home-dir/queryResults/joe@example.com"
}
{- "email": "joe@example.com",
- "isAdmin": true,
- "isDisabled": false,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Count existing people
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
isDisabled | string |
{- "count": 1
}
Transfer Trifacta assets to another user. Assigns ownership of all the contents of one user's workspace to another user. This includes flows, datasets, recipes, and connections–basically any object that can be created and managed through the Trifacta UI.
Note: This API endpoint does not delete the original user account. To delete the user account, another API call is needed.
ℹ️ NOTE: Admin role is required to use this endpoint.
fid required | integer the id of the person to transfer assets from |
tid required | integer the id of the person to transfer assets to |
{ }
Request to reset a user's password.
ℹ️ NOTE: Admin role is required to use this endpoint
ℹ️ NOTE: This endpoint does not generate an email or perform the reset. You must use the reset code to build a reset URL to send separately to the specific user. The above must be built into a URL in the following format:
http://example.com:3005/password-reset?email=<email>&code=<AccountResetCode>
URL element | Example value | Description |
---|---|---|
joe@example.com | User ID (email address) of the user whose password is to be reset | |
AccountResetCode | CD44232791 | Password reset code |
accountId | integer Internal identifier of the user whose password should be reset |
originUrl | string The URL of the login page for the Trifacta application |
{- "accountId": 1,
- "originUrl": "string"
}
{- "code": "string",
- "email": "user@example.com"
}
Run the latest activated snapshot of a plan. At least one snapshot of the plan must exist to be able to use this endpoint.
A snapshot represents the state of the plan at the time it was taken.
Note that the underlying objects (flows, datasets, ...) might have changed after the snapshot was taken.
If some flows or outputs referenced by the plan tasks have been deleted, it will return a
MissingFlowReferences
validation status.
If the plan snapshot is valid, it will be queued for execution.
This endpoint returns a planSnapshotRunId
that can be used to track the plan execution status
using getPlanSnapshotRun.
id required | integer |
x-execution-id | string Example: f9cab740-50b7-11e9-ba15-93c82271a00b Optional header to safely retry the request without accidentally performing the same operation twice.
If a |
{ }
{- "validationStatus": "Valid",
- "planSnapshotRunId": 1
}
An execution of a Plan's snapshot state
Return a plan snapshot run that contains the current status of a plan execution
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "id": 1,
- "status": "Complete",
- "createdAt": "2019-08-24T14:15:22Z",
- "finishedAt": "2019-08-24T14:15:22Z",
- "startedAt": "2019-08-24T14:15:22Z",
- "submittedAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "plan": {
- "id": 1
}, - "nextRun": {
- "id": "string"
}, - "previousRun": {
- "id": "string"
}
}
A Publication object is used to specify a table-based output and is associated with an OutputObject. Settings include the connection to use, path, table type, and write action to apply.
Create a new publication
path required | Array of strings path to the location of the table/datasource. |
tableName required | string name of the table (or of the datasource in case of Tableau) |
targetType required | string e.g. |
action required | string Enum: "create" "load" "createAndLoad" "truncateAndLoad" "dropAndLoad" Type of writing action to perform with the results
|
outputObjectId | integer OutputObject to attach this Publication to. |
connectionIdString (string) or connectionIdBigQuery (string) | |
Array of objects (runParameterDestinationInfo) [ items ] Optional Parameters that can be used to parameterized the | |
parameters | object Additional publication parameters specific to each JDBC data source. Example: isDeltaTable=true for Databricks connections to produce Delta Lake Tables |
{- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputObjectId": 1,
- "connectionId": "55",
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
], - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
{- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputobject": {
- "id": 1
}, - "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
List existing publications
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "data": [
- {
- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputobject": {
- "id": 1
}, - "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
]
}
Count existing publications
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "count": 1
}
Get an existing publication
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputobject": {
- "id": 1
}, - "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
Update an existing publication
id required | integer |
path | Array of strings path to the location of the table/datasource. |
tableName | string name of the table (or of the datasource in case of Tableau) |
targetType | string e.g. |
action | string Enum: "create" "load" "createAndLoad" "truncateAndLoad" "dropAndLoad" Type of writing action to perform with the results
|
parameters | object Additional publication parameters specific to each JDBC data source. Example: isDeltaTable=true for Databricks connections to produce Delta Lake Tables |
{- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
{- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputobject": {
- "id": 1
}, - "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
Patch an existing publication
id required | integer |
path | Array of strings path to the location of the table/datasource. |
tableName | string name of the table (or of the datasource in case of Tableau) |
targetType | string e.g. |
action | string Enum: "create" "load" "createAndLoad" "truncateAndLoad" "dropAndLoad" Type of writing action to perform with the results
|
parameters | object Additional publication parameters specific to each JDBC data source. Example: isDeltaTable=true for Databricks connections to produce Delta Lake Tables |
{- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
A specific instance of a flow package that has been imported.
A Deployment contains multiple Releases among which only one is active.
Create a release for the specified deployment.
Release is created from a local ZIP
containing the package of the flow exported from the source system.
When importing a release, import-mapping rules are executed. These import rules allow you to replace the file location or the table names of different objects during the import for a deployment. See updateObjectImportRules and updateValueImportRules if you need to update the import rules.
This endpoint accept a multipart/form
content type.
Here is how to send the ZIP
package using curl.
curl -X POST http://example.com:3005/v4/deployments/:id/releases \
-H 'authorization: Bearer <api-token>' \
-H 'content-type: multipart/form-data' \
-F 'data=@path/to/flow-package.zip'
The response lists the objects that have been created.
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
folderId | integer |
{- "primaryFlowIds": [
- 1
], - "flows": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
], - "deletedObjects": { },
- "datasources": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
], - "flownodes": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}
], - "outputobjects": [
- {
- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "importRuleChanges": {
- "object": [
- { }
], - "value": [
- { }
]
}, - "release": { }
}
Get the list of releases for the specified deployment
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "data": [
- {
- "notes": "string",
- "packageUuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "active": true,
- "deployment": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}
Test importing flow package, applying all import rules that apply to this deployment, and return information about what objects would be created.
The same payload as for Import Deployment package is expected.
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
folderId | integer |
{- "primaryFlowIds": [
- 1
], - "flows": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
], - "deletedObjects": { },
- "datasources": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
], - "flownodes": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}
], - "outputobjects": [
- {
- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "importRuleChanges": {
- "object": [
- { }
], - "value": [
- { }
]
}, - "release": { }
}
List existing releases
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
deploymentId | integer Apply this filter to show only releases matching the given deployment. |
{- "data": [
- {
- "notes": "string",
- "packageUuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "active": true,
- "deployment": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}
Count existing releases
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "count": 1
}
Get an existing release
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "notes": "string",
- "packageUuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "active": true,
- "deployment": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
Update an existing release
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
notes | string Display value for notes that you can add to describe the release. |
packageUuid | string <uuid> Unique identifier for the package |
active | boolean If |
{- "notes": "string",
- "packageUuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "active": true
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}
Update the specified release.
You can use the following example to make the current release the active one for the deployment.
{"active": true}
ℹ️ NOTE: You can have only one active release per deployment. If this release is made active as part of this execution, the currently active release is made inactive.
✅ TIP: You can use this endpoint to deactivate a release, which prevents its jobs from being run. If there is no active release for the deployment, no jobs are run via the deployment job run endpoint. See runDeployment.
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
notes | string Display value for notes that you can add to describe the release. |
packageUuid | string <uuid> Unique identifier for the package |
active | boolean If |
{- "active": true
}
{- "notes": "string",
- "packageUuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "active": true,
- "deployment": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
Retrieve a package containing the definition of the flow for the specified release.
ℹ️ NOTE: Releases pertain to Production instances of the Trifacta Platform. For more information, see Overview of Deployment Manager.
ℹ️ NOTE: This method exports flows from a Product instance, which is different from exporting using the getFlowPackage, which exports from the Dev instance. Connection identifiers and paths may differ between the two instances. This method is typically used for archiving releases from the Deployment Manager.
Response body is the contents of the package. Package contents are a ZIPped version of the flow definition.
ℹ️ NOTE: A deployment role or a deployment instance is required to use this endpoint.
id required | integer |
get support bundle
ℹ️ NOTE: Admin role is required to use this endpoint.
startTime | string ISO timestamp; only include log events which happened after this time |
endTime | string ISO timestamp; only include log events which happened before this time |
sessionId | string only include log events which belong to this session id |
maxFileSizeInBytes | integer max file size of filtered log files in the support bundle |
Represents the data produced by running a Recipe on some input.
ℹ️ NOTE: In the Trifacta application UI, the WrangledDataset object is called a recipe.
Create a new wrangled dataset
required | object |
required | object |
name required | string |
inferredScript | object |
{- "importedDataset": {
- "id": 1
}, - "inferredScript": { },
- "flow": {
- "id": 1
}, - "name": "string"
}
{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "referenceinfo": {
- "id": 1,
- "name": "string",
- "description": "string"
}, - "wrangled": true
}
List existing wrangled datasets
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "data": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "referenceinfo": {
- "id": 1,
- "name": "string",
- "description": "string"
}, - "wrangled": true
}
]
}
Add this wrangled dataset to a flow as a reference.
id required | integer |
required | object The flow to add this dataset to. |
{- "flow": {
- "id": 1
}
}
{- "flow": {
- "id": 1
}, - "referencedFlowNode": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}
Count existing wrangled datasets
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "count": 1
}
Get an existing wrangled dataset
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "referenceinfo": {
- "id": 1,
- "name": "string",
- "description": "string"
}, - "wrangled": true
}
Update a wrangled dataset. This can mean one of two things.Either this will update the flownode object in our database or the editable script object.
id required | integer |
activesampleId | integer Internal identifier of the currently active |
referenceId | integer Internal identifier for referenceInfo, which contains the name and description of the reference object associated with this flow node. This is how the reference dataset will appear when used in other flows. |
{- "activesampleId": 1,
- "referenceId": 1
}
{- "id": 1,
- "updater": {
- "id": 1
}, - "updatedAt": "2019-08-24T14:15:22Z"
}
Get the dataset that is the primary input for this wrangled dataset. This can be either an imported dataset or a wrangled dataset.
id required | integer |
{- "wrangledDataset": {
- "id": 1
}
}
This action performs a dataset swap for the source of a wrangled dataset, which can be done through the UI.
Update the primary input dataset for the specified wrangled dataset. Each wrangled dataset must have one and only one primary input dataset, which can be an imported or wrangled dataset. If a wrangled dataset from another flow is selected, a reference will be used.
✅ TIP: After you have created a job via API, you can use this API to swap out the source data for the job's dataset. In this manner, you can rapidly re-execute a pre-existing job using fresh data.
id required | integer |
required | object |
{- "wrangledDataset": {
- "id": 1
}
}
{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "referenceinfo": {
- "id": 1,
- "name": "string",
- "description": "string"
}, - "wrangled": true
}
A WriteSetting object defines file-based outputs within an OutputObject. Settings include path, format, compression, and delimiters.
To specify multiple outputs, you can include additional WriteSetting objects in the request.
For example, if you want to generate output to csv
and json
, you can duplicate the writeSettings object for csv and change the format value in the second one to json.
Create a new write setting
path required | string The fully qualified path to the output location where to write the results. |
action required | string Enum: "create" "append" "overwrite" If the output file or directory exists, you can specify one of the following actions
|
format required | string Enum: "csv" "json" "tde" "avro" "pqt" "hyper" Output format for the results. Specify one of the supported values.
|
compression | string Enum: "none" "gzip" "bzip2" "snappy" For csv and json results,
you can optionally compress them using
|
header | boolean For csv results with action set to |
asSingleFile | boolean For |
delim | string The delimiter between field values in an output row. Only relevant if the chosen |
hasQuotes | boolean If true, each field in the output is wrapped in double-quotes. |
outputObjectId | integer OutputObject to attach this WriteSetting to. |
Array of objects (runParameterDestinationInfo) [ items ] Optional Parameters that can be used to parameterized the path |
{- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "outputObjectId": 1,
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
]
}
{- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
List existing write settings
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "data": [
- {
- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}
Count existing write settings
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
limit | integer Default: 25 Maximum number of objects to fetch. |
noLimit | string If set to |
offset | integer Offset after which to start returning objects. For use with |
sort | string Example: sort=-createdAt Defines sort order for returned objects |
filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "count": 1
}
Get an existing write setting
id required | integer |
embed | string Example: embed=flownodes.referenceInfo,flowedges Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
includeDeleted | string If set to "true", will include deleted objects. |
{- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
Update an existing write setting
id required | integer |
path | string The fully qualified path to the output location where to write the results. |
action | string Enum: "create" "append" "overwrite" If the output file or directory exists, you can specify one of the following actions
|
format | string Enum: "csv" "json" "tde" "avro" "pqt" "hyper" Output format for the results. Specify one of the supported values.
|
compression | string Enum: "none" "gzip" "bzip2" "snappy" For csv and json results,
you can optionally compress them using
|
header | boolean For csv results with action set to |
asSingleFile | boolean For |
delim | string The delimiter between field values in an output row. Only relevant if the chosen |
hasQuotes | boolean If true, each field in the output is wrapped in double-quotes. |
{- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true
}
{- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
Patch an existing write setting
id required | integer |
path | string The fully qualified path to the output location where to write the results. |
action | string Enum: "create" "append" "overwrite" If the output file or directory exists, you can specify one of the following actions
|
format | string Enum: "csv" "json" "tde" "avro" "pqt" "hyper" Output format for the results. Specify one of the supported values.
|
compression | string Enum: "none" "gzip" "bzip2" "snappy" For csv and json results,
you can optionally compress them using
|
header | boolean For csv results with action set to |
asSingleFile | boolean For |
delim | string The delimiter between field values in an output row. Only relevant if the chosen |
hasQuotes | boolean If true, each field in the output is wrapped in double-quotes. |
{- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true
}
{- "updated": true,
- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}