Download OpenAPI specification:Download
To enable programmatic control over its objects, the Trifacta Wrangler Platform supports a range of REST API endpoints across its objects. This section provides an overview of the API design, methods, and supported use cases.
Most of the endpoints accept JSON as input and return JSON responses.
This means that you must usually add the following headers to your request:
Content-type: application/json
Accept: application/jsonVersion: 8.6.0+47.20210806164855.b656442
The term resource refers to a single type of object in the Trifacta Wrangler Platform metadata. An API is broken up by its endpoint's corresponding resource.
The name of a resource is typically plural, and expressed in camelCase. Example: jobGroups.
Resource names are used as part of endpoint URLs, as well as in API parameters and responses.
The platform supports Create, Read, Update, and Delete operations on most resources. You can review the standards for these operations and their standard parameters below.
Some endpoints have special behavior as exceptions.
To create a resource, you typically submit an HTTP POST request with the resource's required metadata in the request body.
The response returns a 201 Created response code upon success with the resource's metadata, including its internal id, in the response body.
An HTTP GET request can be used to read a resource or to list a number of resources.
A resource's id can be submitted in the request parameters to read a specific resource.
The response usually returns a 200 OK response code upon success, with the resource's metadata in the response body.
If a GET request does not include a specific resource id, it is treated as a list request.
The response usually returns a 200 OK response code upon success, with an object containing a list of resources' metadata in the response body.
When reading resources, some common query parameters are usually available. e.g.:
/v4/jobGroups?limit=100&includeDeleted=true&embed=jobs| Query Parameter | Type | Description |
|---|---|---|
| embed | string | Comma-separated list of objects to include part of the response. See Embedding resources. |
| includeDeleted | string | If set to true, response includes deleted objects. |
| limit | integer | Maximum number of objects to fetch. Usually 25 by default |
| offset | integer | Offset after which to start returning objects. For use with limit query parameter. |
Updating a resource requires the resource id, and is typically done using an HTTP PUT or PATCH request, with the fields to modify in the request body.
The response usually returns a 200 OK response code upon success, with minimal information about the modified resource in the response body.
Deleting a resource requires the resource id and is typically executing via an HTTP DELETE request. The response usually returns a 204 No Content response code upon success.
Resource names are plural and expressed in camelCase.
Resource names are consistent between main URL and URL parameter.
Parameter lists are consistently enveloped in the following manner:
{ "data": [{ ... }] }Field names are in camelCase and are consistent with the resource name in the URL or with the embed URL parameter.
"creator": { "id": 1 },
"updater": { "id": 2 },When reading a resource, the platform supports an embed query parameter for most resources, which allows the caller to ask for associated resources in the response.
Use of this parameter requires knowledge of how different resources are related to each other and is suggested for advanced users only.
In the following example, the sub-jobs of a jobGroup are embedded in the response for jobGroup=1:
https://yourworkspace.cloud.trifacta.com/v4/jobGroups/1?embed=jobsIf you provide an invalid embedding, you will get an error message. The response will contain the list of possible resources that can be embedded. e.g.
https://yourworkspace.cloud.trifacta.com/v4/jobGroups/1?embed=*Example error:
{
"exception": {
"name": "ValidationFailed",
"message": "Input validation failed",
"details": "No association * in flows! Valid associations are creator, updater, snapshots..."
}
}It is possible to let the application know that you need fewer data to improve the performance of the endpoints using the fields query parameter. e.g.
https://yourworkspace.cloud.trifacta.com/v4/flows?fields=id;nameThe list of fields need to be separated by semi-colons ;. Note that the application might sometimes return more fields than requested.
You can also use it while embedding resources.
https://yourworkspace.cloud.trifacta.com/v4/flows?fields=id;name&embed=flownodes(fields=id)You can limit and sort the number of embedded resources for some associations. e.g.
https://yourworkspace.cloud.trifacta.com/v4/flows?fields=id&embed=flownodes(limit=1,fields=id,sort=-id)Note that not all association support this. An error is returned when it is not possible to limit the number of embedded results.
The Trifacta Wrangler Platform uses HTTP response codes to indicate the success or failure of an API request.
| HTTP Status Code (client errors) | Notes |
|---|---|
| 400 Bad Request | Potential reasons:
|
| 403 Forbidden | Incorrect permissions to access the Resource. |
| 404 Not Found | Resource cannot be found. |
| 410 Gone | Resource has been previously deleted. |
| 415 Unsupported Media Type | Incorrect Accept or Content-type header |
Each request has a request identifier, which can be found in the response headers, in the following form:
x-trifacta-request-id: <myRequestId>ℹ️ NOTE: If you have an issue with a specific request, please include the
x-trifacta-request-idvalue when you contact support
You can use a third party client, such as curl, HTTPie, Postman or the Insomnia rest client to test the Trifacta Wrangler API.
⚠️ When testing the API, bear in mind that you are working with your live production data, not sample data or test data.
Note that you will need to pass an API token with each request.
For e.g., here is how to run a job with curl:
curl -X POST 'https://yourworkspace.cloud.trifacta.com/v4/jobGroups' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <token>' \
-d '{ "wrangledDataset": { "id": "<recipe-id>" } }'Using a graphical tool such as Postman or Insomnia, it is possible to import the API specifications directly:
Note that with Postman, you can also generate code snippets by selecting a request and clicking on the Code button.
ℹ️ NOTE: Each request to the Trifacta Wrangler Platform must include authentication credentials.
API access tokens can be acquired and applied to your requests to obscure sensitive Personally Identifiable Information (PII) and are compliant with common privacy and security standards. These tokens last for a preconfigured time period and can be renewed as needed.
You can create and delete access tokens through the Settings area of the application. With each request, you submit the token as part of the Authorization header.
Authorization: Bearer <tokenValue>As needed, you can create and use additional tokens. There is no limit to the number of tokens you can create. See Manage API Access Tokens for more information.
| Security Scheme Type | HTTP |
|---|---|
| HTTP Authorization Scheme | bearer |
An object used to provide a simpler and more secure way of accessing the REST API endpoints of the Trifacta Wrangler Platform. Access tokens limit exposure of clear-text authentication values and provide an easy method of managing authentication outside of the browser. See the Authentication section for more information.
Create an API Access Token. See the Authentication section for more information about API Access Token.
⚠️ API tokens inherit the API access of the user who creates them. Treat tokens as passwords and keep them in a secure place.
This request requires you to be authenticated.
If you do not have a valid access token to use at this time, you must first create one using the UI.
If you have a valid access token, you can submit that token in your Authentication header with this request.
ref: createApiAccessToken
| lifetimeSeconds required | integer Lifetime in seconds for the access token. Set this value to -1 to create a non-expiring token. |
| description | string User-friendly description for the access token |
{- "lifetimeSeconds": -1,
- "description": "API access token description"
}{- "tokenValue": "eyJ0b2tlbklkIjoiYmFiOTA4ZjctZGNjMi00OTYyLTg1YmQtYzFlOTZkMGNhY2JkIiwic2VjcmV0IjoiOWIyNjQ5MWJiODM4ZWY0OWE1NzdhYzYxOWEwYTFkNjc4ZmE4NmE5MzBhZWFiZDk3OGRlOTY0ZWI0MDUyODhiOCJ9",
- "tokenInfo": {
- "tokenId": "0bc1d49f-5475-4c62-a0ba-6ad269389ada",
- "description": "API access token description",
- "expiredAt": "2019-08-24T14:15:22Z",
- "createdAt": "2019-08-24T14:15:22Z",
- "lastUsed": null
}
}List API Access Tokens of the current user
ref: listApiAccessTokens
{- "data": [
- {
- "tokenId": "0bc1d49f-5475-4c62-a0ba-6ad269389ada",
- "description": "API access token description",
- "expiredAt": "2019-08-24T14:15:22Z",
- "createdAt": "2019-08-24T14:15:22Z",
- "lastUsed": null
}
]
}Get an existing api access token
ref: getApiAccessToken
| tokenId required | string Example: 0bc1d49f-5475-4c62-a0ba-6ad269389ada |
{- "tokenId": "0bc1d49f-5475-4c62-a0ba-6ad269389ada",
- "description": "API access token description",
- "expiredAt": "2019-08-24T14:15:22Z",
- "createdAt": "2019-08-24T14:15:22Z",
- "lastUsed": null
}Delete the specified access token.
⚠️ If you delete an active access token, you may prevent the user from accessing the platform outside of the Trifacta application.
ref: deleteApiAccessToken
| tokenId required | string Example: 0bc1d49f-5475-4c62-a0ba-6ad269389ada |
An object containing information for accessing AWS S3 storage, including details like defaultBucket, credentials, etc.
Create a new AWS config
ref: createAwsConfig
| credentialProvider required | string Enum: "default" "temporary"
|
| defaultBucket | string Default S3 bucket where user can upload and write results |
| extraBuckets | Array of strings |
| role | string AWS IAM Role, required when credential provider is set to temporary |
| externalId | string This identifier is used to manage cross-account access in AWS. This value should not be modified. |
| key | string AWS key string, required when credential provider is set to default |
| secret | string AWS secret string, required when credential provider is set to default |
| personId | integer When creating an AWS configuration, an administrator can insert the personId parameter to assign the configuration to the internal identifier for the user. If this parameter is not included, the AWS configuration is assigned to the user who created it. |
{- "defaultBucket": "bucketName",
- "extraBuckets": [
- "bucket1"
], - "credentialProvider": "default",
- "role": "arn:aws:iam::xxxxxxxxxxxxx:role/sample-role",
- "externalId": "trifacta_****",
- "key": "string",
- "secret": "string",
- "personId": 1
}{- "defaultBucket": "bucketName",
- "extraBuckets": [
- "bucket1"
], - "credentialProvider": "default",
- "role": "arn:aws:iam::xxxxxxxxxxxxx:role/sample-role",
- "externalId": "trifacta_****",
- "credential": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "activeRoleId": 1
}| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "data": [
- {
- "defaultBucket": "bucketName",
- "extraBuckets": [
- "bucket1"
], - "credentialProvider": "default",
- "role": "arn:aws:iam::xxxxxxxxxxxxx:role/sample-role",
- "externalId": "trifacta_****",
- "credential": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "activeRoleId": 1
}
]
}The request body contains the parameters of the awsConfigs object that you wish to modify. You do not have to include parameters that are not being modified.
The following changes the default bucket for the AWS configuration object.
{ "defaultBucket": "testing2" }ref: updateAwsConfig
| id required | integer |
| id | integer unique identifier for this object. |
| defaultBucket | string Default S3 bucket where user can upload and write results |
| extraBuckets | Array of strings |
| credentialProvider | string Enum: "default" "temporary"
|
| role | string AWS IAM Role, required when credential provider is set to temporary |
| key | string AWS key string, required when credential provider is set to default |
| secret | string AWS secret string, required when credential provider is set to default |
| personId | integer When creating an AWS configuration, an administrator can insert the personId parameter to assign the configuration to the internal identifier for the user. If this parameter is not included, the AWS configuration is assigned to the user who created it. |
| externalId | string This identifier is used to manage cross-account access in AWS. This value should not be modified. |
{- "id": 1,
- "defaultBucket": "bucketName",
- "extraBuckets": [
- "bucket1"
], - "credentialProvider": "default",
- "role": "arn:aws:iam::xxxxxxxxxxxxx:role/sample-role",
- "key": "string",
- "secret": "string",
- "personId": 1,
- "externalId": "trifacta_****"
}{- "defaultBucket": "bucketName",
- "extraBuckets": [
- "bucket1"
], - "credentialProvider": "default",
- "role": "arn:aws:iam::xxxxxxxxxxxxx:role/sample-role",
- "externalId": "trifacta_****",
- "credential": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "activeRoleId": 1
}An object containing the AWS IAM Role ARN for authenticating aws resources when using role-base authentication, this object belongs to an awsConfig.
| role required | string |
| personId | integer |
{- "role": "string",
- "personId": 1
}{- "id": 1,
- "awsConfigId": 1,
- "role": "string",
- "createdFrom": "api",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deletedAt": "2019-08-24T14:15:22Z"
}List AWS roles for a user .
ref: listAwsRoles
| personId | integer person id |
{- "data": {
- "id": 1,
- "awsConfig": {
- "id": 1
}, - "role": "string",
- "createdFrom": "api",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deletedAt": "2019-08-24T14:15:22Z"
}
}Update an existing aws role
ℹ️ NOTE: Admin role is required to use this endpoint.
ref: updateAwsRole
| id required | integer |
| personId | integer |
| role | string |
| createdFrom | string Enum: "api" "idp" shows which means created the role |
| createdAt | string <date-time> The time this object was first created. |
| updatedAt | string <date-time> The time this object was last updated. |
| deletedAt | string <date-time> The time this object was deleted. |
{- "personId": 1,
- "role": "string",
- "createdFrom": "api",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deletedAt": "2019-08-24T14:15:22Z"
}{- "id": 1,
- "awsConfigId": 1,
- "role": "string",
- "createdFrom": "api",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deletedAt": "2019-08-24T14:15:22Z"
}Delete an existing aws role
ref: deleteAwsRole
| id required | integer |
An object representing Trifacta Wrangler's connection to an external data source. connections can be used for import, publishing, or both, depending on type.
Create a new connection
ref: createConnection
| vendor required | string String identifying the connection`s vendor |
| vendorName required | string Name of the vendor of the connection |
| type required | string Enum: "jdbc" "rest" "remotefile" Type of connection |
| credentialType required | string Enum: "basic" "securityToken" "iamRoleArn" "iamDbUser" "oauth2" "keySecret" "apiKey" "awsKeySecret" "basicWithAppToken" "basicApp" "transactionKey" "password" "apiKeyWithToken"
|
| name required | string Display name of the connection. |
| params required | object This setting is populated with any parameters that are passed to the source duringconnection and operations. For relational sources, this setting may include thedefault database and extra load parameters. |
| ssl | boolean When |
| description | string User-friendly description for the connection. |
| disableTypeInference | boolean If set to false, type inference has been disabled for this connection. The default is true. When type inference has been disabled, the Trifacta Wrangler Platform does not apply Trifacta Wrangler types to data when it is imported. |
| isGlobal | boolean If NOTE: After a connection has been made public, it cannot be made private again. It must be deleted and recreated. |
| credentialsShared | boolean If |
| host | string Host of the source |
| port | integer Port number for the source |
| bucket | string bucket name for the source |
| oauth2StateId | string |
Array of basic (object) or securityToken (object) or iamRoleArn (object) or iamDbUser (object) or oauth2 (object) or keySecret (object) or apiKey (object) or awsKeySecret (object) or basicWithAppToken (object) or basicApp (object) or transactionKey (object) or password (object) or apiKeyWithToken (object) (credentialsInfo) If present, these values are the credentials used to connect to the database. |
{- "vendor": "oracle",
- "vendorName": "oracle",
- "type": "jdbc",
- "name": "example_oracle_connection",
- "description": "This is an oracle connection",
- "disableTypeInference": false,
- "isGlobal": false,
- "credentialsShared": false,
- "host": "my_oracle_host",
- "port": 1521,
- "params": {
- "service": "my_oracle_service"
}, - "credentialType": "basic",
- "credentials": [
- {
- "username": "my_oracle_username",
- "password": "my_oracle_password"
}
]
}{- "vendor": "oracle",
- "vendorName": "oracle",
- "type": "jdbc",
- "credentialType": "basic",
- "ssl": true,
- "name": "example_oracle_connection",
- "description": "string",
- "disableTypeInference": true,
- "isGlobal": true,
- "credentialsShared": true,
- "host": "example.oracle.test",
- "port": 1521,
- "id": "55",
- "uuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "credentials": [
- {
- "username": "string",
- "password": "string"
}
], - "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "params": {
- "database": "dev"
}
}List existing connections
ref: listConnections
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| sharedRole | string Which type of role to list the connections |
{- "data": [
- {
- "vendor": "oracle",
- "vendorName": "oracle",
- "type": "jdbc",
- "credentialType": "basic",
- "ssl": true,
- "name": "example_oracle_connection",
- "description": "string",
- "disableTypeInference": true,
- "isGlobal": true,
- "credentialsShared": true,
- "host": "example.oracle.test",
- "port": 1521,
- "id": "55",
- "uuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "credentials": [
- {
- "username": "string",
- "password": "string"
}
], - "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "params": {
- "database": "dev"
}
}
]
}Count existing connections
ref: countConnections
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| sharedRole | string Which type of role to count the connections |
{- "count": 1
}Get an existing connection
ref: getConnection
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "vendor": "oracle",
- "vendorName": "oracle",
- "type": "jdbc",
- "credentialType": "basic",
- "ssl": true,
- "name": "example_oracle_connection",
- "description": "string",
- "disableTypeInference": true,
- "isGlobal": true,
- "credentialsShared": true,
- "host": "example.oracle.test",
- "port": 1521,
- "id": "55",
- "uuid": "f9cab740-50b7-11e9-ba15-93c82271a00b",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "credentials": [
- {
- "username": "string",
- "password": "string"
}
], - "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "params": {
- "database": "dev"
}
}Delete an existing connection
ref: deleteConnection
| id required | integer |
Get the connection status
ref: getConnectionStatus
| id required | integer |
{- "result": "string"
}Metadata that controls the behavior of JDBC connectors.
The terminology in the connectivity API is as follows:
The default configuration of each connector has been tuned for optimal performance and standardized type mapping behavior. If you require connector behavior changes, you can leverage the following APIs.
The specified overrides are merged into the current set of overrides for the connector. A new entry is created if no overrides currently exist.
The connector metadata stores a mapping for each Trifacta type to an official JDBC type
and database native type. When Trifacta publishes to a new table, it uses the first type specified
in the vendorTypeList. The rest of the types are used when validating the publish
action during design time.
As an example, let’s override the type mapping behavior for the Postgres connector.
By default it publishes Trifacta integers to bigint, but we can make it publish to
int instead. Make a GET request to /v4/connectormetadata/postgres to get the current
behavior. Locate the section called publishTypeMap and identify the element in the list where
trifactaType is INTEGER. We can see that the first element under the corresponding
vendorTypeList is bigint.
Since we want Postgres to write to int when creating string columns in a new table,
move that value to the beginning of the vendorTypeList. Send a POST request to
/v4/connectormetadata/postgres/overrides with the following body:
ℹ️ NOTE: Overriding the jdbcType is not supported behavior. Please use the same value from the default.
{
"publishMetadata": {
"publishTypeMap": [
{
"vendorTypeList": [
"int",
"bigint",
"int2",
"int4",
"int8",
"smallint",
"serial",
"bigserial",
"text",
"varchar",
"bpchar",
"char",
"character varying",
"character"
],
"jdbcType": 4,
"trifactaType": "INTEGER"
}
]
}
}Rerun the GET request to ensure the values are reflected.
The default performance configurations have been tuned to work well with the majority of systems. There are a few parameters that can be tuned if needed:
numberOfConnections: Number of connections that are used to write data in parallel.batchSize: Number of records written in each database batch.{
"publishMetadata": {
"performanceParams": {
"batchSize": 10000,
"numberOfConnections": 5
}
}
}The default performance configurations have been tuned to work well with the majority
of systems. One parameter that can be tuned is the database fetchSize. By default it is
set to a value of -1, which uses the default specified by the database driver. The following request
body can override this value:
{
"runtimeMetadata": {
"importPerformance": {"fetchSize": 1000}
}
}
| connector required | string |
object | |
object |
{- "publishMetadata": {
- "publishTypeMap": [
- {
- "vendorTypeList": [
- "int",
- "bigint",
- "int2",
- "int4",
- "int8",
- "smallint",
- "serial",
- "bigserial",
- "text",
- "varchar",
- "bpchar",
- "char",
- "character varying",
- "character"
], - "jdbcType": 4,
- "trifactaType": "INTEGER"
}
]
}
}Get the metadata overrides for a connector in a given workspace. These overrides are applied to the base configuration for connectivity operations.
| connector required | string |
{- "connectionMetadata": {
- "name": "string",
- "displayName": "string",
- "type": "string",
- "category": "relational",
- "status": "supported",
- "credentialTypes": [
- "basic"
], - "operation": "import",
- "connectionParameters": [
- {
- "name": "string",
- "displayName": "string",
- "type": "string",
- "required": true,
- "category": "string",
- "defaultValue": "string"
}
]
}, - "runtimeMetadata": {
- "defaultTypeTreatment": "WHITELIST",
- "typeMap": [
- {
- "vendorType": "string",
- "jdbcType": 1,
- "accessorClass": "string",
- "trifactaType": "Array",
- "classification": "WHITELIST"
}
], - "metadataAccessors": { },
- "pathMetadata": {
- "qualifiedPath": "CATALOG"
}, - "limit": {
- "table": "string",
- "query": "string"
}, - "errorHandlers": { },
- "importPerformance": {
- "fetchSize": 1,
- "disableAutoCommit": true,
- "schemaLimit": 1,
- "ormEnabled": true,
- "unload": {
- "stream": true,
- "cli": {
- "script": "string",
- "format": "string",
- "timeout": 1
}
}
}
}, - "publishMetadata": {
- "publishMethod": "direct",
- "publishTypeMap": [
- {
- "jdbcType": 1,
- "trifactaType": "string",
- "defaultValue": "string",
- "vendorTypeList": [
- "string"
]
}
], - "publishValidation": {
- "enabled": true,
- "maxTableNameLength": 1,
- "maxColumnNameLength": 1,
- "validTableNameRegex": "string",
- "validColNameRegex": "string"
}, - "publishQueries": {
- "createTable": "string",
- "createTempTable": "string",
- "copyTable": "string",
- "dropTable": "string",
- "insertTable": "string",
- "truncateTable": "string",
- "addColumn": "string"
}, - "performanceParams": {
- "batchProcessingEnabled": true,
- "batchLoggingEnabled": true,
- "batchSize": 1,
- "numberOfConnections": 1,
- "commitFrequency": 1,
- "queueSize": 1,
- "maxOfferToQueueRetryCount": 1,
- "maxPollFromQueueRetryCount": 1
}, - "publishInfo": {
- "qualifyingPath": [
- "string"
], - "supportedActions": [
- "create"
], - "supportedProtocols": [
- "string"
], - "externalFileFormats": [
- "pqt"
]
}
}
}Get the consolidated metadata for a connector in a given workspace. This metadata is used to defined connectivity, ingestion, and publishing for the connector.
ref: getConnectorConfig
| connector required | string |
{- "connectionMetadata": {
- "name": "string",
- "displayName": "string",
- "type": "string",
- "category": "relational",
- "status": "supported",
- "credentialTypes": [
- "basic"
], - "operation": "import",
- "connectionParameters": [
- {
- "name": "string",
- "displayName": "string",
- "type": "string",
- "required": true,
- "category": "string",
- "defaultValue": "string"
}
]
}, - "runtimeMetadata": {
- "defaultTypeTreatment": "WHITELIST",
- "typeMap": [
- {
- "vendorType": "string",
- "jdbcType": 1,
- "accessorClass": "string",
- "trifactaType": "Array",
- "classification": "WHITELIST"
}
], - "metadataAccessors": { },
- "pathMetadata": {
- "qualifiedPath": "CATALOG"
}, - "limit": {
- "table": "string",
- "query": "string"
}, - "errorHandlers": { },
- "importPerformance": {
- "fetchSize": 1,
- "disableAutoCommit": true,
- "schemaLimit": 1,
- "ormEnabled": true,
- "unload": {
- "stream": true,
- "cli": {
- "script": "string",
- "format": "string",
- "timeout": 1
}
}
}
}, - "publishMetadata": {
- "publishMethod": "direct",
- "publishTypeMap": [
- {
- "jdbcType": 1,
- "trifactaType": "string",
- "defaultValue": "string",
- "vendorTypeList": [
- "string"
]
}
], - "publishValidation": {
- "enabled": true,
- "maxTableNameLength": 1,
- "maxColumnNameLength": 1,
- "validTableNameRegex": "string",
- "validColNameRegex": "string"
}, - "publishQueries": {
- "createTable": "string",
- "createTempTable": "string",
- "copyTable": "string",
- "dropTable": "string",
- "insertTable": "string",
- "truncateTable": "string",
- "addColumn": "string"
}, - "performanceParams": {
- "batchProcessingEnabled": true,
- "batchLoggingEnabled": true,
- "batchSize": 1,
- "numberOfConnections": 1,
- "commitFrequency": 1,
- "queueSize": 1,
- "maxOfferToQueueRetryCount": 1,
- "maxPollFromQueueRetryCount": 1
}, - "publishInfo": {
- "qualifyingPath": [
- "string"
], - "supportedActions": [
- "create"
], - "supportedProtocols": [
- "string"
], - "externalFileFormats": [
- "pqt"
]
}
}
}Get the default metadata for a connector without applying custom overrides. This metadata is used to defined connectivity, ingestion, and publishing for the connector.
ref: getConnectorDefaults
| connector required | string |
{- "connectionMetadata": {
- "name": "string",
- "displayName": "string",
- "type": "string",
- "category": "relational",
- "status": "supported",
- "credentialTypes": [
- "basic"
], - "operation": "import",
- "connectionParameters": [
- {
- "name": "string",
- "displayName": "string",
- "type": "string",
- "required": true,
- "category": "string",
- "defaultValue": "string"
}
]
}, - "runtimeMetadata": {
- "defaultTypeTreatment": "WHITELIST",
- "typeMap": [
- {
- "vendorType": "string",
- "jdbcType": 1,
- "accessorClass": "string",
- "trifactaType": "Array",
- "classification": "WHITELIST"
}
], - "metadataAccessors": { },
- "pathMetadata": {
- "qualifiedPath": "CATALOG"
}, - "limit": {
- "table": "string",
- "query": "string"
}, - "errorHandlers": { },
- "importPerformance": {
- "fetchSize": 1,
- "disableAutoCommit": true,
- "schemaLimit": 1,
- "ormEnabled": true,
- "unload": {
- "stream": true,
- "cli": {
- "script": "string",
- "format": "string",
- "timeout": 1
}
}
}
}, - "publishMetadata": {
- "publishMethod": "direct",
- "publishTypeMap": [
- {
- "jdbcType": 1,
- "trifactaType": "string",
- "defaultValue": "string",
- "vendorTypeList": [
- "string"
]
}
], - "publishValidation": {
- "enabled": true,
- "maxTableNameLength": 1,
- "maxColumnNameLength": 1,
- "validTableNameRegex": "string",
- "validColNameRegex": "string"
}, - "publishQueries": {
- "createTable": "string",
- "createTempTable": "string",
- "copyTable": "string",
- "dropTable": "string",
- "insertTable": "string",
- "truncateTable": "string",
- "addColumn": "string"
}, - "performanceParams": {
- "batchProcessingEnabled": true,
- "batchLoggingEnabled": true,
- "batchSize": 1,
- "numberOfConnections": 1,
- "commitFrequency": 1,
- "queueSize": 1,
- "maxOfferToQueueRetryCount": 1,
- "maxPollFromQueueRetryCount": 1
}, - "publishInfo": {
- "qualifyingPath": [
- "string"
], - "supportedActions": [
- "create"
], - "supportedProtocols": [
- "string"
], - "externalFileFormats": [
- "pqt"
]
}
}
}Create a new environment parameter to be used in the workspace.
ℹ️ NOTE: Admin role is required to use this endpoint.
| overrideKey required | string key/name used when overriding the value of the variable |
required | object |
{- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "myValue"
}
}
}{- "id": 1,
- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "myValue"
}
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deleted_at": "2019-08-24T14:15:22Z",
- "usageInfo": {
- "runParameters": 1
}
}List existing environment parameters
| includeUsageInfo | string Include information about where the environment parameter is used. |
| filter | string Filter environment parameters using the attached overrideKey |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "data": [
- {
- "id": 1,
- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "myValue"
}
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deleted_at": "2019-08-24T14:15:22Z",
- "usageInfo": {
- "runParameters": 1
}
}
]
}Import the environment parameters from the given package.
A ZIP file as exported by the export environment parameters endpoint is accepted.
This endpoint accept a multipart/form content type.
Here is how to send the ZIP package using curl.
curl -X POST https://yourworkspace.cloud.trifacta.com/v4/environmentParameters/package \
-H 'authorization: Bearer <api-token>' \
-H 'content-type: multipart/form-data' \
-F 'data=@path/to/environment-parameters-package.zip'The response lists the objects that have been created.
ℹ️ NOTE: Admin role is required to use this endpoint.
| fromUI | boolean If true, will return the list of imported environment parameters for confirmation. |
{ }{- "data": [
- {
- "id": 1,
- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "myValue"
}
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deleted_at": "2019-08-24T14:15:22Z",
- "usageInfo": {
- "runParameters": 1
}
}
]
}Retrieve a package containing the list of environment parameters.
Response body is the contents of the package. Package contents are a ZIPped version of the list of environment parameters.
The environment parameters package can be used to import the environment parameters in another environment.
ℹ️ NOTE: Admin role is required to use this endpoint.
| hideSecrets | boolean If included, the secret values will be hidden. |
Get an existing environment parameter
ℹ️ NOTE: Admin role is required to use this endpoint.
| id required | integer |
| includeUsageInfo | string Include information about where the environment parameter is used. |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "id": 1,
- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "myValue"
}
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "deleted_at": "2019-08-24T14:15:22Z",
- "usageInfo": {
- "runParameters": 1
}
}A container for wrangling logic. Contains imported datasets, recipe, output objects, and References.
Create a new flow with specified name and optional description and target folder.
ℹ️ NOTE: You cannot add datasets to the flow through this endpoint. Moving pre-existing datasets into a flow is not supported in this release. Create the flow first and then when you create the datasets, associate them with the flow at the time of creation.
ref: createFlow
| name | string Display name of the flow. |
| description | string User-friendly description for the flow. |
object Settings for the flow. | |
| folderId | integer Internal identifier for a Flow folder. |
{- "name": "string",
- "description": "string",
- "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "folderId": 1
}{- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}List existing flows
ref: listFlows
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| flowsFilter | string Which types of flows to list. One of ['all', 'shared', 'owned'] |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}
]
}Import all flows from the given package.
A ZIP file as exported by the export Flow endpoint is accepted.
Before you import, you can perform a dry-run to check for errors. See Import Flow package - Dry run.
This endpoint accept a multipart/form content type.
Here is how to send the ZIP package using curl.
curl -X POST https://yourworkspace.cloud.trifacta.com/v4/flows/package \
-H 'authorization: Bearer <api-token>' \
-H 'content-type: multipart/form-data' \
-F 'data=@path/to/flow-package.zip'The response lists the objects that have been created.
ref: importPackage
| folderId | integer |
| fromUI | boolean If true, will return the list of imported environment parameters for confirmation if any are referenced in the flow. |
| File required | object (importFlowPackageRequestZip) An exported flow zip file. |
Array of environmentParameterMappingToExistingEnvParam (object) or environmentParameterMappingToManualValue (object) (environmentParameterMapping) |
{- "deletedObjects": { },
- "createdObjectMapping": { },
- "importRuleChanges": {
- "object": [
- { }
], - "value": [
- { }
]
}, - "primaryFlowIds": [
- 1
], - "flows": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}
], - "datasources": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
], - "flownodes": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}
], - "flowedges": [
- {
- "inPortId": 1,
- "outPortId": 1,
- "inputFlowNode": {
- "id": 1
}, - "outputFlowNode": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "recipes": [
- {
- "name": "string",
- "description": "string",
- "active": true,
- "nextPortId": 1,
- "currentEdit": {
- "id": 1
}, - "redoLeafEdit": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "outputobjects": [
- {
- "execution": "photon",
- "profiler": true,
- "checkSchemaDrift": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "webhookflowtasks": [
- { }
], - "release": { }
}Test importing flow package and return information about what objects would be created.
The same payload as for Import Flow package is expected.
ref: importPackageDryRun
| folderId | integer |
| File required | object (importFlowPackageRequestZip) An exported flow zip file. |
Array of environmentParameterMappingToExistingEnvParam (object) or environmentParameterMappingToManualValue (object) (environmentParameterMapping) |
{- "deletedObjects": { },
- "createdObjectMapping": { },
- "importRuleChanges": {
- "object": [
- { }
], - "value": [
- { }
]
}, - "primaryFlowIds": [
- 1
], - "flows": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}
], - "datasources": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
], - "flownodes": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}
], - "flowedges": [
- {
- "inPortId": 1,
- "outPortId": 1,
- "inputFlowNode": {
- "id": 1
}, - "outputFlowNode": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "recipes": [
- {
- "name": "string",
- "description": "string",
- "active": true,
- "nextPortId": 1,
- "currentEdit": {
- "id": 1
}, - "redoLeafEdit": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "outputobjects": [
- {
- "execution": "photon",
- "profiler": true,
- "checkSchemaDrift": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "webhookflowtasks": [
- { }
], - "release": { }
}Create a copy of this flow, as well as all contained recipes.
ref: copyFlow
| id required | integer |
| name | string name of the new copied flow. |
| description | string description of the new copied flow. |
| copyDatasources | boolean Default: false If true, Data sources will be copied (i.e. new imported datasets will be created, no data is copied on the file system). Otherwise, the existing imported datasets are reused. |
{- "name": "string",
- "description": "string",
- "copyDatasources": false
}{- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}Run all adhoc destinations in a flow.
(deprecated) If a scheduleExecutionId is provided, run all scheduled destinations in the flow.
The request body can stay empty. You can optionally pass parameters:
{
"runParameters": {
"overrides": {
"data": [{"key": "varRegion", "value": "02"}]
}
}
}You can also pass Spark Options that will be used for the Job run.
{
"sparkOptions": [
{"key": "spark.executor.memory", "value": "4GB"}
]
}Using recipe identifiers, you can specify a subset of outputs in the flow to run. See runJobGroup for more information on specifying wrangledDataset.
{"wrangledDatasetIds": [2, 3]}You can also override each outputs in the Flow using the recipe name.
{
"overrides": {
"my recipe name": {
"profiler": true,
"writesettings": [
{
"path": "<path_to_output_file>",
"action": "create",
"format": "csv",
"compression": "none",
"header": false,
"asSingleFile": false
}
]
}
}
}An array of jobGroup results is returned. Use the flowRunId if you want to track the status of the flow run. See Get Flow Run Status for more information.
ref: runFlow
| id required | integer |
| x-execution-id | string Example: f9cab740-50b7-11e9-ba15-93c82271a00b Optional header to safely retry the request without accidentally performing the same operation twice. If a FlowRun with the same |
| ignoreRecipeErrors | boolean Setting this flag to true will mean the job will run even if there are upstream recipe errors. Setting it to false will cause the Request to fail on recipe errors. |
object (runParameterOverrides) Allows to override parameters that are defined in the flow on datasets or outputs for e.g. | |
| scheduleExecutionId | integer |
Array of objects (outputObjectSparkOptionUpdateRequest) | |
Array of objects (databricksOptionsUpdateRequest) | |
| execution | string Enum: "photon" "emrSpark" Execution language. Indicate on which engine the job was executed. Can be null/missing for scheduled jobs that fail during the validation phase.
|
| wrangledDatasetIds | Array of integers Subset of outputs (identified by identifier of the recipe preceding the output) in this flow to run. When empty or unspecified, all outputs in the flow will be run. |
| overrides | object Overrides for each of the output object. Use the recipe name to specify the overrides. |
{ }{- "flowRunId": 1,
- "data": [
- {
- "id": 1,
- "flowRun": {
- "id": 1
}, - "jobs": {
- "data": [
- {
- "id": 1
}
]
}, - "jobGraph": {
- "edges": [
- {
- "source": 1,
- "target": 1
}
], - "vertices": [
- 1
]
}, - "reason": "Job started",
- "sessionId": "f9cab740-50b7-11e9-ba15-93c82271a00b"
}
]
}Count existing flows
ref: countFlows
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| flowsFilter | string Which types of flows to count. One of ['all', 'shared', 'owned'] |
{- "count": 1
}Get an existing flow
ref: getFlow
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}Update an existing flow based on the specified identifier.
ℹ️ NOTE: You cannot add datasets to the flow through this endpoint. Moving pre-existing datasets into a flow is not supported in this release. Create the flow first and then when you create the datasets, associate them with the flow at the time of creation.
ref: patchFlow
| id required | integer |
| name | string Display name of the flow. |
| description | string User-friendly description for the flow. |
object Settings for the flow. | |
| folderId | integer Internal identifier for a Flow folder. |
{- "name": "string",
- "description": "string",
- "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "folderId": 1
}{- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}Delete an existing flow
ref: deleteFlow
| id required | integer |
Retrieve a package containing the definition of the specified flow.
Response body is the contents of the package. Package contents are a ZIPped version of the flow definition.
The flow package can be used to import the flow in another environment. See the Import Flow Package for more information.
ref: getFlowPackage
| id required | integer |
| comment | string comment to be displayed when flow is imported in a deployment package |
Performs a dry-run of generating a flow package and exporting it, which performs a check of all permissions required to export the package.
If they occur, permissions errors are reported in the response.
ref: getFlowPackageDryRun
| id required | integer |
List flows, with special filtering behaviour
ref: listFlowsLibrary
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| flowsFilter | string Which types of flows to list. One of ['all', 'shared', 'owned'] |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}
]
}Count flows, with special filtering behaviour
ref: countFlowsLibrary
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| flowsFilter | string Which types of flows to count. One of ['all', 'shared', 'owned'] |
{- "count": {
- "flow": 1,
- "folder": 1,
- "all": 1
}
}List all the inputs of a Flow. Also include data sources that are present in referenced flows.
ref: getFlowInputs
| id required | integer |
{- "data": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
]
}List all the outputs of a Flow.
ref: getFlowOutputs
| id required | integer |
{- "data": [
- {
- "execution": "photon",
- "profiler": true,
- "checkSchemaDrift": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}Get all flows contained in this folder.
ref: getFlowsForFolder
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| flowsFilter | string Which types of flows to list. One of ['all', 'shared', 'owned'] |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}
]
}Get the count of flows contained in this folder.
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| flowsFilter | string Which types of flows to count. One of ['all', 'shared', 'owned'] |
{- "count": 1
}Replace the dataset or the specified wrangled dataset (flow node) in the flow with a new imported or wrangled dataset. This allows one to perform the same action as the "Replace" action in the flow UI.
You can get the flow node id (wrangled dataset id) and the imported it from the URL when clicking on a node in the UI.
ref: replaceDatasetInFlow
| id required | integer |
| flowNodeId required | integer |
| newImportedDatasetId required | integer |
{- "flowNodeId": 1,
- "newImportedDatasetId": 1
}{- "newInputNode": {
- "id": 1,
- "scriptId": 1,
- "flowId": 1
}, - "outputNodeEdges": [
- {
- "id": 1,
- "flowId": 1,
- "inFlowNodeId": 1,
- "outFlowNodeId": 1
}
]
}A placeholder for an object in a flow. Can represent an imported dataset, a recipe, or a Reference.
Create edges between nodes
ref: commitEdges
| id required | integer |
required | object |
{- "updateInfo": {
- "deleteOrphaned": true,
- "newEdges": [
- {
- "outPortId": 1,
- "inPortId": 1,
- "outFlowNodeId": 1,
- "inFlowNodeId": 1
}
], - "edgesToRevive": [
- {
- "id": 1
}
], - "portsToDelete": [
- {
- "id": 1
}
]
}
}{- "data": [
- {
- "inPortId": 1,
- "outPortId": 1,
- "inputFlowNode": {
- "id": 1
}, - "outputFlowNode": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}An object representing a flow run.
Get an existing flow run
ref: getFlowRun
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "scheduleExecutionId": 1,
- "requestId": "string",
- "flow": {
- "id": 1
}
}Get the status of a Flow Run. It combines the status of the underlying Job Groups.
ref: getFlowRunStatus
| id required | integer |
"Complete"Get the list of jobGroups.
ref: getFlowRunJobGroups
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "ranfrom": "ui",
- "ranfor": "recipe",
- "status": "Complete",
- "profilingEnabled": true,
- "runParameterReferenceDate": "2019-08-24T14:15:22Z",
- "snapshot": {
- "id": 1
}, - "wrangledDataset": {
- "id": 1
}, - "flowrun": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}Used to override the default value of runParameter in a flow
Create a new flow run parameter override
| flowId required | number |
| overrideKey required | string key/name used when overriding the value of the variable |
required | object |
{- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "myValue"
}
}, - "flowId": 0
}{- "id": 1,
- "flowId": 1,
- "overrideKey": "string",
- "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}Get an existing flow run parameter override
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "myValue"
}
}, - "flow": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}Patch an existing flow run parameter override
| id required | integer |
| overrideKey | string key/name used when overriding the value of the variable |
object |
{- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "myValue"
}
}
}{- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}A collection of flows, useful for organization.
Create a new folder
ref: createFolder
| name | string Display name of the folder. |
| description | string User-friendly description for the folder. |
{- "name": "string",
- "description": "string"
}{- "name": "string",
- "description": "string",
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}List existing folders
ref: listFolders
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
]
}Count existing folders
ref: countFolders
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "count": 1
}Get an existing folder
ref: getFolder
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "name": "string",
- "description": "string",
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}Patch an existing folder
ref: patchFolder
| id required | integer |
| name | string Display name of the folder. |
| description | string User-friendly description for the folder. |
{- "name": "string",
- "description": "string"
}{- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}Delete an existing folder
ref: deleteFolder
| id required | integer |
Get all flows contained in this folder.
ref: getFlowsForFolder
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| flowsFilter | string Which types of flows to list. One of ['all', 'shared', 'owned'] |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}
]
}Get the count of flows contained in this folder.
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| flowsFilter | string Which types of flows to count. One of ['all', 'shared', 'owned'] |
{- "count": 1
}An object representing data loaded into Trifacta Wrangler, as well as any structuring that has been applied to it. imported datasets are the starting point for wrangling, and can be used in multiple flows.
Create an imported dataset from an available resource. Created dataset is owned by the authenticated user.
In general, importing a file is done using the following payload:
{
"uri": "protocol://path-to-file",
"name": "my dataset",
"detectStructure": true
}See more examples in the Request Samples section.
✅ TIP: When an imported dataset is created via API, it is always imported as an unstructured dataset by default. To import a dataset with the inferred recipe, add
detectStructure: truein the payload.
ℹ️ NOTE: Do not create an imported dataset from a file that is being used by another imported dataset. If you delete the newly created imported dataset, the file is removed, and the other dataset is corrupted. Use a new file or make a copy of the first file first.
ℹ️ NOTE: Importing a Microsoft Excel file or a file that need to be converted using the API is not supported yet.
| name required | string Display name of the imported dataset. |
| uri required | string Dataset URI |
| description | string User-friendly description for the imported dataset. |
| type | string Indicate the type of dataset. If not specified, the default storage protocol is used. |
| isDynamic | boolean Default: false indicate if the datasource is parameterized. In that case, a |
| host | string Host for the dataset |
| userinfo | string User info for the dataset |
| detectStructure | boolean Default: false Indicate if a parsing script should be inferred when importing the dataset. By default, the dataset is imported |
| dynamicPath | string Path used when resolving the parameters. It is used when running a job or collecting a sample. It is different from the one used as a storage location which corresponds to the first match. The latter is used when doing a fast preview in the UI. |
| encoding | string Default: "UTF-8" Optional dataset encoding. |
| sanitizeColumnNames | boolean Default: false Indicate whether the column names in the imported file should be sanitized |
| ensureHeader | boolean If provided, forces first row header toggle |
Array of objects (runParameterFileBasedInfo) Description of the dataset parameters if the dataset is parameterized. |
{- "uri": "protocol://path-to-file",
- "name": "my dataset",
- "detectStructure": true
}{- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}Add the specified imported dataset to a flow based on its internal identifier.
ℹ️ NOTE: Datasets can be added to flows based on the permissions of the access token used on this endpoint. Datasets can be added to flows that are shared by the user.
| id required | integer |
required | object The flow to add this dataset to. |
{- "flow": {
- "id": 1
}
}{- "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}Create a copy of an imported dataset
ref: copyDataSource
| id required | integer |
| name | string name of the copied dataset |
{- "name": "string"
}{- "dynamicPath": "string",
- "isSchematized": true,
- "isDynamic": true,
- "isConverted": true,
- "disableTypeInference": true,
- "hasStructuring": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}List all the inputs of a Flow. Also include data sources that are present in referenced flows.
ref: getFlowInputs
| id required | integer |
{- "data": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
]
}Get the specified imported dataset.
Use the following embedded reference to embed in the response data about the connection used to acquire the source dataset if it was created from a custom connection. See embedding resources for more information.
/v4/importedDatasets/{id}?embed=connectionref: getImportedDataset
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}Modify the specified imported dataset. Name, path, bucket etc. for gcs can be modified.
ℹ️ NOTE: Samples will not be updated for the recipes. This results in the recipe showing samples of the older data.
| id required | integer |
| name | string Display name of the imported dataset. |
| description | string User-friendly description for the imported dataset. |
| type | string Indicate the type of dataset. If not specified, the default storage protocol is used. |
| isDynamic | boolean Default: false indicate if the datasource is parameterized. In that case, a |
| host | string Host for the dataset |
| userinfo | string User info for the dataset |
| bucket | string The bucket is required if the datasource is stored in a bucket file system. |
| raw | string Raw SQL query |
| path | string |
| dynamicPath | string Path used when resolving the parameters. It is used when running a job or collecting a sample. It is different from the one used as a storage location which corresponds to the first match. The latter is used when doing a fast preview in the UI. |
Array of objects (runParameterInfo) | |
| dynamicBucket | string |
| dynamicHost | string |
| dynamicUserInfo | string |
| isConverted | boolean Indicate if the imported dataset is converted. This is the case for Microsoft Excel Dataset for e.g. |
{- "name": "My Dataset",
- "description": "string",
- "type": "string",
- "isDynamic": false,
- "host": "string",
- "userinfo": "string",
- "bucket": "string",
- "raw": "SELECT * FROM table",
- "path": "string",
- "dynamicPath": "string",
- "runParameters": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
], - "dynamicBucket": "string",
- "dynamicHost": "string",
- "dynamicUserInfo": "string",
- "isConverted": true
}{- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}Modify the specified imported dataset. Only the name and description properties should be modified.
ref: patchImportedDataset
| id required | integer |
| name | string Display name of the imported dataset. |
| description | string User-friendly description for the imported dataset. |
{- "name": "My Dataset",
- "description": "string"
}{- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}List all the inputs that are linked to this output object. Also include data sources that are present in referenced flows.
| id required | integer |
{- "data": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
]
}List Trifacta Wrangler datasets.
This can be used to list both imported and reference datasets throughout the system,
as well as recipes in a given flow.
ref: listDatasetLibrary
required | string or Array of strings Which types of datasets to list.
Valid choices are: [ |
| ownershipFilter | string Which set of datasets to list.
One of [ |
| schematized | boolean If included, filter to only show schematized imported datasets. |
| currentFlowId | integer Required for including |
| datasourceFlowId | integer When included, filter included datasets to only include those associated to the given flow. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filter | string Example: filter=my-object Value for filtering objects. See |
| flowId | integer When provided, list datasets associated with this flow before other datasets. |
{- "data": [
- {
- "type": "datasource",
- "referenceCount": 1,
- "importedDataset": {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- null
], - "last": {
- "unit": null,
- "number": null,
- "dow": null
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
}
]
}Count Trifacta Wrangler datasets. Gives counts for various types of datasets matching the provided filters.
ref: countDatasetLibrary
| ownershipFilter | string Which set of datasets to count.
One of [ |
| schematized | boolean If included, filter to only show schematized imported datasets. |
| currentFlowId | integer Required for including |
| datasourceFlowId | integer When included, filter included datasets to only include those associated to the given flow. |
| flowId | integer When provided, count datasets associated with this flow before other datasets. |
| filter | string Example: filter=my-object Value for fuzzy-filtering objects. See |
{- "count": {
- "imported": 1,
- "reference": 1,
- "recipe": 1,
- "all": 1
}
}An internal object encoding the information necessary to run a part of a Trifacta Wrangler jobGroup.
This is called a "Stage" on the Job Results page in the UI.
Get information about the batch jobs within a Trifacta Wrangler job.
ref: getJobsForJobGroup
| id required | integer |
{- "data": [
- {
- "id": 1,
- "status": "Complete",
- "jobType": "wrangle",
- "sampleSize": 1,
- "percentComplete": 1,
- "jobGroup": {
- "id": 1
}, - "errorMessage": {
- "id": 1
}, - "lastHeartbeatAt": "2019-08-24T14:15:22Z",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "executionLanguage": "photon",
- "cpJobId": "string",
- "wranglescript": {
- "id": 1
}, - "emrcluster": {
- "id": 1
}
}
]
}Get Job Status.
ref: getJobStatus
| id required | integer |
"Complete"Create a jobGroup, which launches the specified job as the authenticated user. This performs the same action as clicking on the Run Job button in the application.
The request specification depends on one of the following conditions:
In the last case, you must specify some overrides when running the job. See the example with overrides for more information.
ℹ️ NOTE: Override values applied to a job are not validated. Invalid overrides may cause your job to fail.
To run a job, you just specify the recipe identifier (wrangledDataset.id). If the job is successful, all defined outputs are generated, as defined in the outputobject, publications, and writeSettings objects associated with the recipe.
✅ TIP: To identify the wrangledDataset Id, select the recipe icon in the flow view and take the id shown in the URL. e.g. if the URL is
/flows/10?recipe=7, the wrangledDataset Id is7.
{"wrangledDataset": {"id": 7}}If you must change some outputs or other settings for the specific job, you can insert these changes in the overrides section of the request. In the example below, the running environment, profiling option, and writeSettings for the job are modified for this execution.
{
"wrangledDataset": {"id": 1},
"overrides": {
"execution": "spark",
"profiler": false,
"writesettings": [
{
"path": "<path_to_output_file>",
"action": "create",
"format": "csv",
"compression": "none",
"header": false,
"asSingleFile": false
}
]
}
}If you have created a dataset with parameters, you can specify overrides for parameter values during execution through the APIs. Through this method, you can iterate job executions across all matching sources of a parameterized dataset.
In the example below, the runParameters override has been specified for the country. In this case, the value "Germany" is inserted for the specified variable as part of the job execution.
{
"wrangledDataset": {"id": 33},
"runParameters": {
"overrides": {
"data": [{"key": "country", "value": "Germany"}]
}
}
}The response contains a list of jobs which can be used to get a granular status of the JobGroup completion.
The jobGraph indicates the dependency between each of the jobs.
{
"sessionId": "79276c31-c58c-4e79-ae5e-fed1a25ebca1",
"reason": "JobStarted",
"jobGraph": {
"vertices": [21, 22],
"edges": [{"source": 21, "target": 22}]
},
"id": 9,
"jobs": {"data": [{"id": 21}, {"id": 22}]}
}When you create a new jobGroup through the APIs, the internal jobGroup identifier is returned in the response. Retain this identifier for future use. You can also acquire the jobGroup identifier from the application. In the Jobs page, the internal identifier for the jobGroup is the value in the left column.
ref: runJobGroup
| x-execution-id | string Example: f9cab740-50b7-11e9-ba15-93c82271a00b Optional header to safely retry the request without accidentally performing the same operation twice. If a JobGroup with the same |
required | object The identifier for the recipe you would like to run. |
| forceCacheUpdate | boolean Setting this flag to true will invalidate any cached datasources. This only applies to SQL datasets. |
| ignoreRecipeErrors | boolean Default: false Setting this flag to true will mean the job will run even if there are upstream recipe errors. Setting it to false will cause the Request to fail on recipe errors. |
| testMode | boolean Setting this flag to true will not run the job but just perform some validations. |
object (runParameterOverrides) Allows to override parameters that are defined in the flow on datasets or outputs for e.g. | |
| workspaceId | integer Internal. Does not need to be specified |
object Allows to override execution settings that are set on the output object. | |
| ranfrom | string Enum: "ui" "schedule" "api" Where the job was executed from. Does not need to be specified when using the API.
|
{- "wrangledDataset": {
- "id": 7
}
}{- "sessionId": "79276c31-c58c-4e79-ae5e-fed1a25ebca1",
- "reason": "JobStarted",
- "jobGraph": {
- "vertices": [
- 21,
- 22
], - "edges": [
- {
- "source": 21,
- "target": 22
}
]
}, - "id": 9,
- "jobs": {
- "data": [
- {
- "id": 21
}, - {
- "id": 22
}
]
}
}Deprecated. Use listJobLibrary instead.
ref: listJobGroups
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| flowNodeId | integer |
| ranfor | string Default: "recipe,plan" filter jobs based on their type |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "ranfrom": "ui",
- "ranfor": "recipe",
- "status": "Complete",
- "profilingEnabled": true,
- "runParameterReferenceDate": "2019-08-24T14:15:22Z",
- "snapshot": {
- "id": 1
}, - "wrangledDataset": {
- "id": 1
}, - "flowrun": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}Get the specified jobGroup.
A job group is a job that is executed from a specific node in a flow. The job group may contain:
It is possible to only get the current status for a jobGroup:
/v4/jobGroups/{id}/statusIn that case, the response status would simply be a string:
"Complete"If you wish to also get the related jobs and wrangledDataset, you can use embed. See embedding resources for more information.
/v4/jobGroups/{id}?embed=jobs,wrangledDatasetref: getJobGroup
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "name": "string",
- "description": "string",
- "ranfrom": "ui",
- "ranfor": "recipe",
- "status": "Complete",
- "profilingEnabled": true,
- "runParameterReferenceDate": "2019-08-24T14:15:22Z",
- "snapshot": {
- "id": 1
}, - "wrangledDataset": {
- "id": 1
}, - "flowrun": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}| id required | integer |
{- "profilerTypeCheckHistograms": {
- "property1": [
- {
- "key": "VALID",
- "count": 1
}
], - "property2": [
- {
- "key": "VALID",
- "count": 1
}
]
}, - "profilerValidValueHistograms": {
- "property1": [
- {
- "min": 0,
- "max": 0,
- "roundMin": 0,
- "roundMax": 0,
- "buckets": [
- {
- "pos": 1,
- "b": 1
}
], - "quartiles": {
- "q1": 0,
- "q2": 0,
- "q3": 0
}
}
], - "property2": [
- {
- "min": 0,
- "max": 0,
- "roundMin": 0,
- "roundMax": 0,
- "buckets": [
- {
- "pos": 1,
- "b": 1
}
], - "quartiles": {
- "q1": 0,
- "q2": 0,
- "q3": 0
}
}
]
}, - "profilerRules": {
- "property1": [
- {
- "id": 1,
- "type": "string",
- "comment": "string",
- "description": "string",
- "status": "pass",
- "updatedAt": "string",
- "passCount": 1,
- "failCount": 1,
- "totalCount": 1
}
], - "property2": [
- {
- "id": 1,
- "type": "string",
- "comment": "string",
- "description": "string",
- "status": "pass",
- "updatedAt": "string",
- "passCount": 1,
- "failCount": 1,
- "totalCount": 1
}
]
}, - "columnTypes": {
- "property1": [
- "string"
], - "property2": [
- "string"
]
}
}| id required | integer |
{- "profilerTypeCheckHistograms": {
- "property1": [
- {
- "key": "VALID",
- "count": 1
}
], - "property2": [
- {
- "key": "VALID",
- "count": 1
}
]
}, - "profilerValidValueHistograms": {
- "property1": [
- {
- "min": 0,
- "max": 0,
- "roundMin": 0,
- "roundMax": 0,
- "buckets": [
- {
- "pos": 1,
- "b": 1
}
], - "quartiles": {
- "q1": 0,
- "q2": 0,
- "q3": 0
}
}
], - "property2": [
- {
- "min": 0,
- "max": 0,
- "roundMin": 0,
- "roundMax": 0,
- "buckets": [
- {
- "pos": 1,
- "b": 1
}
], - "quartiles": {
- "q1": 0,
- "q2": 0,
- "q3": 0
}
}
]
}, - "profilerRules": {
- "property1": [
- {
- "id": 1,
- "type": "string",
- "comment": "string",
- "description": "string",
- "status": "pass",
- "updatedAt": "string",
- "passCount": 1,
- "failCount": 1,
- "totalCount": 1
}
], - "property2": [
- {
- "id": 1,
- "type": "string",
- "comment": "string",
- "description": "string",
- "status": "pass",
- "updatedAt": "string",
- "passCount": 1,
- "failCount": 1,
- "totalCount": 1
}
]
}, - "columnTypes": {
- "property1": [
- "string"
], - "property2": [
- "string"
]
}
}Get JobGroup Status.
ref: getJobGroupStatus
| id required | integer |
"Complete"Get the job group inputs. Return the list of datasets used when running this jobGroup.
ref: getJobGroupInputs
| id required | integer |
{- "data": [
- {
- "name": "string",
- "inputs": [
- {
- "vendor": "string",
- "databaseConnectString": "string",
- "relationalPath": [
- "string"
], - "table": "string",
- "action": "string",
- "query": [
- "string"
]
}
]
}
]
}Get the job group outputs. Return the list of tables and file paths used as output.
ref: getJobGroupOutputs
| id required | integer |
{- "files": [
- {
- "uri": "string",
- "fileType": "FILE",
- "isPrimaryOutput": true
}
], - "tables": [
- {
- "vendor": "string",
- "databaseConnectString": "string",
- "relationalPath": [
- "string"
], - "table": "string",
- "action": "string",
- "query": [
- "string"
]
}
]
}Get list of all jobGroup accessible to the authenticated user.
Note that it is possible to embed other resources while fetching the jobGroup list. e.g.:
/v4/jobLibrary/?embed=jobs,wrangledDatasetSee embedding resources for more information.
It is possible to filter jobGroups based on their status.
Here is how to get all jobGroups with a Failed status:
/v4/jobLibrary?status=FailedIt is possible to filter only scheduled jobGroups using the following request:
/v4/jobLibrary?ranfrom=scheduleIt is also possible to filter the jobGroups based on the Date. Here is an example:
/v4/jobLibrary?dateFilter[createdAt][gte]=1572994800000&dateFilter[updatedAt][lt]=1581375600000ref: listJobLibrary
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| dateFilter | object for filtering jobgroups by start and end date |
| ranfrom | string filter jobs based on how they were run |
| status | string filter jobs based on their status |
| ranfor | string Default: "recipe,plan" filter jobs based on their type |
{- "data": [
- {
- "name": "string",
- "description": "string",
- "ranfrom": "ui",
- "ranfor": "recipe",
- "status": "Complete",
- "profilingEnabled": true,
- "runParameterReferenceDate": "2019-08-24T14:15:22Z",
- "snapshot": {
- "id": 1
}, - "wrangledDataset": {
- "id": 1
}, - "flowrun": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}Count Trifacta Wrangler jobs with special filter capabilities. See listJobLibrary for some examples.
ref: countJobLibrary
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| dateFilter | object for filtering jobgroups by start and end date |
| ranfrom | string filter jobs based on how they were run |
| status | string filter jobs based on their status |
| ranfor | string Default: "recipe,plan" filter jobs based on their type |
{- "count": 1
}Get information about the batch jobs within a Trifacta Wrangler job.
ref: getJobsForJobGroup
| id required | integer |
{- "data": [
- {
- "id": 1,
- "status": "Complete",
- "jobType": "wrangle",
- "sampleSize": 1,
- "percentComplete": 1,
- "jobGroup": {
- "id": 1
}, - "errorMessage": {
- "id": 1
}, - "lastHeartbeatAt": "2019-08-24T14:15:22Z",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "executionLanguage": "photon",
- "cpJobId": "string",
- "wranglescript": {
- "id": 1
}, - "emrcluster": {
- "id": 1
}
}
]
}Get list of publications for the specified jobGroup.
A publication is an export of job results from the platform after they have been initially generated.
| id required | integer |
{- "data": [
- {
- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputobject": {
- "id": 1
}, - "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
]
}An object containing a list of scriptLines that can be reused across recipes.
Performs an import of a macro package.
ℹ️ NOTE: You cannot import a macro that was exported from a later version of the product.
✅ TIP: You can paste the response of the exported macro page as the request.
ℹ️ NOTE: Modification of the macro definition is not supported outside of the Trifacta Wrangler.
ref: importMacroPackage
| type required | string Type of artifact. This value is always |
| kind required | string This value is |
| hash required | string Hash value used to verify the internal integrity of the macro definition. |
required | object |
required | object |
{- "type": "string",
- "kind": "string",
- "hash": "string",
- "data": {
- "name": "string",
- "description": "string",
- "signature": [
- {
- "name": "Store_Nbr",
- "type": "column"
}
], - "scriptlines": [
- {
- "hash": "string",
- "task": { }
}
]
}, - "metadata": {
- "lastMigration": "20191024143300",
- "trifactaVersion": "6.8.0+4.20191104073802.8b6217a",
- "exportedAt": "2019-08-24T14:15:22Z",
- "exportedBy": 1,
- "uuid": "6b27eee0-0034-11ea-a378-9dc0586de9fb",
- "edition": "Enterprise"
}
}{- "id": 1,
- "name": "string",
- "description": "string",
- "createdBy": 1,
- "updatedBy": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "workspaceId": 1
}Retrieve a package containing the definition of the specified macro. Response body is the contents of the package, which is an importable version of the macro definition.
✅ TIP: The response body can be pasted as the request when you import the macro into a different environment. For more information, see Import Macro Package.
ℹ️ NOTE: Modification of the macro definition is not supported outside of the Trifacta Wrangler.
ref: getMacroPackage
| id required | integer |
{- "type": "string",
- "kind": "string",
- "hash": "string",
- "data": {
- "name": "string",
- "description": "string",
- "signature": [
- {
- "name": "Store_Nbr",
- "type": "column"
}
], - "scriptlines": [
- {
- "hash": "string",
- "task": { }
}
]
}, - "metadata": {
- "lastMigration": "20191024143300",
- "trifactaVersion": "6.8.0+4.20191104073802.8b6217a",
- "exportedAt": "2019-08-24T14:15:22Z",
- "exportedBy": 1,
- "uuid": "6b27eee0-0034-11ea-a378-9dc0586de9fb",
- "edition": "Enterprise"
}
}{ }An outputObject is a definition of one or more types of outputs and how they are generated.
If an outputObject already exists for the recipe (flowNodeId) to which you are posting, you must either modify the object instead or delete it before posting your new object.
ref: createOutputObject
| execution required | string Enum: "photon" "emrSpark" Execution language. Indicate on which engine the job was executed. Can be null/missing for scheduled jobs that fail during the validation phase.
|
| profiler required | boolean Indicate if recipe errors should be ignored for the jobGroup. |
| isAdhoc | |
| ignoreRecipeErrors | |
| checkSchemaDrift | boolean Indicate if schema drift checks should be performed for the jobGroup. |
| flowNodeId | integer FlowNode the outputObject should be attached to. (This is also the id of the wrangledDataset). |
Array of objects (writeSettingCreateRequest) Optionally you can include writeSettings while creating the outputObject | |
Array of objects (sqlScriptCreateRequest) Optionally you can include sqlScripts while creating the outputObject | |
Array of objects (publicationCreateRequest) Optionally you can include publications while creating the outputObject |
{- "execution": "photon",
- "profiler": true,
- "isAdhoc": true,
- "ignoreRecipeErrors": true,
- "checkSchemaDrift": true,
- "flowNodeId": 1,
- "writeSettings": [
- {
- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "outputObjectId": 1,
- "runParameters": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}
}
}
], - "connectionId": "25"
}
], - "sqlScripts": [
- {
- "sqlScript": "string",
- "type": "string",
- "vendor": "string",
- "outputObjectId": "21",
- "connectionId": "55",
- "runParameters": [
- {
- "type": "sql",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}
}
}
]
}
], - "publications": [
- {
- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputObjectId": 1,
- "connectionId": "55",
- "runParameters": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}
}
}
], - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
]
}{- "execution": "photon",
- "profiler": true,
- "checkSchemaDrift": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}List existing output objects
ref: listOutputObjects
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "data": [
- {
- "execution": "photon",
- "profiler": true,
- "checkSchemaDrift": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}List all the outputs of a Flow.
ref: getFlowOutputs
| id required | integer |
{- "data": [
- {
- "execution": "photon",
- "profiler": true,
- "checkSchemaDrift": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}Count existing output objects
ref: countOutputObjects
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "count": 1
}Get the specified outputObject.
Note that it is possible to include writeSettings and publications that are linked to this outputObject. See embedding resources for more information.
/v4/outputObjects/{id}?embed=writeSettings,publicationsref: getOutputObject
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "execution": "photon",
- "profiler": true,
- "checkSchemaDrift": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}Patch an existing output object
ref: patchOutputObject
| id required | integer |
| execution | string Enum: "photon" "emrSpark" Execution language. Indicate on which engine the job was executed. Can be null/missing for scheduled jobs that fail during the validation phase.
|
| profiler | boolean Indicate if recipe errors should be ignored for the jobGroup. |
| ignoreRecipeErrors | |
| checkSchemaDrift | boolean Indicate if schema drift checks should be performed for the jobGroup. |
Array of objects (writeSettingCreateRequest) | |
Array of objects (sqlScriptCreateRequest) | |
Array of objects (publicationCreateRequest) |
{- "execution": "photon",
- "profiler": true,
- "ignoreRecipeErrors": true,
- "checkSchemaDrift": true,
- "writeSettings": [
- {
- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "outputObjectId": 1,
- "runParameters": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}
}
}
], - "connectionId": "25"
}
], - "sqlScripts": [
- {
- "sqlScript": "string",
- "type": "string",
- "vendor": "string",
- "outputObjectId": "21",
- "connectionId": "55",
- "runParameters": [
- {
- "type": "sql",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}
}
}
]
}
], - "publications": [
- {
- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputObjectId": 1,
- "connectionId": "55",
- "runParameters": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}
}
}
], - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
]
}{- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}Delete an existing output object
ref: deleteOutputObject
| id required | integer |
List all the inputs that are linked to this output object. Also include data sources that are present in referenced flows.
| id required | integer |
{- "data": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "dateRange": {
- "timezone": "string",
- "formats": [
- "string"
], - "last": {
- "unit": "years",
- "number": 1,
- "dow": 1
}
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
]
}Get information about the currently logged-in user.
ref: getCurrentPerson
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| uuid | string |
| workspaceId | string |
| includePrivileges | boolean Include the user's maximal privileges and authorization roles |
{- "email": "joe@example.com",
- "isDisabled": false,
- "state": "active",
- "id": 1,
- "outputHomeDir": "/home-dir/queryResults/joe@example.com",
- "uploadDir": "/uploads",
- "lastLoginTime": "2019-08-24T14:15:22Z",
- "lastStateChange": "2019-08-24T14:15:22Z",
- "maximalPrivileges": [
- {
- "operations": [
- "read"
], - "resourceType": "flow"
}
]
}Get an existing person
ref: getPerson
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| uuid | string |
| workspaceId | string |
| includePrivileges | boolean Include the user's maximal privileges and authorization roles |
{- "email": "joe@example.com",
- "isDisabled": false,
- "state": "active",
- "id": 1,
- "outputHomeDir": "/home-dir/queryResults/joe@example.com",
- "uploadDir": "/uploads",
- "lastLoginTime": "2019-08-24T14:15:22Z",
- "lastStateChange": "2019-08-24T14:15:22Z",
- "maximalPrivileges": [
- {
- "operations": [
- "read"
], - "resourceType": "flow"
}
]
}List existing people
ref: listPerson
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
string | |
| state | string |
| isDisabled | string |
| includePrivileges | boolean Include the user's maximal privileges and authorization roles |
| noLimit | string If set to |
{- "data": [
- {
- "email": "joe@example.com",
- "isDisabled": false,
- "state": "active",
- "id": 1,
- "outputHomeDir": "/home-dir/queryResults/joe@example.com",
- "uploadDir": "/uploads",
- "lastLoginTime": "2019-08-24T14:15:22Z",
- "lastStateChange": "2019-08-24T14:15:22Z",
- "maximalPrivileges": [
- {
- "operations": [
- "read"
], - "resourceType": "flow"
}
]
}
]
}Create a new plan
ref: createPlan
| name required | string Display name of the flow. |
| description | string User-friendly description for the flow. |
| originalPlanId | integer unique identifier for this object. |
{- "name": "string",
- "description": "string",
- "originalPlanId": 1
}{- "id": 1,
- "name": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "snapshotted": true,
- "originalPlanId": 1,
- "description": "string",
- "planSnapshotRunCount": 1,
- "notificationsEnabled": true,
- "latestPlanSnapshot": { },
- "latestPlanSnapshotRun": { }
}List existing plans
ref: listPlans
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| includeAssociatedPeople | boolean If true, the returned plans will include a list of people with access. |
| ownershipFilter | string Filter plans by ownership. Valid values are 'all', 'shared', and 'owned'. |
{- "data": [
- {
- "id": 1,
- "name": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "snapshotted": true,
- "originalPlanId": 1,
- "description": "string",
- "planSnapshotRunCount": 1,
- "notificationsEnabled": true,
- "latestPlanSnapshot": { },
- "latestPlanSnapshotRun": { }
}
]
}Run the plan. A new snapshot will be created if required.
If some flows or outputs referenced by the plan tasks have been deleted, it will return a
MissingFlowReferences validation status.
If the plan is valid, it will be queued for execution.
This endpoint returns a planSnapshotRunId that can be used to track the plan execution status
using getPlanSnapshotRun.
ref: runPlan
| id required | integer |
| x-execution-id | string Example: f9cab740-50b7-11e9-ba15-93c82271a00b Optional header to safely retry the request without accidentally performing the same operation twice.
If a |
Array of objects (planNodeOverride) Collection of run parameter overrides that should be applied to flow run parameters of the respective plan node. |
{- "planNodeOverrides": [
- {
- "handle": "string",
- "overrideKey": "string",
- "value": "string"
}
]
}{- "validationStatus": "Valid",
- "planSnapshotRunId": 1
}Get a list of users with whom the plan is shared.
ref: getPlanPermissions
| id required | integer |
{- "data": [
- {
- "id": 1,
- "email": "joe@example.com",
- "name": "Joe Guy"
}
]
}Import the plan and associated flows from the given package.
A ZIP file as exported by the export plan endpoint is accepted.
Before you import, you can perform a dry-run to check for errors. See Import plan package - dry run.
This endpoint accept a multipart/form content type.
Here is how to send the ZIP package using curl.
curl -X POST https://yourworkspace.cloud.trifacta.com/v4/plans/package \
-H 'authorization: Bearer <api-token>' \
-H 'content-type: multipart/form-data' \
-F 'data=@path/to/plan-package.zip'The response lists the objects that have been created.
ref: importPlanPackage
| folderId | integer |
| fromUI | boolean If true, will return the list of imported environment parameters for confirmation if any are referenced in the plan. |
| packageContents required | object (importPlanPackageRequestZip) An exported plan zip file. |
required | Array of environmentParameterMappingToExistingEnvParam (object) or environmentParameterMappingToManualValue (object) (environmentParameterMapping) |
{- "packageContents": { },
- "environmentParameterMapping": [
- {
- "overrideKey": "myVar",
- "mappedOverrideKey": "myVar"
}
]
}{- "flowPackages": [
- {
- "deletedObjects": { },
- "createdObjectMapping": { },
- "importRuleChanges": {
- "object": [
- { }
], - "value": [
- { }
]
}, - "primaryFlowIds": [
- 1
], - "flows": [
- {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}
], - "datasources": [
- {
- "dynamicPath": "string",
- "isDynamic": false,
- "isConverted": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "runParameters": {
- "data": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": null,
- "order": null
}
], - "value": {
- "dateRange": {
- "timezone": null,
- "formats": [ ],
- "last": { }
}
}
}
]
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "name": "My Dataset",
- "description": "string"
}
], - "flownodes": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}
], - "flowedges": [
- {
- "inPortId": 1,
- "outPortId": 1,
- "inputFlowNode": {
- "id": 1
}, - "outputFlowNode": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "recipes": [
- {
- "name": "string",
- "description": "string",
- "active": true,
- "nextPortId": 1,
- "currentEdit": {
- "id": 1
}, - "redoLeafEdit": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "outputobjects": [
- {
- "execution": "photon",
- "profiler": true,
- "checkSchemaDrift": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
], - "webhookflowtasks": [
- { }
], - "release": { }
}
], - "planPackage": {
- "id": 1,
- "name": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "snapshotted": true,
- "originalPlanId": 1,
- "description": "string",
- "planSnapshotRunCount": 1,
- "notificationsEnabled": true,
- "latestPlanSnapshot": { },
- "latestPlanSnapshotRun": { }
}, - "taskCount": 1
}Count existing plans
ref: countPlans
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| ownershipFilter | string Filter plans by ownership. |
{- "count": 1
}List run parameters of a plan. Parameters will be grouped by plannode. Each element in the returned list will only contain resources that have run parameters defined.
ref: planRunParameters
| id required | integer |
{- "planNodeParameters": [
- {
- "handle": "string",
- "planNodeId": 1,
- "flow": {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}, - "conflicts": [
- null
], - "datasources": {
- "data": [
- {
- "dynamicPath": "string",
- "isSchematized": true,
- "isDynamic": true,
- "isConverted": true,
- "disableTypeInference": true,
- "hasStructuring": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
]
}, - "outputObjects": {
- "data": [
- {
- "execution": "photon",
- "profiler": true,
- "checkSchemaDrift": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}, - "planOverrides": { }
}
]
}Read full plan with all its nodes, tasks, and edges.
ref: readFull
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| includeAssociatedPeople | boolean If true, the returned plan will include a list of people with access. |
| includeCreatorInfo | boolean If true, the returned plan will include info about the creators of the flows and plan such as name and email adress. |
{- "id": 1,
- "name": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "snapshotted": true,
- "originalPlanId": 1,
- "description": "string",
- "planSnapshotRunCount": 1,
- "notificationsEnabled": true,
- "latestPlanSnapshot": { },
- "latestPlanSnapshotRun": { }
}List of all schedules configured in the plan.
ref: getSchedulesForPlan
| id required | integer |
{- "data": [
- {
- "name": "string",
- "triggers": [
- {
- "timeBased": {
- "cron": {
- "expression": "string"
}, - "timezone": "string"
}
}
], - "tasks": [
- {
- "runFlow": {
- "flowId": 1
}
}
], - "enabled": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "createdBy": 1,
- "updatedBy": 1
}
]
}Retrieve a package containing the definition of the specified plan.
Response body is the contents of the package. Package contents are a ZIPped version of the plan definition.
The plan package can be used to import the plan in another environment. See the Import Plan Package for more information.
ref: getPlanPackage
| id required | integer |
| comment | string comment to be displayed when plan is imported in a deployment package |
Update plan properties, e.g. name and description
ref: updatePlan
| id required | integer |
| name | string Display name of the flow. |
| description | string User-friendly description for the flow. |
| notificationsEnabled | boolean Indicate if notification will be sent for this plan |
{- "name": "string",
- "description": "string",
- "notificationsEnabled": true
}{- "id": 1,
- "name": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}, - "snapshotted": true,
- "originalPlanId": 1,
- "description": "string",
- "planSnapshotRunCount": 1,
- "notificationsEnabled": true,
- "latestPlanSnapshot": { },
- "latestPlanSnapshotRun": { }
}Delete plan and remove associated schedules.
ref: deletePlan
| id required | integer |
A node representing a task in the plan graph.
Create a new plan node
ref: createPlanNode
| planId required | integer unique identifier for this object. |
| taskType required | string Enum: "flow" "http" |
required | planFlowTaskCreateRequest (object) or planHTTPTaskCreateRequest (object) |
| name required | string |
| inPlanNodeIds | Array of integers |
| outPlanNodeIds | Array of integers |
{- "planId": 1,
- "taskType": "flow",
- "task": {
- "flowId": 1,
- "flowNodeIds": [
- 1
]
}, - "name": "string",
- "inPlanNodeIds": [
- 1
], - "outPlanNodeIds": [
- 1
]
}{- "id": 1,
- "taskType": "flow",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}List run parameters of a plan node. Only resources with run parameters will be included in the response.
| id required | integer |
{- "handle": "string",
- "planNodeId": 1,
- "conflicts": [
- null
], - "flow": {
- "name": "string",
- "description": "string",
- "folder": {
- "id": 1
}, - "id": 1,
- "defaultOutputDir": "string",
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "settings": {
- "optimize": "enabled",
- "optimizers": {
- "columnPruning": "enabled"
}
}, - "workspace": {
- "id": 1
}, - "flowState": {
- "isOpened": true,
- "flow": {
- "id": 1
}, - "person": {
- "id": 1
}, - "zoom": 0,
- "offsetX": 0,
- "offsetY": 0
}
}, - "datasources": {
- "data": [
- {
- "dynamicPath": "string",
- "isSchematized": true,
- "isDynamic": true,
- "isConverted": true,
- "disableTypeInference": true,
- "hasStructuring": true,
- "parsingScript": {
- "id": 1
}, - "storageLocation": {
- "id": 1
}, - "connection": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "workspace": {
- "id": 1
}
}
]
}, - "outputObjects": {
- "data": [
- {
- "execution": "photon",
- "profiler": true,
- "checkSchemaDrift": true,
- "isAdhoc": true,
- "flownode": {
- "id": 1
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}, - "planOverrides": { }
}Delete an existing plan node
ref: deletePlanNode
| id required | integer |
Used to override the default value of runParameter in a plan for future executions.
Create a new plan override
ref: createPlanOverride
| planNodeId required | integer |
| overrideKey required | string key/name used when overriding the value of the variable |
required | object (planRunParameterVariableSchema) |
{- "planNodeId": 1,
- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "string"
}
}
}{ }Update an existing plan override
ref: updatePlanOverride
| id required | integer |
| planNodeId required | integer |
| overrideKey required | string key/name used when overriding the value of the variable |
required | object (planRunParameterVariableSchema) |
{- "planNodeId": 1,
- "overrideKey": "myVar",
- "value": {
- "variable": {
- "value": "string"
}
}
}{ }An execution of a plan's snapshot state
Cancel the plan execution.
| id required | integer |
{ }{- "id": 1,
- "status": "Complete",
- "createdAt": "2019-08-24T14:15:22Z",
- "finishedAt": "2019-08-24T14:15:22Z",
- "startedAt": "2019-08-24T14:15:22Z",
- "submittedAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "failedToCancelSomeJobs": true,
- "plan": {
- "id": 1
}, - "nextRun": {
- "id": "string"
}, - "previousRun": {
- "id": "string"
}
}List existing plan snapshot runs
ref: listPlanSnapshotRuns
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| status | string filter plan executions based on their status |
| dateFilter | object for filtering plan runs by start and end date |
| ranfrom | string filter plan runs based on how they were run |
{- "data": [
- {
- "id": 1,
- "status": "Complete",
- "createdAt": "2019-08-24T14:15:22Z",
- "finishedAt": "2019-08-24T14:15:22Z",
- "startedAt": "2019-08-24T14:15:22Z",
- "submittedAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "failedToCancelSomeJobs": true,
- "plan": {
- "id": 1
}, - "nextRun": {
- "id": "string"
}, - "previousRun": {
- "id": "string"
}
}
]
}Count existing plan snapshot runs
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
| status | string filter plan executions based on their status |
| dateFilter | object for filtering plan runs by start and end date |
| ranfrom | string filter plan runs based on how they were run |
{- "count": 1
}Return a plan snapshot run that contains the current status of a plan execution
ref: getPlanSnapshotRun
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| includeFlowCreatorInfo | string Include info about flow creators such as name and email adress. |
{- "id": 1,
- "status": "Complete",
- "createdAt": "2019-08-24T14:15:22Z",
- "finishedAt": "2019-08-24T14:15:22Z",
- "startedAt": "2019-08-24T14:15:22Z",
- "submittedAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "failedToCancelSomeJobs": true,
- "plan": {
- "id": 1
}, - "nextRun": {
- "id": "string"
}, - "previousRun": {
- "id": "string"
}
}Get the schedule definition that triggered the plan snapshot run.
| id required | integer |
{- "name": "string",
- "triggers": [
- {
- "timeBased": {
- "cron": {
- "expression": "string"
}, - "timezone": "string"
}
}
], - "tasks": [
- {
- "runFlow": {
- "flowId": 1
}
}
], - "enabled": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "createdBy": 1,
- "updatedBy": 1
}A publication object is used to specify a table-based output and is associated with an outputObject. Settings include the connection to use, path, table type, and write action to apply.
Create a new publication
ref: createPublication
| path required | Array of strings path to the location of the table/datasource. |
| tableName required | string name of the table |
| targetType required | string e.g. |
| action required | string Enum: "create" "load" "createAndLoad" "truncateAndLoad" "dropAndLoad" "upsert" Type of writing action to perform with the results
|
| outputObjectId | integer outputObject to attach this publication to. |
connectionIdString (string) or connectionIdBigQuery (string) | |
Array of objects (runParameterDestinationInfo) Optional Parameters that can be used to parameterized the | |
| parameters | object Additional publication parameters specific to each JDBC data source. Example: isDeltaTable=true for Databricks connections to produce Delta Lake Tables |
{- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputObjectId": 1,
- "connectionId": "55",
- "runParameters": [
- {
- "type": "path",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}
}
}
], - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}{- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputobject": {
- "id": 1
}, - "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}List existing publications
ref: listPublications
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "data": [
- {
- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputobject": {
- "id": 1
}, - "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}
]
}Count existing publications
ref: countPublications
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "count": 1
}Get an existing publication
ref: getPublication
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "outputobject": {
- "id": 1
}, - "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}Patch an existing publication
ref: patchPublication
| id required | integer |
| path | Array of strings path to the location of the table/datasource. |
| tableName | string name of the table |
| targetType | string e.g. |
| action | string Enum: "create" "load" "createAndLoad" "truncateAndLoad" "dropAndLoad" "upsert" Type of writing action to perform with the results
|
| parameters | object Additional publication parameters specific to each JDBC data source. Example: isDeltaTable=true for Databricks connections to produce Delta Lake Tables |
{- "path": [
- "string"
], - "tableName": "string",
- "targetType": "string",
- "action": "create",
- "parameters": {
- "property1": {
- "type": "string",
- "default": null
}, - "property2": {
- "type": "string",
- "default": null
}
}
}{- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}Delete an existing publication
ref: deletePublication
| id required | integer |
Contains information about repeated execution of a flow.
Enable a schedule
ref: enableSchedule
| id required | integer |
{ }{- "name": "string",
- "triggers": [
- {
- "timeBased": {
- "cron": {
- "expression": "string"
}, - "timezone": "string"
}
}
], - "tasks": [
- {
- "runFlow": {
- "flowId": 1
}
}
], - "enabled": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "createdBy": 1,
- "updatedBy": 1
}Disable a schedule
ref: disableSchedule
| id required | integer |
{ }{- "name": "string",
- "triggers": [
- {
- "timeBased": {
- "cron": {
- "expression": "string"
}, - "timezone": "string"
}
}
], - "tasks": [
- {
- "runFlow": {
- "flowId": 1
}
}
], - "enabled": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "createdBy": 1,
- "updatedBy": 1
}list schedules owned by the current user
ref: listSchedules
| filter | string Filter schedules using the attached flow name |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
{- "data": [
- {
- "name": "string",
- "triggers": [
- {
- "timeBased": {
- "cron": {
- "expression": "string"
}, - "timezone": "string"
}
}
], - "tasks": [
- {
- "runFlow": {
- "flowId": 1,
- "id": 1,
- "name": "string",
- "description": "string",
- "deleted_at": "string",
- "cpProject": "string",
- "workspaceId": 1,
- "folderId": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "createdBy": 1,
- "creator": { },
- "updatedBy": 1
}
}
], - "enabled": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "createdBy": 1,
- "updatedBy": 1
}
]
}count schedules owned by the current user
ref: countSchedules
| filter | string Filter schedules using the attached flow name |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
{- "count": 1
}Fetch a schedule
ref: getSchedule
| id required | integer |
{- "name": "string",
- "triggers": [
- {
- "timeBased": {
- "cron": {
- "expression": "string"
}, - "timezone": "string"
}
}
], - "tasks": [
- {
- "runFlow": {
- "flowId": 1,
- "id": 1,
- "name": "string",
- "description": "string",
- "deleted_at": "string",
- "cpProject": "string",
- "workspaceId": 1,
- "folderId": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "createdBy": 1,
- "creator": { },
- "updatedBy": 1
}
}
], - "enabled": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "createdBy": 1,
- "updatedBy": 1
}A sqlScript object is used to specify arbitrary SQLs to be run and is associated with an outputObject. Settings include the connection to use and sql type (pre/post),
Create a new sql script
ref: createSqlScript
| sqlScript required | string String of SQL queries to be executed. |
| type required | string Identifier to decide if the SQLs will be executed before or after a job. |
| vendor required | string e.g. |
| outputObjectId | integer outputObject to attach this sqlScript to. |
connectionIdString (string) or connectionIdBigQuery (string) | |
Array of objects (runParameterSqlScriptInfo) Optional Parameters that can be used to parameterized the |
{- "sqlScript": "string",
- "type": "string",
- "vendor": "string",
- "outputObjectId": "21",
- "connectionId": "55",
- "runParameters": [
- {
- "type": "sql",
- "overrideKey": "myVar",
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}
}
}
]
}{- "sqlScript": "string",
- "type": "string",
- "vendor": "string",
- "outputObjectId": "21",
- "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}List existing sql scripts
ref: listSqlScripts
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "data": [
- {
- "sqlScript": "string",
- "type": "string",
- "vendor": "string",
- "outputObjectId": "21",
- "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}
]
}Count existing sql scripts
ref: countSqlScripts
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "count": 1
}Get an existing sql script
ref: getSqlScript
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "sqlScript": "string",
- "type": "string",
- "vendor": "string",
- "outputObjectId": "21",
- "connection": {
- "id": "55"
}, - "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}
}Patch an existing sql script
ref: patchSqlScript
| id required | integer |
| sqlScript | string String of SQL queries to be executed. |
| type | string Identifier to decide if the SQLs will be executed before or after a job. |
| vendor | string e.g. |
connectionIdString (string) or connectionIdBigQuery (string) |
{- "sqlScript": "string",
- "type": "string",
- "vendor": "string",
- "connectionId": "55"
}{- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}Delete an existing sql script
ref: deleteSqlScript
| id required | integer |
A self-contained, configurable space shared by several users,
containing flows, Datasets, connections, and other Trifacta Wrangler objects.
Delete Workspace configuration settings override (reset the settings to their initial values).
| settings required | Array of strings |
{- "settings": [
- "feature.myFeature"
]
}{- "numberOfRowsDeleted": 1
}Delete Workspace configuration settings override (reset the settings to their initial values).
| id required | integer |
| settings required | Array of strings |
{- "settings": [
- "feature.myFeature"
]
}{- "numberOfRowsDeleted": 1
}Get workspace configuration. Settings set to null use the default configuration from the getConfigurationSchema endpoint.
It is possible to filter the configuration to a specific key using the query parameter key:
/v4/workspaces/:id/configuration?key=outputFormats.JSON
[{ "key": "outputFormats.JSON", "value": true }]
| key | string |
[- {
- "key": "feature.feature1",
- "value": 42
}, - {
- "key": "feature.anotherFeature.usingDefaultValue",
- "value": null
}
]Update the workspace configuration for the specified keys. To reset a configuration value to its default, use the delete endpoint.
Use the getConfigurationSchema endpoint to get the list of editable configuration values.
required | Array of objects (configurationKeyValueSchema) |
{- "configuration": [
- {
- "key": "feature.feature1",
- "value": false
}, - {
- "key": "feature.feature2",
- "value": "some value"
}
]
}[- true
]Get workspace configuration. Settings set to null use the default configuration from the getConfigurationSchema endpoint.
It is possible to filter the configuration to a specific key using the query parameter key:
/v4/workspaces/:id/configuration?key=outputFormats.JSON
[{ "key": "outputFormats.JSON", "value": true }]
| id required | integer |
| key | string |
[- {
- "key": "feature.feature1",
- "value": 42
}, - {
- "key": "feature.anotherFeature.usingDefaultValue",
- "value": null
}
]Update the workspace configuration for the specified keys. To reset a configuration value to its default, use the delete endpoint.
Use the getConfigurationSchema endpoint to get the list of editable configuration values.
| id required | integer |
required | Array of objects (configurationKeyValueSchema) |
{- "configuration": [
- {
- "key": "feature.feature1",
- "value": false
}, - {
- "key": "feature.feature2",
- "value": "some value"
}
]
}[- true
]Get configuration schema for the specified workspace.
| id required | integer |
{- "property1": {
- "type": "string",
- "default": null,
- "allowedValues": [
- "string"
]
}, - "property2": {
- "type": "string",
- "default": null,
- "allowedValues": [
- "string"
]
}
}Get configuration schema for the current workspace.
{- "property1": {
- "type": "string",
- "default": null,
- "allowedValues": [
- "string"
]
}, - "property2": {
- "type": "string",
- "default": null,
- "allowedValues": [
- "string"
]
}
}Transfer Trifacta Wrangler assets to another user in the current workspace. For the given workspace, assigns ownership of all the user's contents to another user. This includes flows, datasets, recipes, and connections–basically any object that can be created and managed through the Trifacta Wrangler UI.
ℹ️ NOTE: This API endpoint does not delete the original user account. To delete the user account, another API call is needed.
ℹ️ NOTE: The asset transfer endpoint cannot be applied to deleted users. You must transfer the assets first before deleting the user.
| fromPersonId required | integer the id of the person to transfer assets from |
| toPersonId required | integer the id of the person to transfer assets to |
{- "fromPersonId": 1,
- "toPersonId": 1
}Represents the data produced by running a recipe on some input.
ℹ️ NOTE: In the Trifacta Wrangler application UI, the WrangledDataset object is called a recipe.
Create a new wrangled dataset
required | object |
required | object |
| name required | string |
| inferredScript | object |
{- "importedDataset": {
- "id": 1
}, - "inferredScript": { },
- "flow": {
- "id": 1
}, - "name": "string"
}{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "referenceinfo": {
- "id": 1,
- "name": "string",
- "description": "string"
}, - "wrangled": true
}List existing wrangled datasets
ref: listWrangledDatasets
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "data": [
- {
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "referenceinfo": {
- "id": 1,
- "name": "string",
- "description": "string"
}, - "wrangled": true
}
]
}Add this wrangled dataset to a flow as a reference.
| id required | integer |
required | object The flow to add this dataset to. |
{- "flow": {
- "id": 1
}
}{- "flow": {
- "id": 1
}, - "referencedFlowNode": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "wrangled": true
}Count existing wrangled datasets
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "count": 1
}Get an existing wrangled dataset
ref: getWrangledDataset
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "referenceinfo": {
- "id": 1,
- "name": "string",
- "description": "string"
}, - "wrangled": true
}Update a wrangled dataset. This can mean one of two things.Either this will update the flownode object in our database or the editable script object.
ref: patchWrangledDataset
| id required | integer |
| activesampleId | integer Internal identifier of the currently active |
| referenceId | integer Internal identifier for referenceInfo, which contains the name and description of the reference object associated with this flow node. This is how the reference dataset will appear when used in other flows. |
{- "activesampleId": 1,
- "referenceId": 1
}{- "id": 1,
- "updater": {
- "id": 1
}, - "updatedAt": "2019-08-24T14:15:22Z"
}Get the dataset that is the primary input for this wrangled dataset. This can be either an imported dataset or a wrangled dataset.
ref: getInputDataset
| id required | integer |
{- "wrangledDataset": {
- "id": 1
}
}This action performs a dataset swap for the source of a wrangled dataset, which can be done through the UI.
Update the primary input dataset for the specified wrangled dataset. Each wrangled dataset must have one and only one primary input dataset, which can be an imported or wrangled dataset. If a wrangled dataset from another flow is selected, a reference will be used.
✅ TIP: After you have created a job via API, you can use this API to swap out the source data for the job's dataset. In this manner, you can rapidly re-execute a pre-existing job using fresh data.
ref: updateInputDataset
| id required | integer |
required | object |
{- "wrangledDataset": {
- "id": 1
}
}{- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "flow": {
- "id": 1
}, - "recipe": {
- "id": 1
}, - "activeSample": {
- "id": 1
}, - "referenceinfo": {
- "id": 1,
- "name": "string",
- "description": "string"
}, - "wrangled": true
}A writeSetting object defines file-based outputs within an outputObject. Settings include path, format, compression, and delimiters.
To specify multiple outputs, you can include additional writeSetting objects in the request.
For example, if you want to generate output to csv and json, you can duplicate the writeSettings object for csv and change the format value in the second one to json.
Create a new writesetting
ref: createWriteSetting
| path required | string The fully qualified path to the output location where to write the results. |
| action required | string Enum: "create" "append" "overwrite" If the output file or directory exists, you can specify one of the following actions
|
| format required | string Enum: "csv" "json" "avro" "pqt" "hyper" Output format for the results. Specify one of the supported values.
|
| compression | string Enum: "none" "gzip" "bzip2" "snappy" For csv and json results,
you can optionally compress them using
|
| header | boolean For csv results with action set to |
| asSingleFile | boolean For |
| delim | string The delimiter between field values in an output row. Only relevant if the chosen |
| hasQuotes | boolean If true, each field in the output is wrapped in double-quotes. |
| outputObjectId | integer outputObject to attach this writeSetting to. |
Array of objects (runParameterDestinationInfo) Optional Parameters that can be used to parameterized the path | |
| connectionId | string Internal identifier of the connection to use when writing to a SFTP destination. |
{- "path": "/path/to/file.csv",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "outputObjectId": 7,
- "runParameters": [
- {
- "insertionIndices": [
- {
- "index": 1,
- "order": 1
}
], - "value": {
- "variable": {
- "value": "string"
}, - "overrideKey": "myVar"
}
}
], - "connectionId": "5"
}{- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "connectionId": "25"
}List existing write settings
ref: listWriteSettings
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "data": [
- {
- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "connectionId": "25"
}
]
}Count existing write settings
ref: countWriteSettings
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
| limit | integer Default: 25 Maximum number of objects to fetch. |
| offset | integer Offset after which to start returning objects. For use with |
| filterType | string Default: "fuzzy" Defined the filter type, one of ["fuzzy", "contains", "exact"]. For use with |
| sort | string Example: sort=-createdAt Defines sort order for returned objects |
| filterFields | string Default: "name" Example: filterFields=id,order comma-separated list of fields to match the |
| filter | string Example: filter=my-object Value for filtering objects. See |
{- "count": 1
}Get an existing write setting
ref: getWriteSetting
| id required | integer |
| fields | string Example: fields=id;name;description Semi-colons-separated list of fields |
| embed | string Example: embed=association.otherAssociation,anotherAssociation Comma-separated list of objects to pull in as part of the response. See Embedding Resources for more information. |
| includeDeleted | string If set to "true", will include deleted objects. |
{- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "id": 1,
- "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z",
- "creator": {
- "id": 1
}, - "updater": {
- "id": 1
}, - "connectionId": "25"
}Patch an existing write setting
ref: patchWriteSetting
| id required | integer |
| path | string The fully qualified path to the output location where to write the results. |
| action | string Enum: "create" "append" "overwrite" If the output file or directory exists, you can specify one of the following actions
|
| format | string Enum: "csv" "json" "avro" "pqt" "hyper" Output format for the results. Specify one of the supported values.
|
| compression | string Enum: "none" "gzip" "bzip2" "snappy" For csv and json results,
you can optionally compress them using
|
| header | boolean For csv results with action set to |
| asSingleFile | boolean For |
| delim | string The delimiter between field values in an output row. Only relevant if the chosen |
| hasQuotes | boolean If true, each field in the output is wrapped in double-quotes. |
| connectionId | string Internal identifier of the connection to use when writing to a SFTP destination. |
{- "path": "string",
- "action": "create",
- "format": "csv",
- "compression": "none",
- "header": true,
- "asSingleFile": true,
- "delim": ",",
- "hasQuotes": true,
- "connectionId": "25"
}{- "id": 1,
- "updater": {
- "id": 1
}, - "createdAt": "2019-08-24T14:15:22Z",
- "updatedAt": "2019-08-24T14:15:22Z"
}Delete an existing write setting
ref: deleteWriteSetting
| id required | integer |