To enable programmatic control over its objects, the Cloud Dataprep Platform supports a range of REST API endpoints across its objects. This section provides an overview of the API design, methods, and supported use cases.

Most of the endpoints accept JSON as input and return JSON responses. This means that you must usually add the following hearders to your request:

Content-type: application/json
Accept: application/json

ℹ️ NOTE: Access to APIs must be enabled on a per-project basis. For more information, see enable Api

The term resource refers to a single type of object in the Cloud Dataprep Platform metadata. An API is broken up by its endpoint's corresponding resource. The name of a resource is typically plural, and expressed in camelCase. Example: jobGroups.

Resource names are used as part of endpoint URLs, as well as in API parameters and responses.

The platform supports Create, Read, Update, and Delete operations on most resources. You can review the standards for these operations and their standard parameters below.

Some endpoints have special behavior as exceptions.

Create

To create a resource, you typically submit an HTTP POST request with the resource's required metadata in the request body. The response returns a 201 Created response code upon success with the resource's metadata, including its internal id, in the response body.

Read

An HTTP GET request can be used to read a resource or to list a number of resources.

A resource's id can be submitted in the request parameters to read a specific resource. The response usually returns a 200 OK response code upon success, with the resource's metadata in the response body.

If a GET request does not include a specific resource id, it is treated as a list request. The response usually returns a 200 OK response code upon success, with an object containing a list of resources' metadata in the response body.

When reading resources, some common query parameters are usually available. e.g.:

/v4/jobGroups?limit=100&includeDeleted=true&embed=jobs

Update

Updating a resource requires the resource id, and is typically done using an HTTP PUT or PATCH request, with the fields to modify in the request body. The response usually returns a 200 OK response code upon success, with minimal information about the modified resource in the response body.

Delete

Deleting a resource requires the resource id and is typically executing via an HTTP DELETE request. The response usually returns a 204 No Content response code upon success.

• Resource names are plural and expressed in camelCase.

• Resource names are consistent between main URL and URL parameter.

• Parameter lists are consistently enveloped in the following manner:

  { "data": [{ ... }] }

• Field names are in camelCase and are consistent with the resource name in the URL or with the embed URL parameter.

  "creator": { "id": 1 },
  "updater": { "id": 2 },

When reading a resource, the platform supports an embed query parameter for most resources, which allows the caller to ask for associated resources in the response. Use of this parameter requires knowledge of how different resources are related to each other and is suggested for advanced users only.

In the following example, the sub-jobs of a jobGroup are embedded in the response for jobGroup=1:

https://api.clouddataprep.com/v4/jobGroups/1?embed=jobs

If you provide an invalid embedding, you will get an error message. The response will contain the list of possible resources that can be embedded. e.g.

https://api.clouddataprep.com/v4/jobGroups/1?embed=*

Example error:

{
  "exception": {
    "name": "ValidationFailed",
    "message": "Input validation failed",
    "details": "No association * in flows! Valid associations are creator, updater, snapshots..."
  }
}

Fields

It is possible to let the application know that you need fewer data to improve the performance of the endpoints using the fields query parameter. e.g.

https://api.clouddataprep.com/v4/flows?fields=id;name

The list of fields need to be separated by semi-colons ;. Note that the application might sometimes return more fields than requested.

You can also use it while embedding resources.

https://api.clouddataprep.com/v4/flows?fields=id;name&embed=flownodes(fields=id)

Limit and sorting

You can limit and sort the number of embedded resources for some associations. e.g.

https://api.clouddataprep.com/v4/flows?fields=id&embed=flownodes(limit=1,fields=id,sort=-id)

Note that not all association support this. An error is returned when it is not possible to limit the number of embedded results.

The Cloud Dataprep Platform uses HTTP response codes to indicate the success or failure of an API request.

• Codes in the 2xx range indicate success.
• Codes in the 4xx range indicate that the information provided is invalid (invalid parameters, missing permissions, etc.)
• Codes in the 5xx range indicate an error on the servers. These are rare and should usually go away when retrying. If you experience a lot of 5xx errors, contact support.

Each request has a request identifier, which can be found in the response headers, in the following form:

x-trifacta-request-id: <myRequestId>

ℹ️ NOTE: If you have an issue with a specific request, please include the x-trifacta-request-id value when you contact support

• API versioning is not synchronized to specific releases of the platform.
• APIs are designed to be backward compatible.
• Any changes to the API will first go through a deprecation phase.

You can use a third party client, such as curl, HTTPie, Postman or the Insomnia rest client to test the Cloud Dataprep API.

⚠️ When testing the API, bear in mind that you are working with your live production data, not sample data or test data.

Note that you will need to pass an API token with each request.

For e.g., here is how to run a job with curl:

curl -X POST 'https://api.clouddataprep.com/v4/jobGroups' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <token>' \
-d '{ "wrangledDataset": { "id": "<recipe-id>" } }'

Using a graphical tool such as Postman or Insomnia, it is possible to import the API specifications directly:

1. Download the API specification by clicking the Download button at top of this document
2. Import the JSON specification in the graphical tool of your choice.
   • In Postman, you can click the import button at the top
   • With Insomnia, you can just drag-and-drop the file on the UI

Note that with Postman, you can also generate code snippets by selecting a request and clicking on the Code button.

An object representing data loaded into Cloud Dataprep, as well as any structuring that has been applied to it. imported datasets are the starting point for wrangling, and can be used in multiple flows.

An internal object encoding the information necessary to run a part of a Cloud Dataprep jobGroup.

This is called a "Stage" on the Job Results page in the UI.