Twelve Labs Video Understanding API
Classify Video
Use this method to classify a set of videos based on the entities or actions shown in those videos. Note that this endpoint returns the first page or results. To retrieve the rest of the pages, you must call the [`GET`](/reference/classify-retrieve-specific-page) method of the `/classify/{page_token}` endpoint passing it the token that identifies the page you want to retrieve.
**NOTES**:
- This method has a rate limit of 1 call per second for each API key. If the limit is reached, the platform returns an `HTTP 429 - Too many requests` error response.
- The total number of the prompts you specify in a request must not exceed 100.
- The total duration of the videos you classify in a request must not exceed 10 hours.
Classify Bulk
Use this method to classify all the videos in the specified index based on the entities or actions shown in those videos. Note that this endpoint returns the first page or results. To retrieve the rest of the pages, you must call the [`GET`](/reference/classify-retrieve-specific-page) method of the `/classify/{page_token}` endpoint passing it the token that identifies the page you want to retrieve.
**NOTES**:
- This method has a rate limit of 1 call per second for each API key. If the limit is reached, the platform returns an `HTTP 429 - Too many requests` error response.
- The total number of prompts you specify in a request must not exceed 100.
Classify Retrieve Specific Page
The `classify` and `/classify/bulk` endpoints return the first page or results. You can use this endpoint to retrieve the rest of the pages.
**NOTE**: This method has a rate limit of 1 call per second for each API key. If the limit is reached, the platform returns an `HTTP 429 - Too many requests` error response.
Create Text Enbedding
This endpoint creates text embeddings.
Create Video Embeding Task
This method creates a new video embedding task that uploads a video to the platform and creates one or multiple embeddings.
Note the following about using this endpoint:
- To upload a file that resides on the local file system, pass the `video_file` parameter. It must be a stream of bytes encoded as `multipart/form-data`.
- To upload a file from a publicly accessible URL, pass the `video_url` parameter with the URL of your video. It must be a string.
- You must specify at least one of the `video_file` and `video_url` parameters. If you specify both the `video_url` and `video_file` parameters in the same request, the `video_url` parameter takes precedence over `video_file`.
The videos for which you wish to generate embeddings must meet the following requirements:
- **Duration**: Must be between 4 seconds and 2 hours (7,200s).
- **File size**: Must not exceed 2 GB.
- **Video resolution**: Must be greater or equal than 360p and less or equal than 4K. For consistent results, Twelve Labs recommends you upload 360p videos.
- **Video and audio formats**: The video files must be encoded in the video and audio formats listed on the [FFmpeg Formats Documentation](https://ffmpeg.org/ffmpeg-formats.html) page. For videos in other formats, contact us at [support@twelvelabs.io](mailto:support@twelvelabs.io).
Retrieve Video Embeding
This method retrieves embeddings for a specific video embedding task. Ensure the task status is `ready` before invoking this method. Refer to the [Retrieve a video embedding](/reference/retrieve-video-embeding-task) task page for instructions on checking the task status.
Retrieve Video Embeding Task
This method retrieves the status of a video embedding task.
Each video embedding task represents a request to upload and create one or multiple embeddings. A task starts when you begin uploading a video by calling the `POST` method of the `/embed/task` endpoint, and can have one of the following statuses:
- `processing`: This status indicates that the embeddings are being created.
- `ready`: This status indicates that the platform has finished processing your video, and you can retrieve the embeddings by invoking the [`GET`](/reference/retrieve-video-embeding) method on the `/embed/tasks/{task_id} endpoint`.
- `failed`: This status indicates that the task could not be completed as expected, and the embeddings haven't been created.
List Engines
This method lists all the engines that are available in your account.
Retrieve Engine
This method retrieves an engine.
Generate Text Representation
This endpoint generates open-ended texts based on your videos, including but not limited to tables of content, action items, memos, and detailed analyses.
Generate Gist
This endpoint generates concise overviews in the form of titles, topics, and hashtags representing the key elements or themes of a video.
List Indexes
This method returns a list of the indexes in your account. The API returns indexes sorted by creation date, with the oldest indexes at the top of the list.
Create Index
This method creates an index.
Retrieve Index
This method retrieves details about the specified index.
Update Index
This method updates the name of the specified index.
Delete Index
This method deletes the specified index and all the videos within it. This action cannot be undone.
List Videos
This method returns a list of the videos in the specified index. By default, the API returns your videos sorted by creation date, with the newest at the top of the list.
Retrieve Video Information
This method retrieves information about the specified video.
Update Video Information
Use this method to update the title and the metadata of a video.
Delete Video Information
This method deletes all the information about the specified video This action cannot be undone.
Identify Logos
Use this method to identify the presence of brand logos within your videos.
Retrieve OCR
This method retrieves text recognized in video (OCR).
Retrieve Transcription
This method retrieves a transcription of the spoken words.
Retrieve Thumbnail
To use this feature, you must enable thumbnail generation for the index to which the video has been uploaded. For details, see the [Create an index](/reference/create-index) section.
Make Search Request
Use this method to search for relevant matches in an index.
This method returns the first page of results. To retrieve the subsequent pages, you must call the [`GET`](/reference/search-retrieve-specific-page) method of the `/search/{page-token}` endpoint, passing it the unique identifier of the page you want to retrieve.
**NOTES**:
- This method has a rate limit of 1 call per second for each API key. If the limit is reached, the platform returns an `HTTP 429 - Too many requests` error response.
- When you use pagination, you will not be charged for retrieving subsequent pages of results.
Combined Queries Make Search Request
Use this method to search for relevant matches in an index using combined queries.
For clarity, the combined search feature is documented in this section. However, note that the `/search` endpoint supports all the features of the `/combined-search` endpoint.
This endpoint supports pagination and filtering.
This method returns the first page of results. To retrieve the subsequent pages, you must call the [`GET`](/reference/combined-queries-retrieve-specific-page) method of the `/combined-search/{page-token}` endpoint, passing it the unique identifier of the page you want to retrieve.
**NOTES**:
- Twelve Labs Inc. has disabled calling this method from this page to avoid confusion and recommends you invoke it using one of the available [SKDs](/docs/twelve-labs-sdks). See the [Use combined queries](/docs/search-combined-queries) page for details.
- This method has a rate limit of 1 call per second for each API key. If the limit is reached, the platform returns an `HTTP 429 - Too many requests` error response.
- When you use pagination, you will not be charged for retrieving subsequent pages of results.
Combined Queries Retrieve Specific Page
Use this endpoint to retrieve a specific page of search results.
**NOTE**: When you use pagination, you will not be charged for retrieving subsequent pages of results.
Search Retrieve Specific Page
Use this endpoint to retrieve a specific page of search results.
**NOTE**: When you use pagination, you will not be charged for retrieving subsequent pages of results.
Summarize
This endpoint generates summaries, chapters, or highlights for your videos. Optionally, you can provide a prompt to customize the output.
List Video Indexing Tasks
This method returns a list of the video indexing tasks in your account. The API returns your video indexing tasks sorted by creation date, with the newest at the top of the list.
Create Video Indexing Task
Use this method to create a new video indexing task that uploads a video to the platform and makes it searchable. Note the following about creating a new video indexing task:
- To upload a file that resides on the local file system, pass the `video_file` parameter. It must be a stream of bytes encoded as `multipart/form-data`.
- To upload a file from a publicly accessible URL, pass the `video_url` parameter with the URL of your video. It must be a `string`.
- You must specify at least one of the `video_file` and `video_url` parameters. If you specify both the `video_url` and `video_file` parameters in the same request, the `video_url` parameter takes precedence over `video_file`.
- To provide a transcription file, you must set the `provide_transcription` parameter to `true` and specify at least one of the `transcription_file` and `transcription_url` parameters:
- Use the `transcription_file` parameter to provide a transcription from the local file system. It must be a stream of bytes encoded as `multipart/form-data`.
- Use the `transcription_url` parameter to provide a transcription file from a publicly accessible URL. It must be a `string`.
Upload External Provider
Use this endpoint to upload videos from external providers. The platform currently supports only YouTube as an external provider, but we will add support for additional providers in the future.
Video Indexing Tasks by Status
This method retrieves the number video indexing tasks in each status for the specified index.
Cloud to Cloud Transfer
This method makes a cloud-to-cloud transfer.
Retrieve Video Indexing Task
This method retrieves a video indexing task.
Delete Video Indexing Task
This action cannot be undone.
Note the following about deleting a video indexing task:
- You can only delete video indexing tasks for which the status is `ready` or `failed`.
- If the status of your video indexing task is `ready`, you must first delete the video vector associated with your video indexing task by calling the [`DELETE`](/reference/delete-video-information) method of the `/indexes/videos` endpoint.
