Overview

The Lipdub API allows you to dub a source video with any audio and provides you with an output video with lip movements synchronized to the new audio.

1

Provide a source audio and video

  • Provide a source audio: currently WAV, MP3, M4A and AAC formats are supported.
  • Provide a source video to merge with the source audio: currently non-HDR MP4 and MOV videos are supported.
2

Lipdub your video

  • Use the /submit endpoint to generate your Lipdub video with lip movements that correspond to the new audio.
The Lipdub API is only available for customers who have signed an Enterprise contract. If you would like to use the Lipdub API, please contact support.

Requirements to using the API

To use the Captions API, you must:

  1. Accept the Captions API Terms
  2. Have enough credits to complete your request

Usage, credits, and limitations

  • The Lipdub API consumes 1 credit per second.
  • For example, generating a 30-second video would consume 30 credits.
  • You can view remaining credits and purchase additional credits on the API Dashboard.
  • The maximum length of a video you can generate is 1 minute, so it is important that the provided audio to be Lipdubbed conforms to this requirement.

Instructions

/submit

Begin the Lipdub video generation process.

Request

audioUrl - required - Link to the audio to be dubbed into the source video. Please note that your audio can be hosted anywhere, but must be a publicly available direct link. If you would like to upload a Google Drive link, please read these instructions.

videoUrl - required - Link to the video to be dubbed. Note that the video must contain exactly one face in it, which will be used as the reference to synchronize lip movements.

Bash script
curl -X POST https://api.captions.ai/api/lipdub/submit -H "Content-Type: application/json" -H "x-api-key: <api-key>" -d '{"audioUrl": "<audio-url>", "videoUrl": "<video-url>"}'

Response Types

/poll

Check the status of a Lipdub video generation.

Request

We recommend polling first after 30 seconds, and every 5 seconds thereafter.

Bash script
curl -X POST https://api.captions.ai/api/lipdub/poll -H "x-api-key: <api-key>" -H "x-operation-id: <operation-id>"

Response Types