As mentioned in our API references, you can use our Storyboard API to generate video preview and further use Video Render API to generate Video in mp4 format.

Any Video in Pictory consists of the following components:

  1. Video Scenes (or Visuals): Video is divided into multiple Video Blocks called Scenes. These scenes can be provided in the following ways:
    1. You can choose to provide your video blocks (as URI). OR,
    2. You can provide input text to generate visuals for each scene. The input text is used as a subtitle of the videos as well.
  2. Audio: Video can consist of background music and voice-over.

1. Text to Video Generation

Pictory's Text-to-Video APIs offer a dynamic way to transform the text into engaging videos.

These APIs allow for the creation and editing of videos based on various text inputs. For instance, if you have a blog or an article, you can easily convert it into a captivating video by simply providing the webpage URL. Text-to-Video APIs open up various possibilities, including:

  1. Generating new videos directly from existing text content.
  2. Converting your published blogs or articles into videos using their URLs.

Steps to generate Video from Exiting Text

Pictory streamlines the video creation process by offering a preview of the video before finally generating your video. This feature allows users to review and customize any sections of the final video they wish to change. Here's a simple guide on how to generate a video preview from your content:

  1. Initiate a Video Preview Request: Begin by calling the Storyboard API. This action will generate a job_id in the response, which is essential for tracking your video's progress. You can opt to specify a callback URL using the webhook parameter in the Storyboard API. Once the Storyboard Job is completed, the response will be sent to the specified webhook URL in the request body.
  2. Monitor Job Status and Preview: In order to monitor the Job status you can use the job_id with the GET Job API to monitor the status of your request. Once the job is complete, you will receive a URL for the video preview along with renderParams in the response:
    1. Users can preview the video using the URL provided in preview parameter.
    2. renderParams in the response body detailed settings like the URL for individual scenes and subtitle text for each scene in the video.
  3. Customize Your Scenes: If scenes don’t meet your expectations, you can easily change them. Simply modify the scene url, animation, text etc in the renderParams scene object. This level of customization ensures that every aspect of your video can be tailored to your specific needs before generating the final video.
  4. Initiate Video Generation Request: Once you are happy with the video preview, proceed to the next step by calling the Video Render API. This action will initiate the final video generation process. Keep in mind that generating a video can take some time. Similar to the preview process (Step 1), this API will provide you with a job_id for tracking the progress of your video rendering. Additionally You can opt to specify a callback URL using the webhook parameter in the Render API. Once the Video Rendering Job is completed, the response will be sent to the specified webhook URL in the request body.
  5. Monitor Final Video Status: You can also use the job_id with the GET Job API to monitor the status of your video rendering request. When the rendering process is complete, you will receive the final video in the form of a URL.

Steps to generate video are mentioned in our Text to Video flow inside our Postman collection at Getting Started -> API Flows -> Text to Video Flow

2. Add Transcription/Subtitles to Existing Video

Transcription API of Pictory lets you generate the text from the existing video. Once the transcription(text) has been generated you can use our steps mentioned in this to add transcription to your existing videos.

Steps to add Transcription for your existing video require the following steps:

  1. Upload your Existing Video on the Pictory server. To upload a video:

    1. First, obtain the uploaded video URL using generate URL API. This API gives the upload URL in response (data.url).
    2. Upload the video on the Pictory server using url obtained from the previous step.
  2. Once the file has been uploaded, you can Generate the Transcription of the uploaded Video: This can be done by calling /v2/transcription API.

    1. Transcription API converts speech to text in an array of word formats (instead of generating sentences). Each word object consists of the start_time and end_time of the word that needs to be displayed in the videos.
    2. You would need to write logic to form sentences from words and include the start time of the sentence ( the start time of 1st word in the sentence) and the end time of the sentence (the end time of the last word). The sample is given below:
//Sample transcription API response for Speech "I'm Tom Cruise. "
[{
	"uid": "07092d7a-27cc-452e-a484-f4c6c2f7ceb6",
	"word": "I'm",
	"start_time": 0,
	"end_time": 0.32,
	"speakerId": 1,
	"sentence_index": 0
}, {
	"uid": "d5d28123-b27c-4a54-ac56-f986ae1a8038",
	"word": "Tom",
	"start_time": 0.32,
	"end_time": 0.79999995,
	"speakerId": 1,
	"sentence_index": 0
}, {
	"uid": "9a621347-0d82-4a9f-a7c7-8832a2858da5",
	"word": "Cruise.",
	"start_time": 0.79999995,
	"end_time": 1.12,
	"speakerId": 1,
	"sentence_index": 0
} ]

Sentence: I'm Tom Cruise.
Sentence start time :  0
Sentence end_time: 1.12
  1. Once the sentences are created you can call Storyboard API to add subtitles to the original video. To add subtitles to the scene you need to:
    1. Divide the original video into different scenes.
    2. Add subtitles to each scene.

Storyboard API has a scenes array and for each scene, you need to pass the following parameters:

  1. backgroundUri: original Video URL (AWS S3)
  2. text: this will be the subtitle that needs to be displayed for that particular scene
  3. BackgroundVideoSegment.start: This defines the start time of text in the video. Pass the start time of the sentence here
  4. BackgroundVideoSegment.end: This defines the start time of text in the video. Pass the end time of the sentence here.
  1. Storyboard API returns Job_Id in response. The job status can be seen by calling GET Job REST API. Once this job is complete, it gives a video preview URL and render_video settings in response. Video Render settings returned in this job can be used to generate video in .mp4 format.

  2. Video can be generated by calling Video Render API. Video Render API returns JobId in response. The job status can be seen by calling GET Job REST API. Once this job is complete, it gives the link to the .mp4 video in response.

3. Summarise Video

Video Summary/Highlights API creates a summary of a long video. For example, if you have a 30-minute-long video, you can use our AI to summarize that in a shorter duration (let's say 2 mins, 5 min) video.

Watch the Video on how to generate video from highlights.

To summarise the video, please follow the below-mentioned steps:

  1. Obtain the uploaded video URL using generate URL API. This API gives the upload URL in response (data.url).
  2. Upload the video on the URL obtained from the previous step.
  3. Generate the Transcription of the uploaded Video by passing the URL (on which the video file was uploaded in the request param by using /v2/transcription API. Transcription data will be sent in a callback URL or it can be fetched from GET Job API.
  4. Highlights for the video can be obtained by passing transcription data from step 3 in highlights API . This API requires you to pass the duration for which highlights are needed.

Request Body of highlights API:

{
  "transcript": "{{transcript_settings}}", //obtained from GET transcription API 
  "highlight_duration": 30,
  "webhook": "{{webhookurl}}"
}

As soon as highlights are ready webhook URL with the following body params is called:

//Sample response
{
    "job_id": "b292f6ca-fbf2-4380-93e1-95d26b5a563e",
    "success": true,
    "data": {
        "transcript": [
            {
                "uid": "6e12aac1-0dab-4229-90aa-238c62846e4c",
                "words": [
                    {
                        "uid": "d9f11114-5165-42d7-af5f-3b62e53613dc",
                        "word": "",
                        "start_time": 0,
                        "end_time": 0.5,
                        "sentence_index": 0,
                        "is_pause": true,
                        "pause_size": "small",
                        "state": "active"
                    },
                    {
                        "uid": "279fbfb5-5af4-4022-b694-65253056323e",
                        "word": "",
                        "start_time": 0.5,
                        "end_time": 0.91,
                        "sentence_index": 0,
                        "is_pause": true,
                        "pause_size": "small",
                        "state": "active"
                    },
                    {
                        "uid": "4039e0b1-a830-45d6-bf69-5818a7277779",
                        "word": "Once",
                        "start_time": 0.91,
                        "end_time": 1.24,
                        "sentence_index": 0,
                        "highlight": true
                    },
          }]
      }
 }
              

As of now, Summary API converts speech to text in an array of word formats (instead of generating sentences). Each word object consists of the start_time and end_time of the word that needs to be displayed in the videos.

You would need to write logic to form sentences from words and include the start time of the sentence (the start time of 1st word in the sentence) and the end time of the sentence (the end time of the last word of the sentence). A sample of the response is given above.

  1. Once sentences are created, you can call Storyboard API to add subtitles to the original video. To add subtitles to the scene you need to:
  1. Divide the original video into different scenes.
  2. Add subtitles to each scene.
    Storyboard API has a scenes array and for each scene, you need to pass the following parameters:
    1. backgroundUri: original Video URL (AWS S3)
    2. text: this should be the subtitle of the particular scene.
    3. BackgroundVideoSegment.start: This defines the start time of text in the video. Pass the start time of the sentence in this variable,
    4. BackgroundVideoSegment.end: This defines the start time of text in the video. Pass the end time of the sentence in this variable.
  1. Storyboard API returns Job_Id in response. The job status can be seen by calling GET Job REST API. Once this job is completed, it gives a video preview URL and render_settings in response. Video Render settings returned in this job can be used to generate video in .mp4 format.
  2. Video can be generated by calling Video Render API. Video Render API returns JobId in response. The sjob status can be seen by calling GET Job REST API. Once this job is completed, it gives the link to the video in .mp4 format in response.