Editor-SDK

Editor-SDK is the SDK for the editor part, which can be directly imported into the project to edit and export MP4 files. Editor is developed based on Core-SDK.

github demo

Usage:

// Import the video editor SDK
import { VideoEditorSDK } from "@h5/videoEditorSDK";

// Create an instance of the editor
const vs = new VideoEditorSDK({
  registerId: "test123", // Required, registration ID, need to apply on the official website
  movieData: "Project data", // Optional, either this or appid must be passed. Will prioritize reading project data. If no project data is passed, it will get project data from api server through appid
  EModuleEffectSourcePath: '/assets/effectcanvas', // Effect resource module loading path
  sides: null, // Optional, sidebar configuration, defaults to null, uses system default sidebar
  token: "User token", // Optional, user token for calling api server interfaces
  userInfo: null, // Optional, user information for calling api server interfaces
  resourceHost: "https://cdn.h5ds.com", // Host for resource loading
  appid: "Work ID", // Optional, either this or movieData must be passed
  target: document.body, // Optional, container, default is document.body
  server: "API server configuration", // Optional, if not passed, it will call the API service of video.h5ds.com, API interface configuration, please refer to APIServer
});

// Initialize
const vsdk = await vs.init();

/**
 * Assuming the project data is movieData, modify data -> update view
 */
// 1. Get the first element in the project data
const [element] = movieData.elements;

// 2. Modify the coordinates of the first element to [100, 200]
element.style.x = 100;
element.style.y = 200;

// 3. Update the view
vsdk.editor.updateMovie();

You can get the vs instance through new VideoEditorSDK(...), and get vsdk through the instance vs.init(). The instance vsdk.editor contains many properties and methods, which are introduced below one by one along with usage tips.

// Update view
vsdk.editor.updateMovie();

// Update settings area
vsdk.editor.updateOption();

// Update timeline
vsdk.editor.updateTimeline();

Video Synthesis

import { exportMovie } from "@h5/videoEditorSDK";

interface ExportMovieOptions {
  name?: string; // Export file name
  format: "mp4" | "gif" | "mp3"; // Export format
  fps: number; // Export video frame rate
  resolution: "480P" | "720P" | "1080P" | "2K" | "4K"; // Export video resolution
  data: MovieData; // Export video data
  onBefore: (obj: any) => Promise<any>; // Callback function before export
  onFinish: (obj: any) => Promise<any>; // Callback function after export
  onProgress: (obj: any) => void; // Export progress callback function
  gifTimes?: number[]; // Export gif time range
  gifWidth?: number; // Export gif width
  gifFps?: number; // Export gif frame rate
  gifSpeed?: number; // Export gif playback speed
}

const params: ExportMovieOptions = {
  name: "Export video",
  format: "mp4",
  fps: 30,
  resolution: "720P",
  data: movieData,
  onBefore: async (obj: any) => {
    console.log("onBefore", obj);
  },
  onFinish: async (obj: any) => {
    console.log("Video synthesis completed:", obj.url);
  },
  onProgress: (obj: any) => {
    console.log("onProgress", obj);
  },
};

// Execute export operation, it is recommended to execute the synthesis task in a window.open window to avoid blocking the main thread and support iframe calls
exportMovie(params);

Property Description

The content after the colon indicates the property's data type. @ indicates @observable data monitored by mobx, direct modification can trigger view update.

.movie: Store

Instance of the core video-core, type is the instance of video-core, please refer to the instance description of the movie instance of video-core.

.userInfo: UserInfo

Optional, user information for calling api server interfaces.

export interface UserInfo {
  id: string;
  name: string;
  avatar: string;
  workspaceURL?: string; // Workspace URL
  userCenterURL?: string; // User center URL
  logout: () => void; // Logout
}

.token: string

Optional, user token for calling api server interfaces.

.sides: SideItem[]

Optional, sidebar configuration, defaults to null, uses system default sidebar. For system built-in configurations, the panel parameter is not needed as it's already built-in.

export interface SideItem {
  type: string;
  name: string;
  enName: string;
  icon: React.ReactNode;
  simple?: boolean; // Whether to display in simple mode
  panel?: React.ReactNode;
}

// System default sides are as follows:
const fill = "var(--theme-icon)";
const sides = [
  {
    icon: <Page theme="outline" size="24" fill={fill} />,
    type: "template",
    name: "Template", // Template
    simple: true,
    enName: "Template",
  },
  {
    icon: <UploadOne theme="outline" size="24" fill={fill} />,
    type: "my",
    name: "My", // My
    simple: true,
    enName: "My",
  },
  {
    icon: <TextMessage theme="outline" size="24" fill={fill} />,
    type: "caption",
    name: "Caption", // Caption
    enName: "Caption",
  },
  {
    icon: <PictureOne theme="outline" size="24" fill={fill} />,
    type: "image",
    name: "Image", // Image
    simple: true,
    enName: "Image",
  },
  {
    icon: <Video theme="outline" size="24" fill={fill} />,
    type: "video",
    name: "Video", // Video
    simple: true,
    enName: "Video",
  },
  {
    icon: <Music theme="outline" size="24" fill={fill} />,
    type: "audio",
    name: "Audio", // Audio
    enName: "Audio",
  },
  {
    icon: <Text theme="outline" size="24" fill={fill} />,
    type: "text",
    name: "Text", // Text
    enName: "Text",
  },
  {
    icon: <InnerShadowBottomLeft theme="outline" size="24" fill={fill} />,
    type: "lottie",
    name: "Sticker", // Sticker
    enName: "Sticker",
  },
  {
    icon: <ColorFilter theme="outline" size="24" fill={fill} />,
    type: "filter",
    name: "Filter", // Filter
    enName: "Filter",
  },
  {
    icon: <Effects theme="outline" size="24" fill={fill} />,
    type: "effect",
    name: "Effect", // Effect
    enName: "Effect",
  },
  {
    icon: <InvertCamera theme="outline" size="24" fill={fill} />,
    type: "transition",
    name: "Transition", // Transition
    enName: "Transition",
  },
  {
    icon: <MoreTwo theme="outline" size="24" fill={fill} />,
    type: "more",
    name: "More", // More
    enName: "More",
  },
];

Sidebar configuration, defaults to null, uses system default sidebar. If sidebar configuration is passed in, the passed configuration will be used.

// eg: Call methods in movie, play & pause
vsdk.editor.movie.play();
vsdk.editor.movie.pause();

.data: MovieData

Original JSON data of the video project, data structure please refer to the data description document: video-core/src/react-pixi/types/data.ts

.loginButtonConfig: TargetConfig

Optional, login button configuration, defaults to null, uses system default login target. If login target configuration is passed in, the passed configuration will be used.

export interface TargetConfig {
  id: string; // Container ID
  className: string; // Container class name
  Component?: React.FC; // Login component
}

.exportButtonConfig: TargetConfig

Optional, export button configuration, defaults to null, uses system default export button. If export button configuration is passed in, the passed configuration will be used.

export interface TargetConfig {
  id: string; // Container ID
  className: string; // Container class name
  Component?: React.FC; // Login component
}

.apiServer: APIServer

Externally passed API service configuration, refer to the definition of APIServer (packages\video-editor\src\config\sdk.d.ts), if not passed, the official API will be used by default

import type {
  Err,
  AppDataRes,
  CreateAppParams,
  UpdateAppParams,
  TypesRes,
  TemplateParams,
  MaterialParams,
  MaterialRes,
  CollectRes,
  CollectParams,
  UserMaterialParams,
  UserMaterialTypeParams,
  CreateUserMaterialParams,
  UpdateUserMaterialParams,
} from "@config/sdk.d.ts";

interface APIServer {
  // Get appData
  getAppData: (appid: string) => Promise<[{ url: string }, Err]>;
  // Create new work
  createApp: (params: CreateAppParams) => Promise<[{ id: string }, Err]>;
  // Update work
  updateApp: (params: UpdateAppParams) => Promise<[string, Err]>;
  // Delete work
  deleteApp?: (id: string) => Promise<[string, Err]>;
  // Get template category information
  getTemplateTypes: () => Promise<[{ name: string; id: string }[], Err]>;
  // Get template list
  getTemplates: (
    params: TemplateParams
  ) => Promise<[{ name: string; id: string }[], Err]>;
  // Get material categories
  getMaterialTypes: (
    type: string
  ) => Promise<[{ name: string; id: string }[], Err]>;
  // Get materials
  getMaterials: (
    params: MaterialParams
  ) => Promise<[{ id: string; name: string }[], Err]>;
  // Collect element
  collect: (params: {
    source_id: string;
    type: string;
  }) => Promise<[string, Err]>;
  // Cancel collection
  cancelCollect: (sourceIds: string[]) => Promise<[string, Err]>;
  // Collection list
  getCollects: (
    params: CollectParams
  ) => Promise<[{ id: string; name: string }[], Err]>;
  // Upload base64 image
  uploadBase64: (
    params: UploadBase64Params
  ) => Promise<[{ storage_path: string }, Err]>;
  // Form upload
  formUpdate: (params: FormData) => Promise<[{ storage_path: string }, Err]>;
  // Get user materials
  getUserMaterial: (
    params: UserMaterialParams
  ) => Promise<[{ data: MaterialItemRes[]; total: number }, Err]>;
  // Get user material categories
  getUserMaterialType: (
    params: UserMaterialTypeParams
  ) => Promise<[{ data: { id: string; name: string }[]; total: number }, Err]>;
  // Delete user materials
  deleteUserMaterial: (ids: string[]) => Promise<[string, Err]>;
  // Add user materials
  createUserMaterial: (
    params: CreateUserMaterialParams
  ) => Promise<[MaterialItemRes, Err]>;
  // TTS
  createTTS: (
    params: CreateTTSParams
  ) => Promise<[{ storage_path: string }, Err]>;
  // AI caption, pass in audio url, return taskId, poll taskId to query conversion result
  createCaption: (url: string) => Promise<[{ TaskId: string }, Err]>;
  // Polling
  seekCaptionTask: (taskId: string) => Promise<[any, Err]>;
  // Modify user materials
  updateUserMaterial: (
    params: UpdateUserMaterialParams
  ) => Promise<[MaterialItemRes, Err]>;
}

.copyTempData: any

Cache copied data, which can be text content, element data, etc. After copying, assign it to this property, cache it in memory, and wait for pasting.

.copyTempFrameData: FrameItem[] | null

Cache copied frame data, wait for pasting.

@.previewSource: PreviewSourceParams

When clicking on preview materials, it will be cached for easy display

.activeItems: Record<ctypes.SourceType, SourceItem[]>

After resource switching, cache list data to avoid repeated requests to the server

@.editMode: template' | 'auto'

Editor modes: template replacement mode and free editing mode

@.optionPanelCustom: 'background' | ''

Control switching the panel of the settings area, switch to background settings

@.appid: string

Record the appid of the currently edited project

@.themeUpdateKey: 'dark' | 'light'

Record the current style theme type, modification will trigger view update

@.languageUpdateKey: 'zh-CN' | 'en-US'

Record the current multi-language parameter, modification will trigger view update

@.movieCreateSuccess: boolean

This parameter will be set to true when the movie instance is created successfully

@.currentTime: number

Time corresponding to the cursor position

@.rulerScale: number

Zoom ratio corresponding to 1s on the timeline, default is 50px, meaning 1 second corresponds to 50px

@.playing: boolean

Playback status, two states: playing and paused

@.updateKey: string

Used to control view updates in the settings area

@.movieDataUpdateKey: string

After replacing video data, this parameter needs to be modified, which will force update the canvas area

@.timelineUpdateKey: string

Used to control timeline view updates

@.timelineToolsUpdateKey: string

Used to control the view update of the shortcut toolbar above the timeline

@.totalTimeKey: string

This parameter needs to be modified after the total time changes to ensure synchronous updates in other places

@.frameSelectedId: string

When selecting the frame dot on the timeline, it will switch to frame mode. In frame mode, modifying element's coordinates, scaling, rotation, etc., directly modifies the frame state

@.selectedElementIds: string[]

IDs of selected elements in the editor, modification will automatically synchronize the selection state

@.layoutKeys: Record<ctypes.LayoutName, string>

Used to update layout identifiers, field changes will automatically update the specified module, for example:

// Update components in the header section
editor.layoutKeys.header = String(+new Date());

@.sourceType: ctypes.SourceType

Variable controlling resource panel switching, used with the setSourceType(t: ctypes.SourceType) method.

@.elementOptionType: ctypes.ElementOptionType

Variable controlling settings panel switching, used with the setElementOptionType(t: ctypes.ElementOptionType) method.

Method Description

setTheme(theme: 'dark' | 'light'): void

Set the current style theme type, modification will trigger view update

setLanguage(language: 'zh-CN' | 'en-US'): void

Set the current multi-language parameter, modification will trigger view update

updateMovie(t?: number): void

Update the view in the canvas area

updateOption(): void

Update settings area view

updateTimeline(): void

Update timeline area view

setActiveItems(items: SourceItem[], type: ctypes.SourceType): void

Set cache material resource data, cache already displayed material resource data for drag-and-drop use

getFromActiveItems(id: string, type: ctypes.SourceType): SourceItem

Read data from cached material resource data, mainly used for drag-and-drop

updateTimelineTools(): void

Update timeline toolbar module

record(params: types.RecordItem<types.RecordType>): void

Record user operation logs for redo/undo

export interface RecordItem<T> {
  desc: string; // Description information
  type: T;
}

export type RecordType =
  | "elements_delete"
  | "elements_create"
  | "elements_update"
  | "global";

vsdk.editor.record({
  type: "elements_update",
  desc: "Element updated",
});

setSelectedElementIds(ids: string[]): void

Pass in element IDs to select elements

// Select 2 elements at the same time
vsdk.editor.setSelectedElementIds(["elementId1", "elementId2"]);

setContorlAndSelectedElemenent(ids: string[]): Promise<null>

Select elements and set element controllers. Note that element controller settings are asynchronous, not synchronous.

// Select 2 elements at the same time and set element controllers
await vsdk.editor.setContorlAndSelectedElemenent(["elementId1", "elementId2"]);

updateComponent(...keyName: ctypes.LayoutName[]): void

Batch update specified modules

// Layout name
export type LayoutName =
  | "sources"
  | "timeline"
  | "options"
  | "canvas"
  | "header";

// Batch update timeline and settings area
await vsdk.editor.updateComponent("timeline", "options");

setSourceType(t: ctypes.SourceType): void

Method to switch left resource panel

// Resource panel type
export type SourceType =
  | "my"
  | "template"
  | "image"
  | "video"
  | "audio"
  | "text"
  | "filter"
  | "effect"
  | "transition"
  | "lottie"
  | "background"
  | "more"
  | string;

// Switch to image resource panel
await vsdk.editor.setSourceType("image");

setElementOptionType(t: ctypes.ElementOptionType): void

Method to switch settings panel

// Settings panel type
export type ElementOptionType =
  | "basic"
  | "audio"
  | "speed"
  | "animation"
  | "colour"
  | "caption"
  | "mask";

// Switch to animation settings
await vsdk.editor.setElementOptionType("animation");

getElementData(): types.BaseElement

Get selected element, if multiple elements are selected, only one element will be returned

// If editor.selectedElementIds has data, this method can be called to get the selected element's data
const elementData = vsdk.editor.getElementData();

getGroupElementData(): types.BaseElement[]

Similar to the getElementData method, get multiple selected elements, return an array

mergeCaption(id: string): void

Merge with the next caption into a new caption, pass in the caption ID here. If there is no data for the next caption, merging is not possible

deleteCaption(id: string): void

Delete caption, pass in the ID of the caption to be deleted

addCaption(text: string, index?: number): void

Add caption data, text is the caption content passed in, index is to add after the index + 1 element, if not passed, it will be inserted at the end

// Insert after the 2nd element
vsdk.editor.addCaption("Today is a good day", 1);

pause(time?: number): void

Pause to the specified time, if no parameter is passed, it will pause at the last frame