360 Secure SkillHub
360 安全技能中心

Ai Unified Video Lyrics

面向音乐人和内容创作者的视频歌词同步 Skill：上传最大 500MB 的含音频视频（MP4/MOV/AVI/WebM/MKV 等），用自然语言描述需求，由远端 GPU 节点将歌词按节拍同步为动画字幕并渲染，约 1-2 分钟返回 1080p MP4。流程依赖匿名令牌与 NemoVideo 后端会话，涵盖上传、积分查询、状态查看与导出。

tk8544-b
tk8544-b
热度 34.15
最近更新 2026-05-11 09:58:09
分类：创作依赖：第三方 API应用场景：视频编辑 / 自动化脚本

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add ai-unified-video-lyrics
Metadata
name
ai-unified-video-lyrics
version
1.0.0
displayName
AI Unified Video Lyrics — Sync Lyrics to Music Videos
description
Skip the learning curve of professional editing software. Describe what you want — sync song lyrics as animated captions to the beat of the video — and get lyrics-synced videos back in 1-2 minutes. Upload MP4, MOV, AVI, WebM files up to 500MB, and the AI handles AI lyrics sync automatically. Ideal for musicians and content creators who need lyrics automatically timed and overlaid onto music videos.
metadata
{"openclaw":{"emoji":"🎵","requires":{"env":["NEMO_TOKEN"],"configPaths":["~/.config/nemovideo/"]},"primaryEnv":"NEMO_TOKEN","variant":"greeting_v2"}}

Getting Started

Got video with audio to work with? Send it over and tell me what you need — I'll take care of the AI lyrics sync.

Try saying:

  • "add a 3-minute music video file into a 1080p MP4"
  • "sync song lyrics as animated captions to the beat of the video"
  • "adding synced lyric captions to music videos for musicians and content creators"

First-Time Connection

When a user first opens this skill, connect to the processing backend automatically. Briefly let them know (e.g. "Setting up...").

Authentication: Check if NEMO_TOKEN is set in the environment. If it is, skip to step 2.

  1. Obtain a free token: Generate a random UUID as client identifier. POST to https://mega-api-prod.nemovideo.ai/api/auth/anonymous-token with header X-Client-Id set to that UUID. The response data.token is your NEMO_TOKEN — 100 free credits, valid 7 days.
  2. Create a session: POST to https://mega-api-prod.nemovideo.ai/api/tasks/me/with-session/nemo_agent with Authorization: Bearer <token>, Content-Type: application/json, and body {"task_name":"project","language":"<detected>"}. Store the returned session_id for all subsequent requests.

Keep setup communication brief. Don't display raw API responses or token values to the user.

AI Unified Video Lyrics — Sync Lyrics to Music Videos

Send me your video with audio and describe the result you want. The AI lyrics sync runs on remote GPU nodes — nothing to install on your machine.

A quick example: upload a 3-minute music video file, type "sync song lyrics as animated captions to the beat of the video", and you'll get a 1080p MP4 back in roughly 1-2 minutes. All rendering happens server-side.

Worth noting: cleaner audio tracks produce more accurate lyric sync results.

Matching Input to Actions

User prompts referencing ai unified video lyrics, aspect ratio, text overlays, or audio tracks get routed to the corresponding action via keyword and intent classification.

User says... Action Skip SSE?
"export" / "导出" / "download" / "send me the video" → §3.5 Export ✅
"credits" / "积分" / "balance" / "余额" → §3.3 Credits ✅
"status" / "状态" / "show tracks" → §3.4 State ✅
"upload" / "上传" / user sends file → §3.2 Upload ✅
Everything else (generate, edit, add BGM…) → §3.1 SSE ❌

Cloud Render Pipeline Details

Each export job queues on a cloud GPU node that composites video layers, applies platform-spec compression (H.264, up to 1080x1920), and returns a download URL within 30-90 seconds. The session token carries render job IDs, so closing the tab before completion orphans the job.

All calls go to https://mega-api-prod.nemovideo.ai. The main endpoints:

  1. Session — POST /api/tasks/me/with-session/nemo_agent with {"task_name":"project","language":"<lang>"}. Gives you a session_id.
  2. Chat (SSE) — POST /run_sse with session_id and your message in new_message.parts[0].text. Set Accept: text/event-stream. Up to 15 min.
  3. Upload — POST /api/upload-video/nemo_agent/me/<sid> — multipart file or JSON with URLs.
  4. Credits — GET /api/credits/balance/simple — returns available, frozen, total.
  5. State — GET /api/state/nemo_agent/me/<sid>/latest — current draft and media info.
  6. Export — POST /api/render/proxy/lambda with render ID and draft JSON. Poll GET /api/render/proxy/lambda/<id> every 30s for completed status and download URL.

Formats: mp4, mov, avi, webm, mkv, jpg, png, gif, webp, mp3, wav, m4a, aac.

Skill attribution — read from this file's YAML frontmatter at runtime:

  • X-Skill-Source: ai-unified-video-lyrics
  • X-Skill-Version: from frontmatter version
  • X-Skill-Platform: detect from install path (~/.clawhub/ → clawhub, ~/.cursor/skills/ → cursor, else unknown)

All requests must include: Authorization: Bearer <NEMO_TOKEN>, X-Skill-Source, X-Skill-Version, X-Skill-Platform. Missing attribution headers will cause export to fail with 402.

Draft field mapping: t=tracks, tt=track type (0=video, 1=audio, 7=text), sg=segments, d=duration(ms), m=metadata.

Timeline (3 tracks): 1. Video: city timelapse (0-10s) 2. BGM: Lo-fi (0-10s, 35%) 3. Title: "Urban Dreams" (0-3s)

Backend Response Translation

The backend assumes a GUI exists. Translate these into API actions:

Backend says You do
"click [button]" / "点击" Execute via API
"open [panel]" / "打开" Query session state
"drag/drop" / "拖拽" Send edit via SSE
"preview in timeline" Show track summary
"Export button" / "导出" Execute export workflow

Reading the SSE Stream

Text events go straight to the user (after GUI translation). Tool calls stay internal. Heartbeats and empty data: lines mean the backend is still working — show "⏳ Still working..." every 2 minutes.

About 30% of edit operations close the stream without any text. When that happens, poll /api/state to confirm the timeline changed, then tell the user what was updated.

Error Handling

Code Meaning Action
0 Success Continue
1001 Bad/expired token Re-auth via anonymous-token (tokens expire after 7 days)
1002 Session not found New session §3.0
2001 No credits Anonymous: show registration URL with ?bind=<id> (get <id> from create-session or state response when needed). Registered: "Top up credits in your account"
4001 Unsupported file Show supported formats
4002 File too large Suggest compress/trim
400 Missing X-Client-Id Generate Client-Id and retry (see §1)
402 Free plan export blocked Subscription tier issue, NOT credits. "Register or upgrade your plan to unlock export."
429 Rate limit (1 token/client/7 days) Retry in 30s once

Tips and Tricks

The backend processes faster when you're specific. Instead of "make it look better", try "sync song lyrics as animated captions to the beat of the video" — concrete instructions get better results.

Max file size is 500MB. Stick to MP4, MOV, AVI, WebM for the smoothest experience.

Export as MP4 for widest compatibility across streaming and social platforms.

Common Workflows

Quick edit: Upload → "sync song lyrics as animated captions to the beat of the video" → Download MP4. Takes 1-2 minutes for a 30-second clip.

Batch style: Upload multiple files in one session. Process them one by one with different instructions. Each gets its own render.

Iterative: Start with a rough cut, preview the result, then refine. The session keeps your timeline state so you can keep tweaking.