SDK-first: always use the official SDK — see sdk-integration for policy, setup, and fallback criteria.

References

Consult these sibling skills as needed:

• ../sdk-integration/SKILL.md -- SDK setup, client initialization, error handling, and SDK vs raw API decision guide
• ../sdk-integration/references/sdk-versions.md -- Current SDK versions (auto-synced by CI)
• ../troubleshooting/SKILL.md -- Common errors, gotchas, and verification checklist
• ../live-transcription/SKILL.md -- Live streaming transcription
• ../pre-recorded-transcription/SKILL.md -- Pre-recorded file transcription

name: Gladia description: Use when building speech-to-text transcription features, processing audio or video files, implementing real-time transcription, extracting insights from audio (translation, summarization, speaker identification), or integrating audio intelligence into applications. metadata: mintlify-proj: gladia version: "1.0"

Gladia Skill

Product summary

Gladia is a speech-to-text API that transcribes audio and video files in two modes: pre-recorded (asynchronous, batch) and live (real-time, WebSocket-based). The API returns structured transcripts with word-level timing, confidence scores, and optional audio intelligence features (translation, diarization, summarization, entity recognition, sentiment analysis, PII redaction, subtitles). Use the JavaScript/TypeScript SDK (@gladiaio/sdk) or Python SDK (gladiaio-sdk) for simplified integration, or call REST/WebSocket endpoints directly. Authenticate with x-gladia-key header. Primary docs: https://docs.gladia.io

When to use

• Pre-recorded transcription: Transcribe uploaded audio/video files (MP3, WAV, MP4, YouTube links, etc.) asynchronously. Typical latency: seconds to minutes depending on file length.
• Live transcription: Stream audio in real-time via WebSocket for immediate transcripts (e.g., call centers, live events, voice assistants).
• Audio intelligence: Extract metadata from transcripts — translate to multiple languages, identify speakers, detect sentiment, redact PII, generate summaries, create subtitles, recognize named entities.
• Custom vocabulary: Improve accuracy for domain-specific terms, brand names, proper nouns by providing phonetic hints.
• Multi-speaker scenarios: Use diarization to attribute speech to individual speakers, or send multi-channel audio to preserve speaker identity.

Quick reference

Authentication

# All requests require x-gladia-key header
curl -H "x-gladia-key: YOUR_API_KEY" https://api.gladia.io/v2/pre-recorded

Pre-recorded workflow (SDK)

import { GladiaClient } from "@gladiaio/sdk";
const client = new GladiaClient({ apiKey: "YOUR_KEY" });
const result = await client.preRecorded().transcribe("audio_url_or_local_path");

Live workflow (SDK)

const session = client.liveV2().startSession({
  encoding: "wav/pcm",
  sample_rate: 16000,
  bit_depth: 16,
  channels: 1,
  language_config: { languages: ["en"] }
});
session.on("message", (msg) => console.log(msg));
session.sendAudio(audioChunk);
session.stopRecording();

Audio formats

Limits

Audio intelligence features

Decision guidance

When to use pre-recorded vs. live

When to use SDK vs. raw API

When to use diarization vs. multi-channel audio

When to use custom vocabulary vs. custom spelling

Workflow

Pre-recorded transcription (typical task)

1. Prepare audio: Ensure file is under 1000 MB and 135 minutes. Supported formats: MP3, WAV, MP4, YouTube URL, etc.
2. Choose delivery method: Use SDK for simplicity, or raw API for control.
3. Configure transcription:
   • Set language_config.languages explicitly if known (avoids detection overhead).
   • Enable diarization: true if multiple speakers.
   • Add custom_vocabulary for domain terms.
   • Enable audio intelligence features (translation, summarization, etc.) as needed.
4. Submit job: Call transcribe() (SDK) or POST /v2/pre-recorded (API).
5. Retrieve results: Poll GET /v2/pre-recorded/:id or configure webhooks/callbacks.
6. Parse response: Extract transcription.utterances[] for text and timing, plus any audio intelligence results.

Live transcription (typical task)

1. Initialize session: Call POST /v2/live with audio config (encoding, sample_rate, bit_depth, channels).
2. Connect WebSocket: Use returned URL to open WebSocket connection.
3. Configure messages: Set messages_config to specify which message types to receive (transcripts, partial transcripts, post-processing events).
4. Stream audio: Send audio chunks via sendAudio() (SDK) or binary/base64 JSON (raw API).
5. Handle messages: Listen for transcript messages; check is_final to distinguish partials from finals.
6. Stop recording: Call stopRecording() to trigger post-processing (diarization, translation, etc.).
7. Retrieve final result: Poll GET /v2/live/:id or wait for callback with complete result.

Adding custom vocabulary

1. Identify problem terms: Transcribe without custom vocabulary; note mis-transcribed words.
2. Categorize: Garbled/phonetically wrong → custom vocabulary; recognizable but misspelled → custom spelling.
3. Build vocabulary list:

   {
     "custom_vocabulary": true,
     "custom_vocabulary_config": {
       "vocabulary": [
         "Gladia",
         { "value": "Salesforce", "pronunciations": ["sell force"], "intensity": 0.5 }
       ],
       "default_intensity": 0.4
     }
   }
   

4. Test: Transcribe again; confirm targets appear and check for false positives.
5. Refine: Lower intensity, add pronunciations, or move stubborn terms to custom spelling.

Common gotchas

• Language detection overhead: Always set language_config.languages explicitly if you know the language. Auto-detection adds latency and can fail if audio starts with silence or music.
• Code switching without language list: Never enable code_switching: true with an empty languages array — the model will evaluate against 100+ languages, causing frequent misdetections. Always provide a constrained list (3–5 languages).
• Diarization hints are not hard constraints: number_of_speakers, min_speakers, max_speakers are hints, not guarantees. The model may detect a different count.
• Custom vocabulary intensity tuning: Start at default_intensity: 0.4 and adjust per-entry only. Raising intensity globally increases false positives. Add pronunciations variants before raising intensity.
• Live session 3-hour limit: A single WebSocket session cannot exceed 3 hours. For longer events, close the session and start a new one before hitting the limit.
• Pre-recorded 135-minute limit: Files longer than 135 minutes will fail. Split into ~60-minute chunks using ffmpeg or similar tools.
• Audio format conversion overhead: Large video files (e.g., AVI, MOV) take ~1 minute to convert to WAV/PCM. Plan for this latency.
• Polling without webhooks: If you poll GET /v2/pre-recorded/:id in a tight loop, you'll hit rate limits. Use webhooks or callbacks instead, or poll with exponential backoff.
• Multi-channel billing: Transcribing multi-channel audio is billed as duration × number_of_channels. A 10-minute 3-channel stream costs 30 minutes of usage.
• Partial transcripts in live mode: Partial transcripts are low-latency but less accurate. Always check is_final: true before using a transcript for critical decisions.
• Missing audio_url on upload: After uploading a file, the response includes audio_url — use this URL in the transcription request, not the local file path.
• WebSocket reconnection: If the WebSocket disconnects, reconnect to the same URL (returned from init) to resume the session without losing context.

Verification checklist

Before submitting transcription work:

• API key is valid and passed in x-gladia-key header.
• Audio file is under 1000 MB and 135 minutes (pre-recorded) or 3 hours (live).
• Audio format is supported (MP3, WAV, MP4, etc.).
• Language is set explicitly in language_config.languages if known.
• If using code switching, languages list is constrained to 3–5 expected languages.
• Diarization is enabled if multiple speakers need attribution.
• Custom vocabulary entries have realistic intensity (0.4–0.6) and pronunciations.
• Webhooks or callbacks are configured if polling is not feasible.
• Live sessions are closed before 3 hours; pre-recorded jobs are split if over 135 minutes.
• Response includes expected fields: transcription.utterances[], metadata, and any requested audio intelligence results.
• Confidence scores and timing (start, end) are present for quality validation.
• Multi-channel audio is correctly interleaved if merging multiple sources.

Resources

• Comprehensive page listing: https://docs.gladia.io/llms.txt
• Getting started guide: https://docs.gladia.io/chapters/introduction/getting-started
• Pre-recorded quickstart: https://docs.gladia.io/chapters/pre-recorded-stt/quickstart
• Live transcription quickstart: https://docs.gladia.io/chapters/live-stt/quickstart
• API reference: https://docs.gladia.io/api-reference
• Recommended parameters by use case: https://docs.gladia.io/chapters/pre-recorded-stt/recommended-parameters
• Audio intelligence features: https://docs.gladia.io/chapters/audio-intelligence
• Supported formats and limits: https://docs.gladia.io/chapters/limits-and-specifications/supported-formats

For additional documentation and navigation, see: https://docs.gladia.io/llms.txt

This file is auto-synced from https://docs.gladia.io/.well-known/agent-skills/gladia/skill.md Do not edit manually — changes will be overwritten by CI. For additional documentation and navigation, see: https://docs.gladia.io/llms.txt