Build on Agentyk

Quick start

Connect as for live transcription and add translation to the start message. Then send audio and stop as usual; every transcript frame is unchanged, and the translations arrive beside them.

{"type":"start","language":"nl","refine":true,"translation":{"target_languages":["en","de"]}}

{"type":"translation","segment_id":3,"source_language":"nl","target_language":"en","text":"Currently, only dragonflies cannot fold their wings.","start":12.4,"end":17.85,"speaker":2}

// Live translated captions from Agentyk Scribe Live.
//
//   npm install ws
//   AGENTYK_API_KEY=agk_... node live-translation.mjs talk.wav nl en,de
//
// Streams a 16 kHz mono 16-bit WAV file at real time and prints the transcript and the
// translations as they arrive. In a browser, use the built-in WebSocket with the subprotocols
// ["agentyk.v1", "agentyk-key." + key] instead of the Authorization header.
import { readFileSync } from "node:fs";
import WebSocket from "ws";

const [file, language = "nl", targets = "en"] = process.argv.slice(2);
const key = process.env.AGENTYK_API_KEY;
if (!file || !key) {
  console.error("usage: AGENTYK_API_KEY=agk_... node live-translation.mjs <file.wav> <language> <target,target>");
  process.exit(2);
}

// Minimal WAV reader: finds the PCM data chunk.
const wav = readFileSync(file);
let pcm = null;
for (let off = 12; off < wav.length;) {
  const id = wav.toString("ascii", off, off + 4);
  const size = wav.readUInt32LE(off + 4);
  if (id === "data") { pcm = wav.subarray(off + 8, off + 8 + size); break; }
  off += 8 + size + (size % 2);
}

const ws = new WebSocket("wss://api.agentyk.xyz/v1/realtime?model=agentyk-scribe-live-1.0", {
  headers: { Authorization: `Bearer ${key}` },
});

ws.on("open", () => {
  ws.send(JSON.stringify({
    type: "start",
    language,
    refine: true,
    sample_rate: 16000,
    translation: { target_languages: targets.split(",") },
  }));
});

function streamAudio() {
  const frameBytes = 3200; // 100 ms at 16 kHz, 16-bit mono
  const t0 = Date.now();
  let sent = 0;
  const tick = () => {
    if (ws.readyState !== WebSocket.OPEN) return;
    if (sent >= pcm.length) { ws.send(JSON.stringify({ type: "stop" })); return; }
    ws.send(pcm.subarray(sent, sent + frameBytes));
    sent += frameBytes;
    setTimeout(tick, Math.max(0, t0 + (sent / 32000) * 1000 - Date.now())); // real time
  };
  tick();
}

ws.on("message", (data, isBinary) => {
  if (isBinary) return;
  const msg = JSON.parse(data.toString());
  switch (msg.type) {
    case "busy":
      console.error(`busy, retry in ${msg.retry_after_ms} ms`);
      break;
    case "ready":
      if (msg.translation === false) console.error("translation is not available for this session");
      streamAudio();
      break;
    case "utterance":
      console.log(`[${msg.id}] ${msg.text}`);
      break;
    case "translation":
      console.log(`[${msg.segment_id}] ${msg.target_language}: ${msg.text}`);
      break;
    case "translation_error":
      console.error(`[${msg.segment_id}] ${msg.target_language}: (${msg.code})`);
      break;
    case "error":
      console.error(`session error: ${msg.code}: ${msg.message}`);
      break;
    case "done":
      console.error(`done: ${(msg.audio_ms / 1000).toFixed(1)} s of audio`);
      break;
  }
});

ws.on("close", () => process.exit(0));

Both examples stream a 16 kHz mono 16-bit WAV file at real time and print the captions. In a browser, pass the key as the subprotocols agentyk.v1 and agentyk-key.agk_... instead of the Authorization header.

The start message

Languages

24 target languages. A session takes up to 5 of them.

What you receive

ready carries a translation field when you asked for it:

{"type":"ready","language":"nl","refine":true,"translation":{"target_languages":["en","de"]}}

• {"target_languages": [...]}: translation is on for this session, into these languages (lower-case, in your order).
• false: translation is not available for this session. The session runs as plain live transcription; nothing else changes.

translation: one frame per finished segment and target language.

translation_error: only that segment and that language are affected; the session and the transcript carry on. It is deliberately not an error frame, because an error ends a session.

{"type":"translation_error","segment_id":4,"target_language":"de","code":"timeout","message":"translation took too long for this segment"}

How it behaves

• Transcript frames are never held back by translation: partial, final, utterance, transcript and done arrive exactly as without translation, and translations arrive beside them.
• For each target language, translation and translation_error frames arrive in segment order. Frames of different target languages may interleave.
• Each translated segment gets exactly one frame per target language: a translation or a translation_error. Filler and repeated segments (below) get none.
• Translation uses spare capacity and gives way to other work. When the service is busy it can fall behind; once more than 8 segments per target language are waiting, the oldest one is skipped with translation_error code backlog so the captions stay current. Skipped segments are not billed.
• Each segment is translated with the 3 segments before it as context. Translations are shown once and not revised: a later correction of the transcript (for example the transcript frame at stop) does not re-send them.
• Segments made only of filler sounds (such as "uh" or "hmm"), or of a caption line that speech recognition sometimes writes for silence, are not translated and not billed. A segment of four or more words that repeats one of the last three is not translated again.
• At stop the last sentences are translated before the session ends: their frames arrive before done. stop waits up to 15 seconds for them; anything still open then gets translation_error code cancelled.
• Targets are set once, in start. To change them, start a new session.
• Translation works on the recognised text, so a recognition error carries into the translation. Very quiet or near-silent audio may produce no text, and therefore no translation.

Measured on the live service on 2026-11-03 (Dutch speech into English, German and French, 309 captions):

• A translation arrives about 0.4 seconds after its segment is finished (95 percent within 0.7 seconds).
• From the moment the speaker finishes a sentence to its translated caption: about 2.3 seconds (95 percent within 3 seconds).

Errors

A translation option that cannot be accepted ends the session at once, with an error frame of code bad_request:

Billing

Questions