A custom voice for Vapi: your own TTS server in 30 lines

What Vapi sends and expects

According to Vapi’s custom TTS guide, Vapi sends a POST with a JSON body whose message object has type voice-request, the text and the sampleRate it wants. The answer must be raw PCM with no header: mono, 16-bit signed, little-endian, at exactly that sample rate, with the content type application/octet-stream. With an inline secret, Vapi sends it in the X-VAPI-SECRET header.

The server

Install the packages and save this as vapi_voice.py:

pip install fastapi uvicorn scipy superclonica-cpu

import math, os
import numpy as np
from fastapi import FastAPI, Header, HTTPException, Request, Response
from fastapi.concurrency import run_in_threadpool
from scipy.signal import resample_poly
from superclonica import Superclonica

VOICE = os.environ.get("TTS_VOICE", "F1")  # M1..M5, F1..F5 or a voice file from the catalog
LANGUAGE = os.environ.get("TTS_LANGUAGE", "en")
SECRET = os.environ["VAPI_SECRET"]
app = FastAPI()
tts = Superclonica()

def voice_audio(text):
    audio, rate = tts.synthesize(text, voice=VOICE, lang=LANGUAGE)
    return np.asarray(audio, dtype=np.float32).reshape(-1), rate

@app.post("/synthesize")
async def synthesize(request: Request, x_vapi_secret: str = Header(None)):
    if x_vapi_secret != SECRET:
        raise HTTPException(401)
    message = (await request.json())["message"]
    audio, rate = await run_in_threadpool(voice_audio, message["text"])
    target = int(message["sampleRate"])
    if target != rate:
        divisor = math.gcd(target, rate)
        audio = resample_poly(audio, target // divisor, rate // divisor)
    pcm = (np.clip(audio, -1, 1) * 32767).astype(np.int16).tobytes()
    return Response(pcm, media_type="application/octet-stream")

The engine voices the text at 44.1 kHz in a worker thread, scipy resamples it to the rate Vapi asked for, and the server returns it as 16-bit PCM. Requests without the right secret get 401. Start it with uvicorn:

uvicorn vapi_voice:app --host 0.0.0.0 --port 8000

Connect it to your assistant

Vapi has to reach the server over public HTTPS: put it behind your usual reverse proxy or, for a test, a tunnel. Then set the assistant’s voice, for example when you update the assistant through the Vapi API:

{
  "voice": {
    "provider": "custom-voice",
    "server": {
      "url": "https://your-server.example.com/synthesize",
      "secret": "the value of VAPI_SECRET",
      "timeoutSeconds": 30
    }
  }
}

Set VAPI_SECRET on the server to the same value. Vapi’s guide also describes credentials created in its dashboard and a fallback voice, which is worth adding so a call keeps talking if your server is down.

The same server through the API

If your server is too small for the model, voice the text through the SuperClonica speech API instead: replace voice_audio with this function and set SUPERCLONICA_KEY and VOICE_ID. The resampling and the answer stay the same.

import io, wave
import requests

API = "https://superclonica.com/api/v1/speech"
VOICE_ID = os.environ["VOICE_ID"]  # the ID on the page of a catalog voice

def voice_audio(text):
    answer = requests.post(API, timeout=60,
                           headers={"Authorization": "Bearer " + os.environ["SUPERCLONICA_KEY"]},
                           json={"text": text, "voice": VOICE_ID, "language": LANGUAGE, "format": "wav"})
    answer.raise_for_status()
    with wave.open(io.BytesIO(answer.content)) as file:
        rate = file.getframerate()
        pcm = np.frombuffer(file.readframes(file.getnframes()), dtype=np.int16)
    return pcm.astype(np.float32) / 32768, rate

Every catalog voice works by the ID on its page, on any plan. Starter costs $5 a month for up to 3 minutes of speech in every 10-minute window for all calls together, Pro 20 minutes.

Speed and voices

The server voices each request in full before it answers. In our measurements a short phrase took a median of 0.43 s on a desktop Ryzen with four threads and 0.12 s on an RTX 5070 Ti, so keep the assistant’s sentences short. TTS_VOICE takes one of the ten stock voices, M1 to M5 and F1 to F5, or a voice file from the catalog; Supertonic 3 speaks 31 languages but not Chinese yet.