Examples · Programming

Text to speech in Python

Turn a text into an MP3 with one of the school's six voices, then let the transcription model listen to it and check that it says what you wrote.

Beginner · 20 min

What you need

A key and the openai library, as in Your first API call in Python.

The voices

Six voices, each made for one language. Write the text in the language of its voice: an English voice reads an Italian text with English pronunciation.

Voice	Language
`paola`	Italian, female
`riccardo`	Italian, male
`amy`	English, female
`ryan`	English, male
`thorsten`	German, male
`siwis`	French, female

Names are in lower case: Amy is refused, and the error lists the valid names.

The program

Save it as speak.py:

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.ai.itsincom.org/v1",
    api_key=os.environ["API_KEY"],
)

speech = client.audio.speech.create(
    model="tts-1",
    voice="amy",
    input="Hello! I am a program written by a student of ITS INCOM, and today I learned to speak.",
    response_format="mp3",
)
speech.write_to_file("hello.mp3")
print("hello.mp3 is ready")

Run it with python3 speak.py and open hello.mp3 with any player. It takes a second or two.

Two things work differently from OpenAI:

model is required by the library, but the school has one voice engine: write tts-1, the value is not used.
Without response_format="mp3" you get a WAV, several times bigger. If you then save it as hello.mp3, the name lies about what is inside.

Did it say what you wrote?

You cannot listen to a hundred files. Let the transcription model listen for you. Save it as check.py:

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.ai.itsincom.org/v1",
    api_key=os.environ["API_KEY"],
)

with open("hello.mp3", "rb") as f:
    heard = client.audio.transcriptions.create(model="whisper-1", file=f, language="en")

print("heard:", heard.text)

Run it with python3 check.py.

What you should see

heard: Hello, I am a program written by a student of its income, and today I learned to speak.

Almost. Names and acronyms are where speech goes wrong: the voice may read letters as a word, and the transcription may write the word it knows best. Tell the transcription which names to expect with prompt, in check.py:

with open("hello.mp3", "rb") as f:
    heard = client.audio.transcriptions.create(model="whisper-1", file=f, language="en", prompt="ITS INCOM")

When we tried it, the school's name came back right.

Good to know

Up to 5,000 characters per request. For a longer text, make one file per paragraph.
Each request is a new take: the same text comes out a little different every time. When you like a take, keep the file.
No speed control: speed is accepted and ignored. Every voice speaks at its own pace.
The text is checked before it is read. Stories, villains and battle cries are fine; sexual content, hate and attacks on real people are not. A refused text raises openai.BadRequestError with e.code set to input_refused, and the message says why. If the check cannot be made for a moment, you get 503 with the code moderation_unavailable: the library already tries again twice, so wait a minute and run the program again.
The MP3 says it was made by AI, in its metadata. When you publish it, say so in words too.

Exercise

Make paola read an Italian sentence and thorsten a German one. Check them with check.py, with language="it" and language="de".
Give amy an Italian sentence and listen. Then transcribe it with language="it": what comes back?
Put your name, or the name of your town, in the text. Does it come back right? And with prompt?
Read a long text, one MP3 per paragraph: pathlib.Path("notes.txt").read_text().split("\n\n") gives you the paragraphs.

All examples