Create speech-enabled apps with Microsoft Foundry | AI-103 | Episode 18

Azure AI Speech provides specialized APIs for adding speech-to-text (STT) and text-to-speech (TTS) to applications. Compared with using a general-purpose generative model for speech, dedicated Speech services provide more predictable latency, cost, audio formats, voices, logging, and output—but they do not reason about the meaning or intent of the content.

Core SDK pattern

Both directions start with SpeechConfig, which connects the SDK to the Azure/Foundry Speech resource using an endpoint + key or Entra ID credential.

Speech → Text:
SpeechConfig + input AudioConfig → SpeechRecognizer → recognize_once_async() → text + result metadata

Text → Speech:
SpeechConfig + output AudioConfig → SpeechSynthesizer → speak_text_async() → audio + result metadata

import azure.cognitiveservices.speech as speechsdk

speech = speechsdk.SpeechConfig(
    subscription=KEY, endpoint=ENDPOINT)

# Speech → Text
audio = speechsdk.audio.AudioConfig(filename="input.wav")
recognizer = speechsdk.SpeechRecognizer(speech, audio)
text = recognizer.recognize_once_async().get().text

# Text → Speech
audio = speechsdk.audio.AudioOutputConfig(filename="output.wav")
synth = speechsdk.SpeechSynthesizer(speech, audio)
synth.speak_text_async("Hello Azure!").get()

Remember the responsibilities

SpeechConfig = service connection + speech settings
AudioConfig = where audio comes from or goes
SpeechRecognizer = audio → text
SpeechSynthesizer = text → audio

For finer TTS control, use SSML (Speech Synthesis Markup Language) instead of plain text. SSML can control voice, speaking style, pronunciation, pauses, pitch, rate, and how specific values are spoken.

💡 Key distinction: Change the voice through SpeechConfig; use SSML when you need detailed control over how speech is delivered.