Task 4.2: Implement speech solutions

• ATrain a custom speech model with audio and matching transcripts

  Incorrect

  Custom speech improves accuracy further but needs data and training, which rules it out today.

• BIncrease the speaking rate of the agent's text to speech voice

  Incorrect

  Speaking rate affects the agent's voice output, not how callers are recognized.

• CAdd the names and acronyms to a phrase list for each session

  Correct

  Phrase lists are applied at runtime, need no training or large data set, and work with real-time transcription.

• DSwitch the calls to the batch transcription API with diarization

  Incorrect

  Batch is for prerecorded audio, and it doesn't support phrase lists.

For a short list of names and terms, a phrase list is the just-in-time fix: no training, applied per recognition session. Move to custom speech when you need thousands of phrases or adaptation to audio conditions.

• AA phoneme element around each brand name in every sentence

  Incorrect

  Phoneme tags work per occurrence; editing every sentence is what the team wants to avoid.

• BA custom lexicon file referenced in the SSML of every request

  Correct

  A custom lexicon defines pronunciations for many terms once, and the SSML lexicon element applies them to all text.

• CA phrase list containing the 150 names, loaded by each app

  Incorrect

  Phrase lists improve speech recognition input, not synthesized output.

• DA slower speaking rate set on the voice for each brand name

  Incorrect

  Speaking rate changes speed, not how a name is pronounced.

SSML offers element-level fixes for single spots and a custom lexicon for a shared pronunciation dictionary. Many terms across many apps point to the lexicon.

• AThe Voice Live API for managed speech-to-speech interaction

  Correct

  Voice Live combines recognition, generative AI and synthesis in one managed interface, with noise suppression, echo cancellation and interruption detection.

• BBatch synthesis of every possible answer ahead of time

  Incorrect

  Pre-rendered audio can't respond to what each caller says.

• CBatch transcription of recorded calls followed by a reply email

  Incorrect

  Batch processing of recordings can't hold a live conversation.

• DSpeech to text, a chat model and text to speech chained in app code

  Incorrect

  This works but is the orchestration the team wants to avoid, and chaining modules adds latency.

Voice Live is the managed speech-to-speech layer for voice agents. It can front a model directly or a Foundry agent, and it handles turn taking, noise and interruptions for you.

• ATrain a custom speech model on the audio and checked transcripts

  Correct

  A custom model trained on audio with reference transcripts improves domain vocabulary, and custom models work with batch transcription.

• BStream each recording through real-time transcription

  Incorrect

  Real-time recognition is for live audio; replaying 50,000 files through it is slower and costlier than batch.

• CUse the batch transcription API to process the whole archive

  Correct

  Batch transcription is built for large volumes of prerecorded audio.

• DUse batch synthesis to read the recordings back for human review

  Incorrect

  Batch synthesis turns text into speech, the opposite direction.

• EAdd the 3,000 terms to a phrase list on the batch jobs

  Incorrect

  The batch transcription API doesn't support phrase lists, and lists shouldn't exceed 2,000 phrases.

Prerecorded volume points to batch transcription. Because batch can't use phrase lists, and the term list is large, accuracy comes from a custom speech model trained on the insurer's own audio and transcripts.

• AMAI-Transcribe-2 with diarization on each interview file

  Incorrect

  MAI-Transcribe-2 offers diarization, timestamps and clean or verbatim styles, but it supports neither custom prompting nor translation.

• BSpeech to text with its default display text formatting

  Incorrect

  Display formatting applies fixed rules such as inverse text normalization and capitalization; you can't give it instructions.

• CLLM Speech, the enhanced mode of fast transcription

  Correct

  LLM Speech uses a multimodal model and supports custom prompting and translation in transcription.

• DPronunciation assessment during recognition of each interview

  Incorrect

  Pronunciation assessment scores how speech is pronounced; it doesn't apply formatting instructions to a transcript.

LLM Speech brings generative capabilities to transcription, such as custom prompting and translation, alongside diarization and timestamps.

• AThe Azure Speech MCP server, connected as an MCP tool

  Correct

  The Speech MCP server exposes speech capabilities to agents through the MCP tool, on resources that support MCP tools.

• BA Foundry IQ knowledge base built from audio files

  Incorrect

  Knowledge bases retrieve content; they don't run speech operations.

• CThe web search tool with "transcribe" in the query

  Incorrect

  Web search returns web results, not transcriptions.

• DThe code interpreter tool with an audio library

  Incorrect

  It doesn't call the Speech service for the agent.

Speech in Foundry Tools reaches agents through its MCP server. Check that the resource and region support MCP tools first.

• ABatch transcription of each video with diarization turned on

  Incorrect

  Transcripts alone don't replace the narration.

• BPronunciation assessment of the narrator

  Incorrect

  Pronunciation scoring doesn't translate anything.

• CReal-time speech translation from a live microphone stream

  Incorrect

  Real-time translation handles live audio streams, not producing dubbed video files.

• DVideo translation of each recorded training video

  Correct

  Video translation replaces a video's original language with translated speech, cutting the cost of human recording.

Azure Speech covers live translation for conversations and video translation for localizing recorded video content.

• ASwitching models requires deleting and recreating the endpoint

  Incorrect

  You can change the endpoint's model without downtime.

• BModels expire; retrain on newer base models and switch endpoints

  Correct

  Older models expire as new base models arrive; retrain periodically and change the endpoint's model without downtime.

• CExpired models keep working, but at a lower level of accuracy

  Incorrect

  An expired model can't be used for transcription.

• DCustom models never expire once they are deployed to an endpoint

  Incorrect

  Expired models are no longer available for transcription.

Treat custom speech as a lifecycle: monitor expiration dates, retrain from the latest base model and update endpoints before models expire.

• AText to speech with a high definition voice for each sentence

  Incorrect

  Synthesis produces speech; it doesn't assess the learner's.

• BLanguage identification run on each of the recordings

  Incorrect

  Identifying the language doesn't score pronunciation.

• CVideo translation of each of the learner's recordings

  Incorrect

  Translation doesn't evaluate pronunciation.

• DPronunciation assessment during speech recognition of each reading

  Correct

  Pronunciation assessment evaluates spoken pronunciation and supports streaming without a time limit through the Speech SDK.

Speech offers more than transcription: pronunciation assessment for learning scenarios, priced like speech to text.