Snippset Feed

Voice-enabled generative AI is essentially a two-way conversion pipeline: speech → text lets an application understand spoken input, while text → speech turns generated responses back into audio. Azure AI Foundry provides specialized models for both inference tasks.

Know which model solves which problem:

A transcription model such as GPT-4o-mini-transcribe accepts audio and returns text. A TTS model such as GPT-4o-mini-tts performs the reverse and can also follow instructions affecting characteristics such as tone.

The implementation pattern is straightforward: deploy the appropriate model in Foundry → create an authenticated Azure OpenAI client → call the corresponding audio API → handle text or binary audio output. Streaming is useful for TTS because audio bytes can be consumed as they arrive rather than waiting for the complete response.

# Speech → Text
with open("speech.wav", "rb") as audio:
    text = client.audio.transcriptions.create(
        model="gpt-4o-mini-transcribe",
        file=audio
    )

# Text → Speech
with client.audio.speech.with_streaming_response.create(
    model="gpt-4o-mini-tts",
    voice="alloy",
    input="Hello from Azure AI"
) as audio:
    audio.stream_to_file("speech.mp3")

Remember the direction: Transcribe = audio in, text out. TTS = text in, audio out. The audio side is binary data, so applications must correctly read input files or stream/write generated audio.

Microsoft 365 Copilot Chat and Microsoft 365 Copilot use the same conversational AI experience, but differ significantly in what information they can use. The easiest way to remember the distinction is web AI vs. work-aware AI.

The key difference: grounding

Copilot Chat

LLM
 ├─ Web
 └─ Files/content you explicitly provide

Microsoft 365 Copilot

LLM
 ├─ Web
 └─ Work IQ
     └─ Microsoft Graph
         ├─ Emails
         ├─ Teams chats & meetings
         ├─ OneDrive / SharePoint files
         └─ People & organizational context

With a Microsoft 365 Copilot license, Work IQ lets Copilot reason over work information the user is permitted to access. Without that license, Copilot Chat is primarily web-grounded, although uploaded files and some app-specific experiences can provide additional context.

At a glance

Copilot Chat is included with eligible Microsoft 365 subscriptions, while Microsoft 365 Copilot requires an additional license.

Mental model:
Copilot Chat → AI assistant for work
Microsoft 365 Copilot → AI assistant that understands your work context

Getting better results from a generative AI model does not automatically mean fine-tuning it. The key is identifying what is wrong with the output and choosing the least complex technique that solves it.

The decision you should remember

Start with prompt engineering. System instructions can define the model's role, constraints, tone and expected output. Examples and few-shot prompting can further improve consistency.

RAG = give the model knowledge

Retrieval-Augmented Generation is appropriate when the required information is too large, specialized or dynamic to include directly in the prompt:

Question → Vectorize → Retrieve relevant chunks → Question + context → LLM → Answer

The important distinction is that RAG does not retrain the model. Relevant external information is retrieved and added to the model's context at runtime.

Fine-tuning = change behavior

Fine-tuning trains a base model using many examples of desired input/output behavior. A supervised dataset commonly contains conversations such as:

{"messages":[
  {"role":"user","content":"Suggest a destination."},
  {"role":"assistant","content":"Absolutely! What kind of trip interests you?"}
]}

Microsoft Foundry supports different customization methods depending on the selected model, including supervised fine-tuning and Direct Preference Optimization (DPO). Model and region must support the chosen method.

Key distinction: Fine-tuning is comparatively expensive and time-consuming, produces a new fixed model that must be deployed, and must be repeated when training requirements change.

Remember

Prompt → instructions. RAG → knowledge. Fine-tuning → behavior.

When uncertain, try prompting first, use RAG for missing knowledge, and reserve fine-tuning for behavior that prompting cannot reliably achieve.

Generative AI models are powerful, but their trained knowledge is limited. Tools extend models beyond text generation, allowing them to access real-time information, take actions, ground responses in facts, extend functionality, and build intelligent workflows.

Know the Tools

Remember: current information → web_search · uploaded/private documents → file_search · calculations/code → code_interpreter · application-specific actions → function

Responses API

Tools are provided through the tools collection. The model can determine which available tool is appropriate for a request.

response = client.responses.create(
    model=model_name,
    input="Answer the user's request using the available tools.",
    tools=[
        {"type": "code_interpreter", "container": {"type": "auto"}},
        {"type": "web_search"},
        {"type": "file_search", "vector_store_ids": [vector_store.id]}
    ]
)

print(response.output_text)

Core flow: User → Responses API → Model → Tool → Result → Model → Response

For file_search, documents are stored in a vector store and prepared for semantic retrieval:

Files → Chunking → Embeddings → Vector Store → Retrieval → Model

This lets the model answer using relevant document content rather than relying only on its trained knowledge. Uploaded company policies or private documents → File Search + Vector Store.

Function Calling

Functions are different because the application executes the function, not the model. The model identifies the required function and returns a function-call request:

User → Model → Function Call → Application → Function → Result → Model → Response

The application executes the requested code and returns its result. This process can run in a loop when multiple tool calls are needed.

Key distinction: built-in tools extend the model with predefined capabilities; function calling connects the model to your own application logic and actions.

Related Snipps on Snippset

• What Microsoft Foundry Provides for Building AI Agents — Overview of Foundry capabilities for models, agents, tools, data, security, and monitoring.
• Microsoft Foundry SDK vs. Foundry Tools SDKs — Explains the difference between platform-level Foundry development and specialized AI service SDKs.
• Building Production-Ready AI Agents with Microsoft Agent Framework Course — Covers tools, memory, workflows, orchestration, monitoring, and production-ready AI agents.

A chat application becomes much easier to design once you understand three decisions: which endpoint to use, which API to call, and where conversation state is maintained.

Endpoint & API choices

Key concept: LLMs are inherently stateless. Chat history must therefore be supplied or referenced. With Chat Completions, your application maintains and resends the message array. With Responses, server-side context can be continued by passing the previous response identifier.

Core pattern

response = client.responses.create(
    model="my-deployment",
    instructions="You are a helpful assistant.",
    input=user_input,
    previous_response_id=last_response_id
)

print(response.output_text)
last_response_id = response.id

For authentication, prefer Microsoft Entra ID over embedded API keys. DefaultAzureCredential is useful because the same application code can obtain credentials across local development and Azure-hosted environments.

Also know the configuration controls: temperature influences response variability, while max tokens constrains output size.

For applications performing other I/O while waiting for model responses, use the asynchronous client + await to avoid blocking execution.

Remember: responses.create() generates a response; response.output_text retrieves its text; response.id can connect the next turn.

Choosing a model is more than picking the most capable option. In Microsoft Foundry, the practical lifecycle is Select → Deploy → Evaluate: balance model capability, performance, cost, data-location requirements, and measured output quality.

1. Select the model

Use the Model Catalog and Leaderboard to narrow candidates by capabilities, supported languages, context window, fine-tuning support, and benchmarks.

Key trade-off: a larger model may deliver better results, while a smaller model can offer higher throughput and lower cost.

2. Choose the deployment

Know these deployment choices:

• Global → broadest capacity and potentially highest throughput

• Data Zone → processing stays within a geographic zone such as the EU or US

• Regional → maximum control over the processing region, but capacity is constrained to it

• Standard → usage/token-based; suitable for general workloads

• Provisioned → predictable, guaranteed throughput

• Batch → high-volume, non-interactive processing where latency is less important

• Developer → lightweight testing of fine-tuned models

Remember: Global Standard is the general-purpose choice highlighted for obtaining the largest available quota.

3. Evaluate before production

Model performance isn't just speed. Evaluate quality, relevance, fluency, and groundedness.

Manual evaluation: run representative prompts and edge cases against models side-by-side.

Automated evaluation: use a larger prompt dataset, expected behavior, and evaluators to systematically score responses and safety.

High-value distinctions to remember:
Throughput = speed/capacity of responses · Fluency = natural, linguistically correct output · Groundedness = whether responses align with known information

Mental model:
Requirements → Catalog/Benchmarks → Deployment → Test Dataset → Evaluators → Compare → Improve

An LLM can identify entities or PII itself—but an agent can instead delegate these tasks to a specialized Azure Language tool through MCP. This separates agent reasoning from deterministic NLP processing.

Architecture: Prompt → Agent → discover/select MCP tool → Azure Language → tool result → final response

What to know

Azure Language MCP Server exposes Azure Language capabilities as tools that an agent can dynamically discover and invoke. Core capabilities include PII detection, language detection, and Named Entity Recognition (NER); additional Language capabilities are also exposed through MCP.

The important distinction is:

• Agent/LLM: reasons about the request and chooses an appropriate tool.

• MCP: standardizes tool discovery and invocation.

• Azure Language: performs the specialized NLP operation.

Tool selection is not hard-coded. The MCP server advertises available tools and their descriptions; the agent matches the user's intent to those descriptions. Good agent instructions further guide when Azure Language should be used.

Minimal mental model

# Agent is already configured with Azure Language MCP
response = openai_client.responses.create(
    input="Find and redact PII in this text...",
    extra_body={"agent": {"name": "text-agent"}}
)

print(response.output_text)

Behind this simple call:

Agent
 └─ discovers MCP tools
     └─ selects PII tool
         └─ Azure Language analyzes text
             └─ result returns to Agent

Watch for approval: MCP tool calls can require user/application approval. Either handle the approval request in code or configure appropriate tools for automatic approval.

Remember: MCP exposes tools; the agent selects them; Azure Language executes the NLP task.

Before building an AI application or agent, understand how Microsoft Foundry organizes models, tools, knowledge, and development resources. These relationships form the foundation for everything that follows.

Foundry architecture at a glance

Think of the structure as:

Foundry Resource → Project → Models + Agents + Tools + Knowledge

The Foundry resource is the underlying Azure resource and infrastructure boundary. A project lives within that resource and organizes the models, agents, tools, and knowledge used by an AI solution. The resource is the foundation; the project is the development workspace.

Microsoft Foundry provides access to generative models alongside Foundry Tools for specialized AI capabilities such as Language, Speech, Translation, and Document Intelligence. These services complement models when an application needs capabilities such as speech recognition or structured information extraction.

Know which SDK fits

Key distinction: use the OpenAI SDK when targeting a model directly; move toward the Foundry SDK when working with the broader agentic platform, including tools and grounding.

For development, Visual Studio Code with the Microsoft AI Toolkit is the recommended combination presented in the course. In the Foundry portal, Discover is primarily for finding models, tools and templates, while Build is where deployed resources are configured and tested.

Remember the six Responsible AI principles

Fairness • Reliability & Safety • Privacy & Security • Inclusiveness • Transparency • Accountability

These are not an afterthought: they influence grounding, prompts, guardrails, UX, evaluation, and ongoing operation of the solution.

Memory model: Resource hosts → Project organizes → Model reasons → Knowledge grounds → Tools act → Responsible AI governs.

Related Snipps

• What Microsoft Foundry Provides for Building AI Agents — Explains models, tools, knowledge, security, monitoring, and managed agent capabilities.
• Building Production-Ready AI Agents with Microsoft — Covers agents, tools, orchestration, monitoring, and production-oriented architecture.
• Choosing the Right Platform for Building AI Agents — Places Foundry and Azure within the broader range of agent-development approaches.

Building an AI solution goes beyond calling a model. The focus is on creating production-ready AI applications and agents with Microsoft Foundry that can use enterprise data, interact with tools, process different content types, and collaborate to complete real tasks.

Core capabilities to know

Exam focus: Understand not just what these capabilities do, but when and why you would use them together in an Azure AI solution.

Think in solution flows

A useful mental model for AI-103 is:

User → AI App/Agent → Model → Data + Tools → Action/Response

For more complex solutions:

User → Orchestrator → Agent A + Agent B + Agent C → Tools/Data → Result

An agent therefore isn't simply a chatbot. It combines a model's reasoning capabilities with instructions, knowledge, and tools so it can perform useful work.

Preparing effectively

The course assumes working knowledge of Python, REST APIs/SDKs, Azure fundamentals, and generative AI concepts. Hands-on practice is important: build applications in Microsoft Foundry, connect models to data, add tools to agents, experiment with multimodal inputs, and create multi-agent workflows.

Key takeaway: Think beyond prompts and models. AI-103 is about assembling the components required for an end-to-end AI solution:

Models → Grounding → Tools → Agents → Orchestration → Production

Related Snipps

• What Microsoft Foundry Provides for Building AI Agents — Core Foundry capabilities for tools, conversations, models, security, storage, and monitoring.
• Microsoft Foundry SDK vs. Foundry Tools SDKs — Understand platform-level and specialized SDKs when developing Foundry solutions.
• Building Production-Ready AI Agents with Microsoft Agent Framework Course — Explores agent architecture, tools, workflows, memory, monitoring, and deployment.