How to Pick the Right AI Model in Microsoft Foundry — and Why Voice Agents Make It Matter

By Jeff, Microsoft Ecosystem

For most of the last few years, choosing an AI model felt like choosing a car you planned to drive for a decade. You evaluated once, committed, and hoped the bet aged well. That model of model selection is already outdated. In the last week alone, Microsoft added the full GPT-6 family from OpenAI and Claude Opus 5.5 from Anthropic to Microsoft Foundry, rolled out native voice-agent support, and shipped tools that turn production traces into better prompts, skills, and model choices.

The message underneath the announcements is clear: the best model for your business will keep changing, and the platform you build on should make that a feature rather than a problem. Foundry is designed as a model-and-harness-agnostic foundation for agents, so teams can adopt better models as they arrive without rebuilding the architecture around a single provider.

Model choice is now a continuous advantage

Model choice used to be a one-time architectural decision. Today it is closer to capacity planning: you revisit it as workloads, models, and costs change. The right model for a coding agent this quarter may not be the right model for research, customer service, or high-volume inference next quarter.

Microsoft Foundry handles that reality in three ways:

1. Broad model access. Foundry brings together frontier models from leading providers, including the recently added GPT-6 Sol, GPT-6 Luna, and Claude Opus 5.5.
2. Evaluation against your data. Run candidate models against your own workloads and compare quality, latency, and cost rather than relying on leaderboard benchmarks.
3. Production-to-improvement loop. Foundry uses production traces to refine instructions, skills, tools, and model choice, so the agent improves after it ships.

This is the hill-climbing approach Microsoft describes: observe, evaluate, optimize, validate, and repeat, with people in control.

Start with the job, not the leaderboard

When you are deciding which model to use, start with the work the agent will actually do:

What the agent does	Prioritize	Ask yourself
Coding or complex reasoning	Quality, context window, tool-use accuracy	Does it follow multi-step instructions and recover from errors?
Customer-facing chat or voice	Latency, cost, safety, voice quality	Can it answer in real time? What happens on a fallback?
Research or document synthesis	Context window, reasoning, citation quality	Can it hold long documents and cite sources accurately?
High-volume inference	Cost, throughput, caching	What is the per-token cost at scale? Can repeated context be cached?
Prototyping	Flexibility, ease of switching	How quickly can I swap models without rewriting connectors?

Foundry makes the last row especially powerful. Because the agent harness is model-agnostic, you can prototype with a fast, inexpensive model, evaluate a frontier model on the same workload, and promote the winner without rebuilding orchestration.

This protects you from two common mistakes: choosing a model because it is famous, and choosing a model because it is cheap. The right choice is the one that delivers the best outcome per dollar for the specific job.

Evaluate before you commit

Benchmarks are useful signals, but they are not your workload. The model that tops a reasoning benchmark may be overkill for a form-filling agent, and the cheapest model may hallucinate on your specific documents. A practical evaluation loop is to capture real traces, build a small golden set, run candidates on the same inputs, score correctness, latency, cost, and satisfaction, and promote the winner to a small audience first.

Foundry lets you run these evaluations against your own data and define success in business terms, making model choice a measured decision rather than a guess.

Voice agents need their own design discipline

Voice is rapidly becoming one of the most important ways people interact with AI agents, but voice is not just chat with a microphone. A voice agent has to manage turn-taking, latency, interruptions, background noise, and fallback gracefully. Foundry Agent Service now supports voice experiences and long-running work on the same foundation as text agents, so you do not need a separate stack for speech.

If you are building a voice agent, keep these five things in mind:

1. Design for turn-taking. Plan how the agent knows when the user has finished speaking, when to interrupt, and when to yield.
2. Keep latency honest. Set latency budgets and test under realistic network conditions.
3. Build graceful fallbacks. When recognition fails, the agent should clarify or offer text without breaking the flow.
4. Use shared context. Tie the voice agent into Microsoft Graph, connectors, and the same policies as your text agents.
5. Monitor real sessions. Review actual traces for barge-ins, false positives, and moments where the user gives up.

Voice is also a great place to apply the model-choice framework. The model that writes beautiful prose may be too slow for live speech, and a faster model may need stricter guardrails. Foundry lets you test that tradeoff with the same tooling.

Governance is what makes choice safe

More model choice is only useful if the organization stays in control. The recent Copilot Managed Runtime announcement fits here: it provides an enterprise-grade platform to run code and agents inside the Microsoft 365 tenant boundary, governed by IT and connected to Microsoft Entra identity, organizational policies, and a central app inventory.

That means a model chosen in Foundry and an app built in Copilot Code or Copilot Studio can run on a consistent runtime with the same sign-in, policies, and compliance surface as the rest of Microsoft 365. Developers keep freedom to choose tools; IT keeps one management model.

For model selection specifically, answer these questions: Which models are approved for which data classifications? Where are prompts and data processed? How are model versions tracked, evaluated, and rolled back? And who can swap a model in production?

Foundry and the Managed Runtime do not replace your AI governance policy, but they give you a consistent place to enforce it.

For Visual Studio developers: bring your model with you

The same week Foundry expanded its model lineup, Visual Studio shipped its September update with a stronger Bring Your Own Model preview. Developers can now connect Visual Studio to a Microsoft Foundry deployment or to other supported providers, including OpenAI, Anthropic, and Ollama, and use the model in the new Agent preview.

This closes the loop between model choice in the cloud and model choice in the IDE. The model you selected for your Foundry agent can be the same model you use while fixing NuGet vulnerabilities, reviewing pull requests, or scaffolding a feature in Visual Studio. The model picker is in Copilot Chat, and unsupported features are marked clearly instead of failing silently.

The September update also added a useful security improvement: when NuGet Audit flags a vulnerable package in the Error List, a sparkle action opens GitHub Copilot Chat in agent mode and asks Copilot to fix it using the NuGet MCP server. The model you chose can now help you close security gaps the moment they surface.

A practical starting plan

If you want to turn these announcements into action this week, try this sequence: audit your current model choices and mark anything older than six months for review; run one Foundry evaluation on a single workload; identify one voice use case and sketch its turn-taking and fallback design; review which models, data sources, and endpoints are approved; and, if you are a developer, try BYOM in Visual Studio's new Agent preview.

None of these steps require a big migration. They build the habit of continuous model choice without disrupting the work already running.

Bottom line

Microsoft Foundry's recent updates make model choice a practical, everyday capability rather than a strategic bet. By bringing more frontier models, native voice agents, and continuous optimization into one foundation, Microsoft gives organizations a way to keep up with AI progress without fragmenting their architecture or losing governance.

The best model for your next agent is the one that fits the job, your data, and your budget. Foundry makes it possible to find that model, evaluate it honestly, and promote it safely. Voice agents raise the stakes on latency and design, but they run on the same foundation. And for developers, Visual Studio's Bring Your Own Model preview means the model you chose in the cloud can travel with you into the IDE.

Start small, measure carefully, and treat model choice as something you improve over time. The tools are ready.