Microsoft.Extensions.AI

Every AI library has its own way to call a model: OpenAI’s SDK, Azure’s SDK, Ollama’s client — each with different interfaces, different types, different patterns. Microsoft.Extensions.AI solves this by providing a unified abstraction layer that works with any AI provider, the same way ILogger unified logging in .NET.

This guide covers the core abstractions, shows practical code with Azure OpenAI, and demonstrates how the middleware pipeline lets you add caching, logging, and rate limiting without changing your application code.

The problem it solves

Consider a .NET application that uses Azure OpenAI today. Your code is tightly coupled to the Azure SDK — if you want to add local Ollama support for development or switch to another provider, you’re rewriting every call site. Unit testing means mocking provider-specific types.

Microsoft.Extensions.AI introduces provider-agnostic interfaces that any AI library can implement. Your application codes against the interface; the provider is a configuration decision.

Core interfaces

The library defines two primary interfaces:

Both interfaces ship in the Microsoft.Extensions.AI.Abstractions package — a lightweight, dependency-free package that library authors reference. Application developers reference the provider-specific packages which include the implementations.

Getting started

dotnet new console -n AISdkDemo
cd AISdkDemo
dotnet add package Microsoft.Extensions.AI.OpenAI
dotnet add package Azure.AI.OpenAI

Basic chat completion

using Azure.AI.OpenAI;
using Microsoft.Extensions.AI;

var azureClient = new AzureOpenAIClient(
    new Uri("https://your-resource.openai.azure.com/"),
    new DefaultAzureCredential());

IChatClient client = azureClient
    .GetChatClient("gpt-4o")
    .AsIChatClient();

var response = await client.CompleteAsync("Explain dependency injection in 3 sentences.");
Console.WriteLine(response.Message.Text);

The key line is .AsIChatClient() — this extension method wraps the Azure-specific client into the generic IChatClient interface. From here on, your code only knows about IChatClient.

Streaming responses

await foreach (var update in client.CompleteStreamingAsync("Write a haiku about Azure."))
{
    Console.Write(update.Text);
}

Structured conversations

For multi-turn conversations, build a list of ChatMessage objects:

var messages = new List<ChatMessage>
{
    new(ChatRole.System, """
        You are a .NET architecture advisor.
        Give concise, opinionated recommendations.
        Cite specific NuGet packages when relevant.
        """),
    new(ChatRole.User, "Should I use MediatR or just call services directly?")
};

var response = await client.CompleteAsync(messages);
Console.WriteLine(response.Message.Text);

// Continue the conversation
messages.Add(response.Message);
messages.Add(new(ChatRole.User, "What about in a modular monolith?"));

var followUp = await client.CompleteAsync(messages);

Tool calling (function calling)

The AI SDK supports tool calling through the AIFunction abstraction. Define tools as regular methods and let the SDK handle serialization:

using System.ComponentModel;
using Microsoft.Extensions.AI;

public static class WeatherTools
{
    [Description("Gets the current weather for a city")]
    public static string GetWeather(
        [Description("City name, e.g. Seattle")] string city)
    {
        // In production, call a real weather API
        return $"Weather in {city}: 72°F, partly cloudy";
    }

    [Description("Gets the 5-day forecast for a city")]
    public static string GetForecast(
        [Description("City name")] string city,
        [Description("Number of days (1-5)")] int days = 5)
    {
        return $"{days}-day forecast for {city}: sunny, then rain Thursday";
    }
}

Wire the tools into the chat options:

var tools = new[]
{
    AIFunctionFactory.Create(WeatherTools.GetWeather),
    AIFunctionFactory.Create(WeatherTools.GetForecast)
};

var options = new ChatOptions
{
    Tools = tools,
    ToolMode = ChatToolMode.Auto
};

var response = await client.CompleteAsync(
    "What's the weather like in Seattle and will it rain this week?",
    options);

Console.WriteLine(response.Message.Text);
// The model calls both GetWeather and GetForecast, then composes a natural answer

The middleware pipeline

One of the most powerful features is the ability to compose middleware around any IChatClient — just like ASP.NET middleware for HTTP requests:

using Microsoft.Extensions.AI;
using Microsoft.Extensions.Caching.Distributed;

IChatClient client = new ChatClientBuilder(azureClient.GetChatClient("gpt-4o").AsIChatClient())
    .UseOpenTelemetry()
    .UseDistributedCache(cache)
    .UseFunctionInvocation()
    .Build();

Each Use* call wraps the client in a delegating handler. The order matters — requests flow left to right, responses right to left. In this example:

• OpenTelemetry records traces and metrics for every call.
• DistributedCache returns cached responses for identical prompts (great for deterministic queries).
• FunctionInvocation handles the tool-call loop, executing your AIFunction tools when the model requests them.

Dependency injection

The SDK integrates naturally with .NET’s DI container — essential for ASP.NET Core and background services:

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddDistributedMemoryCache();

builder.Services.AddChatClient(services =>
{
    var azureClient = new AzureOpenAIClient(
        new Uri(builder.Configuration["AzureOpenAI:Endpoint"]!),
        new DefaultAzureCredential());

    return new ChatClientBuilder(azureClient.GetChatClient("gpt-4o").AsIChatClient())
        .UseOpenTelemetry()
        .UseDistributedCache()
        .UseFunctionInvocation()
        .Build();
});

var app = builder.Build();

// Now inject IChatClient anywhere
app.MapPost("/chat", async (IChatClient chat, ChatRequest req) =>
{
    var response = await chat.CompleteAsync(req.Message);
    return Results.Ok(new { response.Message.Text });
});

Embeddings

The embedding interface follows the same pattern:

IEmbeddingGenerator<string, Embedding<float>> embedder = azureClient
    .GetEmbeddingClient("text-embedding-3-small")
    .AsIEmbeddingGenerator();

var embeddings = await embedder.GenerateAsync(new[]
{
    "Azure OpenAI provides enterprise-grade AI models",
    "Kubernetes orchestrates containerized workloads",
    "Semantic Kernel connects LLMs to your business logic"
});

foreach (var e in embeddings)
{
    Console.WriteLine($"Dimension: {e.Vector.Length}");  // 1536
}

Swapping providers

The real power of the abstraction: switch from Azure to a local Ollama instance for development by changing one line:

// Production: Azure OpenAI
IChatClient client = azureClient.GetChatClient("gpt-4o").AsIChatClient();

// Development: Ollama running locally
IChatClient client = new OllamaChatClient("http://localhost:11434", "llama3.1");

Every downstream consumer that injects IChatClient works identically. Your middleware pipeline (logging, caching) applies to both. No code changes needed.

Microsoft.Extensions.AI vs. Semantic Kernel

They’re complementary, not competing. Use Microsoft.Extensions.AI when you need straightforward AI calls with clean architecture. Use Semantic Kernel when you need an agent that orchestrates multiple tools and reasons across steps. Semantic Kernel is built on top of these same abstractions.

Building a custom middleware

You can write your own middleware by implementing DelegatingChatClient:

public class RateLimitingChatClient : DelegatingChatClient
{
    private readonly SemaphoreSlim _semaphore;

    public RateLimitingChatClient(IChatClient inner, int maxConcurrency)
        : base(inner)
    {
        _semaphore = new SemaphoreSlim(maxConcurrency);
    }

    public override async Task<ChatCompletion> CompleteAsync(
        IList<ChatMessage> messages,
        ChatOptions? options = null,
        CancellationToken ct = default)
    {
        await _semaphore.WaitAsync(ct);
        try
        {
            return await base.CompleteAsync(messages, options, ct);
        }
        finally
        {
            _semaphore.Release();
        }
    }
}

// Use it in the pipeline
IChatClient client = new ChatClientBuilder(innerClient)
    .Use(inner => new RateLimitingChatClient(inner, maxConcurrency: 5))
    .UseOpenTelemetry()
    .Build();

Production patterns

1. Always use DI — register IChatClient as a singleton and inject it. Don’t create clients per-request.
2. Configure via appsettings — put endpoints and model names in configuration so you can swap providers without redeploying.
3. Add OpenTelemetry early — the telemetry middleware captures token usage, latency, and errors. Essential for cost tracking.
4. Cache deterministic calls — if you’re classifying text or extracting structured data, caching saves both latency and money.
5. Handle streaming for UX — use CompleteStreamingAsync in user-facing applications so responses appear incrementally.
6. Set reasonable timeouts — AI calls can take 10-30 seconds. Configure HttpClient timeouts and cancellation tokens.

Next steps

1. Explore the official samples: github.com/dotnet/ai-samples
2. Add AI to an existing API — inject IChatClient into an ASP.NET Core controller and add a summarization or classification endpoint.
3. Build a RAG pipeline — combine IEmbeddingGenerator with a vector database to ground answers in your data.
4. Read the docs: learn.microsoft.com/dotnet/ai