Structured Turns & Questions

A voice conversation needs to know one thing that a chat UI does not: is the AI waiting on me? If it is, the listener stays open and the user answers without saying the wake word again. If it isn’t, the microphone should close.

IChatClient has no way to tell you that. ChatFinishReason is only Stop / Length / ToolCalls / ContentFilter, and the InputRequestContent hierarchy in Microsoft.Extensions.AI covers tool approvals, not conversational questions. The only signal in the box is the wording of the reply — and “does it end in a question mark” gets a voice assistant wrong often enough to be annoying.

So the conversation service asks the model for a structured turn instead.

The shape

public record AiTurn(string Reply, AiQuestion[]? Questions = null);
public record AiQuestion(string Id, string Text, AiChoice[]? Choices = null, bool AllowMultiple = false);
public record AiChoice(string Id, string Label);

Reply is what the user sees and hears. Questions drives the interface. The model still phrases the question naturally inside Reply — nothing reads the structured question text aloud, and nothing synthesizes speech from the choice list. Trying to speak the structure gets you “Question one of two. Option A…”; letting the model write the prose gets you “Do you want the 9am or the 2pm?”.

Reading a turn

aiService.AiResponded += response =>
{
    // Always use AiResponse.Text - Response.Text is the raw JSON envelope
    var text = response.Text;

    if (response.ExpectsResponse)
    {
        foreach (var question in response.Questions)
        {
            Console.WriteLine(question.Text);

            foreach (var choice in question.Choices ?? [])
                Console.WriteLine($"  - {choice.Label}");
        }
    }
};

Caution

AiResponse.Response.Text is the raw JSON when structured output is on. Render and speak AiResponse.Text, which is the parsed reply (or the raw text when the turn was unstructured).

The question queue

IAiConversationService.PendingQuestions holds what the AI is currently waiting on. Every turn replaces the queue. The model is the source of truth for what it still needs, so anything it stops asking about is treated as answered:

Turn 1  →  Questions: [time?, party size?]      PendingQuestions = [time?, party size?]
user answers "9am"
Turn 2  →  Questions: [party size?]             PendingQuestions = [party size?]
user answers "four"
Turn 3  →  Questions: []                        PendingQuestions = []   → listener closes

Don’t keep a parallel queue and don’t pop questions locally — send the answer and let the model re-ask whatever is still outstanding. It stays correct even when one utterance answers two questions at once.

Answers go back as ordinary text through TalkTo. Don’t fuzzy-match a spoken answer against the choices — the model already has the question and its options in context and resolves “uh, the afternoon one” itself. Local matching only applies to a tapped button, which sends choice.Label verbatim.

The follow-up window

While a turn carries questions, the microphone stays open for the answer without the wake word. FollowUpTimeout (default 20 seconds) bounds that wait:

aiService.FollowUpTimeout = TimeSpan.FromSeconds(30);
aiService.FollowUpTimeout = null;   // wait indefinitely

If nobody answers before it expires, PendingQuestions is cleared and the conversation goes back to requiring the wake word. That bound matters: an abandoned question with no timeout leaves the mic hot, so the next unrelated thing said in the room gets sent to the AI as the answer.

In ListenAndTalk the first listen is unbounded — the user tapped the microphone, so they get as long as they need. Only the AI-initiated follow-ups time out.

Voice asks one question at a time

The service adapts the contract prompt to how the reply is being delivered:

Acknowledgement	Instruction to the model
Above AudioBlip (spoken aloud)	Ask at most one question per turn
None / AudioBlip (text only)	May ask several when they’re genuinely independent

Three questions in one breath renders fine as chips and is unusable as audio.

Provider support

Endpoints disagree about what they’ll accept, so IChatClientProvider.StructuredOutputMode declares the best mode for each one. It’s a default interface member — custom providers only override it when the default is wrong.

Mode	Request	Use for
JsonSchema (default)	Native schema-constrained response format	OpenAI and compatible endpoints
Json	JSON response format, shape described in the prompt	GitHub Copilot, models without schema support
Prompt	Shape in the prompt only, no format constraint	Endpoints that reject both of the above
None	Plain text — opts out entirely	Providers that can’t do any of it

public class MyChatClientProvider : IChatClientProvider
{
    public Task<IChatClient> GetChatClient(CancellationToken cancelToken = default) => ...;

    public AiStructuredOutputMode StructuredOutputMode => AiStructuredOutputMode.Json;
}

Both GitHub Copilot providers default to Json, because the Copilot proxy passes json_schema support through per-model and rejects it on several. Override the provider’s choice per app:

aiService.StructuredOutputMode = AiStructuredOutputMode.JsonSchema;

It always degrades, never breaks

Every path falls back to plain text rather than surfacing an error:

• Parsing tolerates markdown fences and surrounding prose
• A request the endpoint rejects outright is retried without the response format
• On any failure AiResponse.Turn is null, AiResponse.Text is the raw reply, and ExpectsResponse falls back to checking whether the reply ends in a question

So null-check Turn, and treat PendingQuestions as possibly empty even when ExpectsResponse is true — a model that ignored the contract still gets you a working conversation, just without the chips.

Opting out

aiService.StructuredOutputMode = AiStructuredOutputMode.None;

Replies become plain text, ExpectsResponse falls back to the wording heuristic, PendingQuestions stays empty, and choice buttons never render.

Multiple choice in the UI

AiChatView renders a button per AiChoice under the bubble — see MAUI Chat UI for ShowChoiceButtons, multi-select, and how to render choices inside your own message template.