AI Conversations | Getting Started

Shiny.AiConversation is a centralized AI conversation service that combines Microsoft.Extensions.AI chat completions with Shiny.Speech for speech recognition, text-to-speech, wake word detection, and audio feedback — all behind a single IAiConversationService interface.

Building a conversational AI experience in a mobile or web app typically means stitching together many concerns yourself: managing chat client authentication, wiring up speech-to-text and text-to-speech, handling microphone permissions, coordinating state between listening/thinking/responding phases, playing audio cues, persisting message history, and exposing tools for the AI to call. Each of these is a separate library with its own lifecycle and threading model.

Shiny.AiConversation collapses all of that into a single service registration and one interface. By default, it resolves IChatClient directly from your DI container — just register your chat client and the library handles the rest. For advanced scenarios (on-demand authentication, token refresh), you can implement IChatClientProvider. Either way, the library manages the entire flow — from capturing the user’s voice, streaming it through speech recognition, sending the transcript to the chat model, reading the response aloud, and optionally persisting the conversation. The entire flow is observable through a simple Status property so your UI always knows what’s happening.

This means you can go from zero to a fully functional voice-enabled AI assistant with wake word support, audio feedback, and chat history in under 50 lines of registration code — and swap AI backends (OpenAI, Azure, Ollama, GitHub Copilot) without changing your app logic.

GitHub
Downloads

Frameworks

.NET

.NET MAUI

Operating Systems

Android

iOS

Windows

Web

Features

• Access checking — RequestAccess() verifies speech-to-text permissions before starting voice features
• Text chat — send messages via TalkTo() and receive streaming AI responses
• Voice chat — ListenAndTalk() captures speech and sends it to the AI in one call
• Wake word — hands-free activation with StartWakeWord("Hey Copilot") that listens continuously
• Text-to-speech — AI responses can be read aloud automatically
• Audio feedback — configurable sound effects for thinking, responding, ok, cancel, and error states
• Acknowledgement modes — None, AudioBlip, LessWordy, or Full text-to-speech
• Context providers — pluggable IContextProvider visitor pattern for populating an AiContext per request. Each provider’s Apply(AiContext) method receives a mutable context containing system prompts, AI tools, quiet words, speech-to-text options, and text-to-speech options. A built-in ContextProvider handles time-based prompts, acknowledgement-aware voice prompts, and DI-registered AITool instances. Register additional providers to inject custom prompts, tools, or override speech settings.
• Message persistence — optional IMessageStore for chat history storage and retrieval
• AI chat lookup tool — automatically available when an IMessageStore is registered, lets the AI search its own conversation history
• Conversation continuation — AI responses ending with a question automatically keep the microphone open for a reply
• Voice interruption — configurable quiet words (e.g., “stop”, “cancel”) immediately silence TTS; any other speech during TTS interrupts and continues the conversation with the new utterance
• Speech options — configurable SpeechToTextOptions and TextToSpeechOptions (culture, silence timeout, voice, speech rate, etc.)
• State tracking — observable Status property (Idle, Listening, Thinking, Responding)
• Pluggable providers — default resolves IChatClient from DI; optionally implement IChatClientProvider for custom auth (OpenAI, Copilot, Ollama, etc.)
• AOT compatible — fully trimmer-safe with IsAotCompatible enabled
• Cross-platform — works on MAUI (Android, iOS, Windows) and Blazor (Server, WASM)

Architecture

┌─────────────────────────────────┐
│     IAiConversationService      │  ← Your app talks to this
├─────────────────────────────────┤
│  IChatClientProvider            │  ← Default resolves IChatClient from DI
│  ISpeechToTextService           │  ← From Shiny.Speech
│  ITextToSpeechService           │  ← From Shiny.Speech
│  IAudioPlayer                   │  ← From Shiny.Speech
│  IMessageStore?                 │  ← Optional persistence (you implement)
│  IEnumerable<IContextProvider>  │  ← System prompts & tools (pluggable)
└─────────────────────────────────┘

Setup

MAUIBlazor

• NuGet Packages
• Project File
• MauiProgram.cs
• AndroidManifest.xml
• Info.plist
• PrivacyInfo.xcprivacy
• Package.appxmanifest

Shiny.AiConversation

1. Install the NuGet package

   

   dotnet add package Shiny.AiConversation

2. Register a chat client

   The simplest approach is to register an IChatClient in DI — the library resolves it automatically:

   builder.Services.AddChatClient(new OpenAIClient("your-api-key").GetChatClient("gpt-4o").AsIChatClient());

   Or use one of the built-in provider packages:

   Package	Description
   Shiny.AiConversation.OpenAi	Static OpenAI-compatible client (OpenAI, Azure, Ollama)
   Shiny.AiConversation.Maui.GithubCopilot	GitHub Copilot with device code auth (MAUI only)

   // OpenAI
   opts.AddStaticOpenAIChatClient("your-api-key", "https://api.openai.com/v1", "gpt-4o");
   
   // GitHub Copilot (MAUI only — handles auth, token storage, everything)
   opts.AddGithubCopilotChatClient();

   For other backends, implement IChatClientProvider — see Chat Client Provider.

3. Register the service in your app

   MAUI (MauiProgram.cs):

   builder.Services.AddShinyAiConversation(opts =>
   {
       // Optional: add message store, configure settings
   });

   Blazor (Program.cs):

   builder.Services.AddShinyAiConversation(opts =>
   {
       // Optional: add message store, configure settings
   });

4. Add custom system prompts (optional)

   System prompts, tools, and speech settings are supplied by IContextProvider implementations using a visitor pattern. A ContextProvider is auto-registered. Add your own:

   public class MyContextProvider : IContextProvider
   {
       public Task Apply(AiContext context)
       {
           context.SystemPrompts.Add("You are a helpful assistant.");
           // context.Tools.Add(...) to add AI tools
           // context.Acknowledgement is available to adjust behavior per mode
           // context.QuietWords — modify quiet words for voice interruption
           // context.SpeechToTextOptions — set/override speech recognition options
           // context.TextToSpeechOptions — set/override text-to-speech options
           return Task.CompletedTask;
       }
   }
   
   // Register in DI (multiple providers are supported, executed in sequence)
   builder.Services.AddShinyAiConversation(opts =>
   {
       opts.AddContextProvider<MyContextProvider>();
   });

5. Use the service

   public class ChatViewModel(IAiConversationService aiService)
   {
       public async Task Send(string message)
       {
           aiService.AiResponded += response =>
           {
               if (response.Response.Text is { } text)
                   Console.WriteLine($"AI: {text}");
           };
   
           await aiService.TalkTo(message, CancellationToken.None);
       }
   }

Quick Example

// Check access before using voice features
var access = await aiService.RequestAccess();
if (access != AccessState.Available)
    return; // Speech is not available

// Text chat
await aiService.TalkTo("What is .NET MAUI?", cancellationToken);

// Voice chat (push-to-talk)
await aiService.ListenAndTalk(cancellationToken);

// Hands-free wake word
await aiService.StartWakeWord("Hey Copilot");
// ... user says "Hey Copilot, what's the weather?"
// Service automatically captures and sends to AI
aiService.StopWakeWord();

// Chat history (requires IMessageStore)
var history = await aiService.GetChatHistory(limit: 50);
await aiService.ClearChatHistory();

// Acknowledgement modes
aiService.Acknowledgement = AiAcknowledgement.Full;      // Read response aloud
aiService.Acknowledgement = AiAcknowledgement.LessWordy;  // Concise TTS
aiService.Acknowledgement = AiAcknowledgement.AudioBlip;   // Sound effects only
aiService.Acknowledgement = AiAcknowledgement.None;        // Silent

Platform Notes

Android Speech Recognition

Android’s native speech recognition engine works in short recording segments — it automatically stops listening after a period of silence and must restart for the next segment. This creates brief pauses during continuous listening (wake word, ListenAndTalk) that are noticeable compared to iOS, which streams continuously.

If smooth, uninterrupted speech recognition is important for your app, consider using Azure Speech Services via Shiny.Speech. Azure provides continuous streaming recognition on all platforms, eliminating the start/stop behavior on Android.

builder.Services.AddAzureSpeech("your-subscription-key", "eastus");

Platform Permissions

Since AI Conversations uses Shiny.Speech for voice features, you must declare speech and microphone permissions.

Android — Add to AndroidManifest.xml:

<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />

MODIFY_AUDIO_SETTINGS is required for the TTS audio-level Visualizer and for the native STT beep suppression.

iOS — Add to Info.plist:

<key>NSSpeechRecognitionUsageDescription</key>
<string>This app uses speech recognition</string>
<key>NSMicrophoneUsageDescription</key>
<string>This app uses the microphone for speech recognition</string>

Windows — Add the Microphone capability to your Package.appxmanifest:

<Capabilities>
    <DeviceCapability Name="microphone" />
</Capabilities>

Dependencies

All required dependencies are pulled in transitively by the Shiny.AiConversation package — you only need to install the one package.

Package	Purpose
Microsoft.Extensions.AI	IChatClient abstraction for AI chat completions
Shiny.Speech	Speech-to-text, text-to-speech, and audio playback

Samples

• Sample MAUI App — Full chat app with GitHub Copilot authentication, settings, and aura visualization
• Sample Blazor WASM App — Blazor WebAssembly app with static OpenAI provider

AI Coding Assistant

Claude Code GitHub Copilot

Step 1 — Add the marketplace:

claude plugin marketplace add shinyorg/skills

Step 2 — Install plugins:

claude plugin install shiny-client@shiny

BLE, GPS, Jobs, Notifications, Push, HTTP Transfers, OBD, Music, Health, DataSync — iOS, Android, Windows, MacOS, Linux, Web

claude plugin install shiny-maui@shiny

Shell, Contact Store

claude plugin install controls@shiny

TableView, BottomSheet, PillView, ImageViewer, Scheduler, Markdown, Mermaid Diagrams — MAUI and Blazor

claude plugin install shiny-mediator@shiny

Mediator/CQRS with middleware and source generators

claude plugin install shiny-data@shiny

DocumentDB and Spatial data libraries

claude plugin install shiny-aspire@shiny

Orleans and Gluetun Aspire integrations

claude plugin install shiny-extensions@shiny

DI, Stores, Reflector, Localization, Hosting modules

Step 1 — Add the marketplace:

copilot plugin marketplace add https://github.com/shinyorg/skills

Step 2 — Install plugins:

copilot plugin install shiny-client@shiny

BLE, GPS, Jobs, Notifications, Push, HTTP Transfers, OBD, Music, Health, DataSync — iOS, Android, Windows, MacOS, Linux, Web

copilot plugin install shiny-maui@shiny

Shell, Contact Store

copilot plugin install controls@shiny

TableView, BottomSheet, PillView, ImageViewer, Scheduler, Markdown, Mermaid Diagrams — MAUI and Blazor

copilot plugin install shiny-mediator@shiny

Mediator/CQRS with middleware and source generators

copilot plugin install shiny-data@shiny

DocumentDB and Spatial data libraries

copilot plugin install shiny-aspire@shiny

Orleans and Gluetun Aspire integrations

copilot plugin install shiny-extensions@shiny

DI, Stores, Reflector, Localization, Hosting modules

View Skills Repository

Next Steps

• Chat Client Provider — How to implement IChatClientProvider for different AI backends
• Message Store — Persist and query chat history
• Acknowledgements & Sound — Configure audio feedback and text-to-speech modes
• Wake Word — Hands-free voice activation
• AI Tools — Register AI tools including the built-in chat history lookup