Skip to content
Introducing AI Conversations: Natural Language Interaction for Your Apps! Learn More

Blog

Introducing Shiny.AiConversation — AI Conversation

NuGet package Shiny.AiConversation

Building an AI-powered app today means stitching together a chat client, speech recognition, text-to-speech, audio playback, message persistence, and state management — across platforms, with proper lifecycle handling. That’s a lot of plumbing before you write your first prompt.

Shiny.AiConversation wraps all of that into a single IAiConversationService interface. Text chat, voice chat, hands-free wake word activation, configurable audio feedback, and persistent chat history — registered with one DI call, consumed through one service.


Every AI chat app ends up building the same infrastructure:

  • An authenticated chat client that handles token refresh
  • Speech-to-text so users can talk instead of type
  • Text-to-speech so the AI can respond out loud
  • Sound effects for state transitions (thinking, responding, error)
  • A wake word listener for hands-free mode
  • Message persistence for chat history
  • State management so the UI knows what’s happening
  • Thread safety so nothing blows up

Each of these is a separate library, a separate abstraction, and a separate set of platform quirks. You spend weeks on infrastructure before you ship a single feature.

// Register your chat client in DI
builder.Services.AddChatClient(new OpenAIClient("your-api-key").GetChatClient("gpt-4o").AsIChatClient());
builder.Services.AddShinyAiConversation(opts =>
{
opts.SetMessageStore<MyMessageStore>(); // optional
});

That’s it. The service registers IAiConversationService with all the wiring — speech services from Shiny.Speech, chat completions from Microsoft.Extensions.AI, audio playback, time provider, and optional message persistence. The default IChatClientProvider resolves IChatClient straight from DI, so for most apps you just register your chat client and go. For advanced scenarios (on-demand auth, token refresh), you can still implement IChatClientProvider directly.

The simplest path. Send a message, get a streaming response:

aiService.AiResponded += response =>
{
if (response.Response.Text is { } text)
Console.WriteLine($"AI: {text}");
};
await aiService.TalkTo("What is .NET MAUI?", cancellationToken);

The service handles the full lifecycle — acquires the chat client, prepends system prompts, streams the response, stores both messages if a message store is configured, fires the event, and manages state transitions throughout.

One method call captures speech and sends it to the AI:

await aiService.ListenAndTalk(cancellationToken);

The service activates speech-to-text, waits for the user to stop speaking, sends the transcribed text through TalkTo(), and optionally reads the response aloud via text-to-speech. If the AI responds with a question, the service automatically keeps listening for the user’s reply — creating a natural back-and-forth conversation without requiring another button press.

This is the “Hey Siri” experience:

await aiService.StartWakeWord("Hey Copilot");

The service enters a continuous loop: listen for the wake phrase, capture the utterance that follows, send it to the AI, loop back. If the AI asks a follow-up question, the loop skips wake word detection and listens directly for the user’s reply. The user never touches the screen. Call StopWakeWord() when you’re done.

Control how the AI delivers responses:

ModeWhat Happens
NoneSilent — text only, delivered via the AiResponded event
AudioBlipShort sound effects at each state transition
LessWordyText-to-speech with a “be concise” system prompt
FullFull text-to-speech of the complete response

Sound effects are driven by string file names and a SoundResolver callback — the library stays platform-agnostic while you provide the stream:

aiService.SoundResolver = name => FileSystem.OpenAppPackageFileAsync(name);
aiService.ThinkSound = "think.mp3";
aiService.OkSound = "ok.mp3";

During TTS playback, the service listens for voice interruptions. Say a “quiet word” like “stop” or “cancel” and TTS is silenced immediately, breaking out of the conversation. Say anything else and TTS stops, but your new utterance is sent to the AI as the next message — the conversation continues seamlessly.

// Default quiet words: cancel, quiet, shut up, stop, nevermind, never mind, hush
// Customize or disable:
aiService.QuietWords = ["stop", "cancel"];
aiService.QuietWords = null; // disable interruption

Quiet words only trigger when they’re the user’s entire utterance. “Cancel this appointment” won’t interrupt — it’ll be treated as a new message to the AI.

Register an IMessageStore and every message is automatically persisted. But the interesting part is the AI chat lookup tool — it’s an AITool that lets the AI search its own conversation history:

“What did we talk about yesterday?” “Find the recipe you gave me last week.”

The tool is registered automatically when you call SetMessageStore(). The AI gets search parameters (text, date range, limit) and queries your store directly.

The service exposes its current state and fires events:

aiService.StatusChanged += state =>
{
// state: Idle, Listening, Thinking, Responding
UpdateUI(state);
};

This is what powers the “Aura” visualization in our sample app — a pulsing orb that changes color based on what the AI is doing.

The library doesn’t care which AI you use. By default, it resolves IChatClient from DI — just register one and you’re done. For advanced auth scenarios, implement IChatClientProvider to return any IChatClient from Microsoft.Extensions.AI:

  • OpenAInew OpenAIClient(apiKey).GetChatClient("gpt-4o").AsIChatClient()
  • GitHub Copilot — OAuth device code flow with Copilot API token exchange
  • Azure OpenAI — Managed identity or API key
  • Ollama — Local model, no auth needed
  • Anything else — If it implements IChatClient, it works

The sample apps include a complete GitHub Copilot implementation with device code flow, token caching, automatic re-authentication, and the custom HTTP headers the Copilot API requires.

The library targets plain net10.0 — no MAUI dependency in the library itself. Shiny.Speech handles the platform abstraction for speech and audio, so the same IAiConversationService works on:

  • MAUI — Android, iOS, Windows, Mac Catalyst
  • Blazor — Server-side and WebAssembly (speech via Web Audio API)

We ship two sample apps that prove it: a full MAUI sample with chat, settings, and an animated aura visualization, plus a Blazor sample with the same features translated to Razor components and CSS animations.

The library is built with IsAotCompatible=true. Generic type parameters on SetChatClientProvider<T>() and SetMessageStore<T>() carry [DynamicallyAccessedMembers] attributes so the trimmer knows what to keep. No reflection surprises at runtime.

Terminal window
dotnet add package Shiny.AiConversation

The library is MIT licensed and open source. We’d love to hear what you build with it.

The Feedback Service — One Hook to Rule Them All

NuGet package Shiny.Maui.Controls

Every tap, swipe, and keystroke in your app is an opportunity. An opportunity to confirm the user’s action, guide their attention, or add a layer of polish that separates “functional” from “delightful.” Most apps handle this with scattered HapticFeedback.Default.Perform() calls sprinkled across code-behind files. It works — until you want text-to-speech for accessibility, sound effects for a kiosk app, analytics for product telemetry, or different feedback for different controls. Then you’re threading conditional logic through every view in your app.

Shiny Controls v1.0 ships with IFeedbackService — a single injectable service that every interactive control in the library already calls. You implement it once. Every control uses it automatically.


Every Shiny control that supports feedback has a UseFeedback property (default: true). When a user interaction occurs — a message sent, a pin digit entered, a panel opened — the control calls IFeedbackService.OnRequested() with three things:

public interface IFeedbackService
{
void OnRequested(object control, string eventName, object? args = null);
}
  • control — the actual control instance, not a Type. Pattern match directly: control is ChatView, control is SecurityPin.
  • eventName — what happened: "MessageReceived", "DigitEntered", "Opened".
  • args — contextual data. For ChatView, this is the full ChatMessage object. For standard MAUI controls, it’s the native EventArgs. For SecurityPin completion, it’s "LongPress".

The default HapticFeedbackService does what you’d expect — click haptic for most events, long press haptic for completion events. But the real power is in replacing it.

Here’s a real example from our sample app. One service, three behaviors — haptic, text-to-speech for incoming chat messages, and audio cues for PIN entry:

public class MyCustomFeedbackService(
ITextToSpeechService textToSpeech,
IAudioManager audioManager
) : HapticFeedbackService
{
public override async void OnRequested(object control, string eventName, object? args)
{
// haptic first — always
base.OnRequested(control, eventName, args);
// speak incoming chat messages aloud
if (control is ChatView && args is ChatMessage { IsFromMe: false } msg)
{
await textToSpeech.SpeakAsync(
$"Message from {msg.SenderId}. {msg.Text}"
);
}
// click and success sounds for PIN entry
else if (control is SecurityPin)
{
var sound = eventName.Equals("completed", StringComparison.OrdinalIgnoreCase)
? "pin_success.wav"
: "pin_click.wav";
var raw = await FileSystem.OpenAppPackageFileAsync(sound);
audioManager.CreatePlayer(raw).Play();
}
}
}

Register it in one line:

builder.UseShinyControls(cfg =>
{
cfg.SetCustomFeedback<MyCustomFeedbackService>();
});

Because control is the live instance and args carries typed data, you can make nuanced decisions without parsing strings. The ChatMessage gives you sender, timestamp, text, and image URL. The SecurityPin instance gives you its current value and length. Cast, match, and go.

Shiny’s own controls call IFeedbackService internally. But what about standard MAUI controls — Button, Slider, Entry? The MauiControlFeedbackBuilder hooks them in automatically, with an AOT-compatible, fully pluggable design:

cfg.AddDefaultMauiControlFeedback();

This registers hooks for 12 standard MAUI controls — Button.Clicked, Entry.TextChanged, Slider.ValueChanged, Switch.Toggled, and more. Each hook passes the control instance as control and the native event args as args.

cfg.AddDefaultMauiControlFeedback(x =>
{
x.Hook<MyCustomControl>(nameof(MyCustomControl.Tapped),
(c, h) => c.Tapped += h,
(c, h) => c.Tapped -= h);
});
cfg.AddMauiControlFeedback(x =>
{
x.Hook<Button>(nameof(Button.Clicked),
(btn, h) => btn.Clicked += h,
(btn, h) => btn.Clicked -= h);
x.Hook<Slider, ValueChangedEventArgs>(nameof(Slider.ValueChanged),
(s, h) => s.ValueChanged += h,
(s, h) => s.ValueChanged -= h);
});

Two overloads cover every case:

  • Hook<TControl>(eventName, subscribe, unsubscribe) for plain EventHandler events
  • Hook<TControl, TEventArgs>(eventName, subscribe, unsubscribe) for typed EventHandler<TEventArgs> events

Under the hood, each hook uses a ConditionalWeakTable to track handlers per control instance — no leaks, no dictionaries to manage, proper unsubscription when controls leave the visual tree. Zero reflection, fully AOT-safe.

Every Shiny control fires feedback through this system. Here’s the full event catalog:

ControlEvents
ChatViewMessageSent, MessageReceived, MessageTapped (all pass ChatMessage), AttachImage
SecurityPinDigitEntered, Completed
FloatingPanelOpened, Closed, DetentChanged
ImageViewerOpened, Closed, DoubleTapped
ImageEditorToolModeChanged, Undo, Redo, Rotate, Reset, CropApplied, Saved
Fab / FabMenuClicked, Toggled
SchedulerDaySelected, EventSelected, TimeSlotSelected
TableView CellsTapped
ToastShow

Any control’s feedback can be suppressed per-instance with UseFeedback="False".

Most feedback systems are either too simple (a global haptic toggle) or too complex (per-control event subscriptions scattered across your app). IFeedbackService sits in the sweet spot:

  1. One service, all controls. Implement once, every control calls it.
  2. Instance, not type. You get the actual control, not typeof(Button). Inspect properties, check state, make decisions.
  3. Typed args, not strings. ChatMessage, ValueChangedEventArgs, ToggledEventArgs — not "the message text".
  4. Pluggable hooks, not hardcoded events. Add your own controls to the system with three lambdas.
  5. AOT-safe. No reflection, no expressions, no Delegate.CreateDelegate. Just generics and delegates.

Whether you’re building an accessible app that speaks every incoming message, a kiosk that plays sound effects, or just want consistent haptic feedback across your entire UI — IFeedbackService is one implementation away.

Check out the full documentation and the sample app for a working demo with TTS and audio integration.

Turn Any Interface Into an AI Tool — Shiny DI 3.0

What if every service interface you already have could become an AI tool with a single attribute? Shiny Extensions DI 3.0 makes that happen — no adapter classes, no hand-rolled schemas, no registration boilerplate. Mark your interface with [Tool], add [Description] to the methods that matter, and the source generator handles the rest.

You’ve built your services. Clean interfaces, proper DI registration, everything wired up. Now someone asks you to expose a few of those operations as AI tools for an LLM agent. Suddenly you’re writing AIFunction subclasses by hand — one per operation — each with a constructor that takes the service, a metadata property with hand-written parameter schemas, and an InvokeCoreAsync override that extracts arguments from a dictionary and forwards them to your service method.

For one or two tools, it’s fine. For ten or twenty, it’s tedious. And every time you change a method signature, you have to remember to update the corresponding tool class. The schema drifts, the argument parsing breaks, and the bugs only show up when the LLM calls the tool at runtime.

[Tool]
[Description("Manages customer orders")]
public interface IOrderService
{
[Description("Places a new order for a customer")]
Task<OrderResult> PlaceOrderAsync(
[Description("The customer identifier")] Guid customerId,
[Description("The product SKU")] string sku,
[Description("Number of units to order")] int quantity
);
[Description("Cancels an existing order")]
Task CancelOrderAsync(
[Description("The order to cancel")] Guid orderId,
[Description("Reason for cancellation")] string reason
);
// No [Description] — not exposed as a tool
Task<List<Order>> GetInternalAuditLogAsync();
}

That’s it. The source generator produces a fully typed AIFunction subclass for each described method, wires up the parameter metadata, and generates a registration extension — all at compile time.

For PlaceOrderAsync above, the generator emits a class like this:

public class IOrderServicePlaceOrderAsyncAITool : AIFunction
{
private readonly IOrderService _service;
private static readonly AIFunctionMetadata _metadata =
new AIFunctionMetadata("IOrderServicePlaceOrderAsync")
{
Description = "Places a new order for a customer",
Parameters = new AIFunctionParameterMetadata[]
{
new("customerId")
{
Description = "The customer identifier",
ParameterType = typeof(Guid),
IsRequired = true
},
new("sku")
{
Description = "The product SKU",
ParameterType = typeof(string),
IsRequired = true
},
new("quantity")
{
Description = "Number of units to order",
ParameterType = typeof(int),
IsRequired = true
}
}
};
public Guid CustomerId { get; set; }
public string Sku { get; set; }
public int Quantity { get; set; }
public IOrderServicePlaceOrderAsyncAITool(IOrderService service)
{
_service = service;
}
public override AIFunctionMetadata Metadata => _metadata;
protected override async Task<object?> InvokeCoreAsync(
IEnumerable<KeyValuePair<string, object?>>? arguments,
CancellationToken cancellationToken)
{
// argument extraction and service call
return await _service.PlaceOrderAsync(
this.CustomerId, this.Sku, this.Quantity);
}
}

A second class is generated for CancelOrderAsync. The GetInternalAuditLogAsync method is skipped because it has no [Description].

All generated tools are registered with a single call:

services.AddGeneratedAITools();

This registers each tool as Transient<AITool, GeneratedToolClass>. You can then resolve all tools and pass them to any IChatClient:

var tools = serviceProvider.GetServices<AITool>().ToList();
var options = new ChatOptions { Tools = tools };
var response = await chatClient.GetResponseAsync(messages, options);

The AI tool code is only generated when Microsoft.Extensions.AI is referenced in your project. If you don’t reference it, the [Tool] attribute still compiles (it’s just an attribute), but no AIFunction classes or registration code are emitted. This means existing projects that add the DI package won’t get unexpected dependencies.

The generated InvokeCoreAsync handles the JsonElement-vs-already-deserialized argument problem that trips up most hand-written AI tools. For every standard type, the generator emits a direct JsonElement accessor:

TypeExtractionReflection-free
stringGetString()Yes
int, long, short, byteGetInt32(), GetInt64(), etc.Yes
boolGetBoolean()Yes
double, float, decimalGetDouble(), GetSingle(), GetDecimal()Yes
GuidGetGuid()Yes
DateTimeGetDateTime()Yes
DateTimeOffsetGetDateTimeOffset()Yes
DateOnly, TimeOnly, TimeSpanParse(GetString())Yes
EnumsEnum.Parse<T>(GetString())Yes
Complex typesJsonSerializer.Deserialize<T>()Needs JsonSerializerContext

If the argument arrives as a JsonElement (common when the framework hasn’t pre-deserialized), the correct accessor is used. If it arrives already typed (some frameworks do this), a direct cast is used. Both paths are handled with a single is JsonElement check — no try/catch, no Convert.ChangeType.

If your service method accepts a CancellationToken, the generator does the right thing automatically:

[Description("Searches products")]
Task<List<Product>> SearchAsync(
[Description("Search query")] string query,
CancellationToken cancellationToken // not exposed as a tool parameter
);

The CancellationToken is excluded from the tool’s parameter metadata and properties. In InvokeCoreAsync, it’s passed through from the framework’s cancellation token — not extracted from the argument dictionary.

Only methods with [Description] become tools. This gives you fine-grained control over what’s exposed to the LLM. Internal methods, admin operations, or anything you don’t want an AI agent calling — just don’t add the attribute.

The [Tool] attribute goes on interfaces, while [Singleton] / [Scoped] / [Transient] go on implementation classes — same as before. You keep using AddGeneratedServices() for your service registrations and add AddGeneratedAITools() alongside it:

services.AddGeneratedServices();
services.AddGeneratedAITools(); // only if M.E.AI is referenced

The two generators are independent. AI tool generation doesn’t affect or depend on your service registrations.

  1. Add [Tool] to the interface
  2. Add [Description] to the interface and the methods you want exposed
  3. Add [Description] to parameters (optional but recommended — it helps the LLM)
  4. Reference Microsoft.Extensions.AI in your project
  5. Call services.AddGeneratedAITools() at startup
  6. Resolve IEnumerable<AITool> and pass to your chat client

Check the DI documentation for the full setup guide and the release notes for the complete changelog.