Shiny.Maui.Shell v6 support for AI routing tools Learn More

Getting Started


GitHub
Core
Cloud
Azure
ElevenLabs

Frameworks

.NET

.NET MAUI

Operating Systems

Android

iOS

Windows

Web

Shiny.Speech provides a unified API for speech-to-text, text-to-speech, audio capture, and audio playback across Android, iOS, Windows, and Browser (Blazor WebAssembly) — with pluggable cloud providers for Azure AI Speech and ElevenLabs.

Features

Platform-native speech-to-text with streaming results via IAsyncEnumerable
Platform-native text-to-speech with voice selection, rate, pitch, and volume control
Raw audio capture from the device microphone (16kHz, 16-bit, mono PCM)
Audio playback for MP3 streams
Pluggable cloud provider architecture — swap between on-device and cloud STT/TTS
Azure AI Speech integration (STT + TTS) with SSML prosody control
ElevenLabs integration (TTS) with multilingual voice support
Permission management via AccessState and RequestAccess()
Listen-until-silence dictation mode for simple transcription scenarios
Continuous recognition mode for real-time streaming transcription

Packages

Package	Purpose
`Shiny.Speech`	Core library — platform-native STT, TTS, audio capture, and playback
`Shiny.Speech.Cloud`	Cloud provider abstractions (included transitively by Azure/ElevenLabs)
`Shiny.Speech.Azure`	Azure AI Speech provider (STT + TTS)
`Shiny.Speech.ElevenLabs`	ElevenLabs TTS provider

Setup

Shiny.Speech

Platform Permissions

Android — Add to AndroidManifest.xml:

<uses-permission android:name="android.permission.RECORD_AUDIO" />

iOS — Add to Info.plist:

<key>NSSpeechRecognitionUsageDescription</key>
<string>This app uses speech recognition</string>
<key>NSMicrophoneUsageDescription</key>
<string>This app uses the microphone for speech recognition</string>

Browser (Blazor WebAssembly) — No manifest changes needed. The browser prompts for microphone access automatically. Include the JS interop module in index.html:

<script src="shiny-speech.js"></script>

IAudioSource (raw PCM capture) is not supported in the browser. The Web Speech API handles audio internally.

Quick Example

public class MyViewModel
{
    readonly ISpeechToTextService _stt;
    readonly ITextToSpeechService _tts;

    public MyViewModel(ISpeechToTextService stt, ITextToSpeechService tts)
    {
        _stt = stt;
        _tts = tts;
    }

    async Task ListenAndRespond(CancellationToken ct)
    {
        // 1. Request permission
        var access = await _stt.RequestAccess();
        if (access != AccessState.Available)
            return;

        // 2. Listen until the user stops speaking
        var text = await _stt.ListenUntilSilence(
            new SpeechRecognitionOptions
            {
                Culture = CultureInfo.GetCultureInfo("en-US"),
                SilenceTimeout = TimeSpan.FromSeconds(3)
            },
            ct
        );

        if (text != null)
        {
            // 3. Speak the result back
            await _tts.SpeakAsync($"You said: {text}");
        }
    }

    async Task StreamRecognition(CancellationToken ct)
    {
        // Continuous recognition with streaming results
        await foreach (var result in _stt.ContinuousRecognize(cancellationToken: ct))
        {
            Console.WriteLine($"{result.Text} (final: {result.IsFinal}, confidence: {result.Confidence})");
        }
    }
}

Samples

AI Coding Assistant

Step 1 — Add the marketplace:

claude plugin marketplace add shinyorg/skills

Step 2 — Install plugins:

claude plugin install shiny-client@shiny

BLE, GPS, Jobs, Notifications, Push, HTTP Transfers, OBD, Music, Health, DataSync — iOS, Android, Windows, MacOS, Linux, Web

claude plugin install shiny-maui@shiny

Shell, Contact Store

claude plugin install shiny-controls@shiny

TableView, BottomSheet, PillView, ImageViewer, Scheduler, Markdown, Mermaid Diagrams — MAUI and Blazor

claude plugin install shiny-mediator@shiny

Mediator/CQRS with middleware and source generators

claude plugin install shiny-data@shiny

DocumentDB and Spatial data libraries

claude plugin install shiny-aspire@shiny

Orleans and Gluetun Aspire integrations

claude plugin install shiny-extensions@shiny

DI, Stores, Reflector, Localization, Hosting modules

Step 1 — Add the marketplace:

copilot plugin marketplace add https://github.com/shinyorg/skills

Step 2 — Install plugins:

copilot plugin install shiny-client@shiny

BLE, GPS, Jobs, Notifications, Push, HTTP Transfers, OBD, Music, Health, DataSync — iOS, Android, Windows, MacOS, Linux, Web

copilot plugin install shiny-maui@shiny

Shell, Contact Store

copilot plugin install shiny-controls@shiny

TableView, BottomSheet, PillView, ImageViewer, Scheduler, Markdown, Mermaid Diagrams — MAUI and Blazor

copilot plugin install shiny-mediator@shiny