Audio (Monitor & Devices)

Shiny.Audio is the standalone audio backbone under Shiny.Speech — usable on its own for recording and playback without the speech stack. Alongside capture (IAudioSource) and playback (IAudioPlayer) — both with normalized VU metering — it provides a live microphone monitor (IAudioMonitor) and audio route enumeration/selection (IAudioDevices), all discoverable through a single IAudio facade.

Registration

builder
    .UseMauiApp<App>()
    .UseShiny();                       // required on Android for RECORD_AUDIO permission routing

builder.Services.AddAudioServices();   // IAudioPlayer + IAudioSource + IAudioMonitor + IAudioDevices + IAudio

AddSpeechServices() already registers the capture/playback pieces; call AddAudioServices() when you also want the monitor/devices/facade. The individual registrations still exist (AddAudioSource(), AddAudioPlayer(), AddAudioMonitor(), AddAudioDevices()), and every focused interface remains injectable directly — the facade doesn’t replace them.

The IAudio facade

Inject one service to discover the whole surface. Each member is the same service you could inject directly; lifetimes are preserved (player/monitor/devices singletons, IAudioSource a fresh transient per access).

using Shiny.Audio;

public class AudioViewModel(IAudio audio)
{
    Task Play()        => audio.Player.PlayAsync("https://example.com/clip.mp3");
    Task StartMic()    => audio.Monitor.Start();
    var  Outputs       => audio.Devices.GetOutputs();
}

Monitor and Devices are implemented on iOS / Mac Catalyst and Android only; accessing them on other platforms throws PlatformNotSupportedException.

Microphone monitor (IAudioMonitor)

Routes the microphone straight to the current audio output in near-real-time — a PA / megaphone. Ideal for “talk into the phone, hear it over a Bluetooth speaker”.

using Shiny.Audio;

public partial class MicViewModel(IAudio audio) : ObservableObject
{
    [ObservableProperty] double level;   // bind to a VU bar

    public async Task Toggle()
    {
        var monitor = audio.Monitor;
        if (monitor.IsMonitoring)
        {
            await monitor.Stop();
            return;
        }

        if (await monitor.RequestAccess() != AccessState.Available)
            return;

        monitor.InputLevelChanged += (_, l) =>
            MainThread.BeginInvokeOnMainThread(() => Level = l);

        // No Processing → routes to a Bluetooth A2DP speaker (see the trade-off below).
        await monitor.Start(new AudioMonitorOptions { Gain = 1.0 });
    }
}

The output goes to the current route — the **built-in speaker, a wired output, or a connected Bluetooth speaker (A2DP) using the phone’s own mic. Gain (0.0–1.0) is adjustable live; InputLevelChanged emits a normalized level for a meter.

Bluetooth speaker or echo cancellation — not both, on iOS

Enabling AudioProcessingOptions (echo cancellation) engages iOS’s voice-processing unit, which forces Bluetooth onto the low-quality HFP call profile. A Bluetooth speaker (A2DP, no mic) can’t provide HFP, so audio falls back to the phone. So: leave processing off to reach a Bluetooth speaker; turn it on only when outputting to the phone speaker up close and feedback is a problem.

Feedback

When output is the phone speaker, the mic can hear it and howl — keep distance, lower Gain, or enable echo cancellation. Always stop the monitor when the page is dismissed and when the app backgrounds.

Implementation: iOS/Mac Catalyst use AVAudioEngine (input node → main mixer → output) in a PlayAndRecord session (no DefaultToSpeaker, so it follows a Bluetooth/wired route; Default mode unless processing is requested); the engine rebuilds on route changes so monitoring follows a newly-connected output. Android uses AudioRecord → AudioTrack. The audio session is snapshotted and restored on Stop.

AirPlay (HomePod / Apple TV) is not supported for a live mic — iOS only permits AirPlay for playback, not while recording. Use Bluetooth for a wireless live PA.

Capturing for a model, not a listener

If the captured audio feeds a model — speaker recognition/verification, wake words, anything that produces an embedding — capture it with AudioProcessingOptions.Analysis:

var pcm = await source.StartCaptureAsync(AudioProcessingOptions.Analysis, ct);

That is no effects plus AllowBluetooth = false, and both halves matter:

• AEC / noise suppression / AGC are adaptive and non-linear. They exist to normalize away speaker and channel characteristics — exactly what a speaker embedding encodes. Because they adapt per session, two recordings of one person come out different.
• A Bluetooth mic runs over HFP at 8 kHz narrowband. Fricatives (/s/, /ʃ/, /f/) live above that, so the bandwidth you actually record depends on what happens to be paired.

On iOS/Mac Catalyst, requesting no effects puts the session in Measurement mode (minimum system input processing) rather than VoiceChat; AllowBluetooth = false additionally drops the Bluetooth options from the session category. Android’s raw mic source and Windows capture never route to a Bluetooth mic implicitly, so Analysis is a no-op there.

Capture enrollment and matching the same way

The embedding encodes the channel as well as the voice. A template recorded through different capture settings than the probe carries a systematic offset that no matching threshold can correct.

None of this applies to transcription — for STT, the effects are usually what you want.

Microphone metering (IAudioSource)

Capture raises InputLevelChanged with a normalized 0.0–1.0 level while it streams PCM — a mic VU meter for a record button, a “we’re hearing you” indicator, or a silence detector. Unlike the playback meter it needs no capability check: it’s computed from the captured PCM itself, so it works on every platform, including the browser.

using Shiny.Audio;

public partial class RecorderViewModel(IAudio audio) : ObservableObject
{
    [ObservableProperty] double level;   // bind to a VU bar

    public async Task Record(CancellationToken ct)
    {
        var source = audio.Source;
        if (await source.RequestAccess() != AccessState.Available)
            return;

        source.InputLevelChanged += (_, l) =>
            MainThread.BeginInvokeOnMainThread(() => Level = l);

        var pcm = await source.StartCaptureAsync(cancellationToken: ct);
        // consume the 16 kHz / 16-bit / mono stream...
        await source.StopCaptureAsync();
        Level = 0;
    }
}

Events are raised on the capture thread and throttled to roughly 20/sec (peak-held in between), so a bound bar stays smooth without flooding the UI thread. If you’d rather meter the PCM yourself — say you’re buffering it elsewhere — the same mapping is public:

var level = AudioLevel.FromPcm16(buffer.AsSpan(0, bytesRead));   // 0.0 - 1.0

AudioLevel maps linear RMS to dBFS against a -50 dB noise floor. Every meter in the library goes through it, so mic and playback bars read on the same scale.

Audio devices (IAudioDevices)

Its most reliable use is displaying where audio is flowing and reacting to route changes — e.g. “Output: JBL Flip · Bluetooth”. Read CurrentInput / CurrentOutput (each an AudioDevice with a normalized Type) and subscribe to Changed:

using Shiny.Audio;

var devices = audio.Devices;

string input  = devices.CurrentInput?.Name  ?? "—";   // .Type: BuiltInMic, Bluetooth, …
string output = devices.CurrentOutput?.Name ?? "—";   // .Type: BuiltInSpeaker, BluetoothA2dp, …

devices.Changed += (_, _) => Refresh();   // route added / removed / switched

Each AudioDevice carries Id, Name, Io (input/output), a normalized Type (BuiltInMic, BluetoothA2dp, WiredHeadphones, BuiltInSpeaker, …) and IsCurrent.

Classifying the route — wired, Bluetooth, or built-in

AudioDeviceType covers wired routes as well as Bluetooth: WiredHeadphones (output only) and WiredHeadset (output plus a mic). Both map on each platform — iOS from AVAudioSession.PortHeadphones / PortHeadsetMic, Android from WiredHeadphones / WiredHeadset.

Rather than matching every enum member by hand, use the classification helpers. Each exists on both AudioDevice and AudioDeviceType:

using Shiny.Audio;

var output = audio.Devices.CurrentOutput;

if (output?.IsWired() == true)      { /* 3.5mm jack, Lightning, or USB-C */ }
if (output?.IsBluetooth() == true)  { /* HFP/SCO or A2DP */ }
if (output?.IsBuiltIn() == true)    { /* phone speaker or earpiece — nothing attached */ }

// "Is audio private to the user?" — e.g. before speaking a TTS reply out loud.
if (output?.IsHeadphones() == true)
    await audio.Player.PlayAsync(replyUrl);

// Will capture come from the accessory, or fall back to the phone's own mic?
bool accessoryMic = output?.HasMicrophone() == true;

Helper	True for
IsWired()	WiredHeadphones, WiredHeadset, Usb
IsBluetooth()	Bluetooth (HFP/SCO), BluetoothA2dp
IsBuiltIn()	BuiltInMic, BuiltInSpeaker, BuiltInReceiver
IsHeadphones()	WiredHeadphones, WiredHeadset, Bluetooth, BluetoothA2dp
HasMicrophone()	WiredHeadset, Bluetooth

IsWired() includes USB on purpose

On handsets with no 3.5mm jack, the wired option is USB-C — and neither platform reports those as a Wired* type (Android says UsbHeadset/UsbDevice, iOS says PortUsbAudio). Including Usb is what makes “are wired headphones plugged in?” answer correctly on modern hardware. The trade-off is that a USB audio interface or DAC also matches; use IsHeadphones() when you specifically mean something worn on the head, accepting that it misses USB-C earbuds.

HasMicrophone() excludes Usb for the opposite reason — neither platform reveals whether a USB route has a mic from the output device alone. Check GetInputs() for a matching Usb entry instead.

Bluetooth cannot be narrowed further. A paired A2DP route is equally a set of earbuds or a room speaker, and no platform API distinguishes them — so IsHeadphones() treats all Bluetooth as private.

Combine with Changed to react to plug/unplug — it fires on both platforms (iOS route-change notification, Android device callback).

Selection (GetInputs/GetOutputs + IAudioMonitor.SetInputDevice/SetOutputDevice) is a full selector on Android only:

await audio.Monitor.SetInputDevice(chosenInput);    // Android + iOS
await audio.Monitor.SetOutputDevice(chosenOutput);  // Android only (best-effort elsewhere)

Device selection is platform-limited — treat routes as display on iOS

Apple deliberately does not let apps enumerate or freely pick output routes. On iOS, GetOutputs() returns only the current output, SetOutputDevice can only toggle the built-in speaker, and AirPlay/Bluetooth output selection is owned by the system route picker (AVRoutePickerView) — there is no API to route to a specific HomePod/speaker programmatically. Input is selectable on iOS (AVAudioSession.SetPreferredInput). Android enumerates and selects both input and output fully. So: use CurrentInput/CurrentOutput as a display property everywhere; use the selectors on Android.

Volume (IAudioPlayer)

IAudioPlayer exposes the device media volume — the level the hardware buttons control, normalized 0.0–1.0 and independent of any per-request TTS Volume. Reading works on every platform; setting is platform-limited — always guard a set with IsVolumeControlSupported.

using Shiny.Audio;

public partial class VolumeViewModel(IAudio audio) : ObservableObject
{
    [ObservableProperty] float volume;          // bind to a Slider
    public bool CanSet => audio.Player.IsVolumeControlSupported;

    public VolumeViewModel(IAudio audio) : this(audio)
    {
        Volume = audio.Player.Volume;           // reading works everywhere

        audio.Player.VolumeChanged += (_, v) => // hardware buttons, OS UI, or a successful set
            MainThread.BeginInvokeOnMainThread(() => Volume = v);
    }

    partial void OnVolumeChanged(float value)
    {
        if (audio.Player.IsVolumeControlSupported)   // iOS / Mac Catalyst setter throws
            audio.Player.Volume = value;
    }
}

Platform	Read	Set	VolumeChanged source	Backing API
Android	✅	✅	system settings observer	AudioManager STREAM_MUSIC (integer-stepped → set values quantize)
Windows	✅	✅	endpoint volume callback	WASAPI IAudioEndpointVolume (default render endpoint)
macOS	✅	✅ †	CoreAudio property listener	HAL virtual main volume of the default output device
iOS / Mac Catalyst	✅	❌	KVO on outputVolume	AVAudioSession.OutputVolume (read-only)
Browser (WASM)	✅	✅	echoed on set	HTMLAudioElement.volume — app-local, not the OS volume

Setting the volume is not universal

On iOS / Mac Catalyst Apple exposes no supported API to change the system volume, so the setter throws NotSupportedException — let the user adjust it with the hardware buttons or an MPVolumeView. † macOS is settable when the current default output device exposes a settable virtual main volume (most built-in / USB devices do; some HDMI / aggregate devices don’t), reported by IsVolumeControlSupported. In the browser the OS volume is sandboxed, so Volume is the app’s own media-element volume (settable, and it persists across plays). Marshal VolumeChanged to the UI thread before touching bound properties.

Linux

Linux support ships in a separate Shiny.Audio.Linux package. It implements all four services over PulseAudio/PipeWire, falling back to ALSA where no sound server is running — headless servers, minimal containers, Raspberry Pi images.

It is a separate package because it carries a managed MP3 decoder (NLayer): Linux has no system media decoder to hand a stream to, unlike MediaPlayer on Windows/Android or AVAudioPlayer on Apple.

dotnet add package Shiny.Audio.Linux

using Shiny;

// Registers IAudioSource + IAudioPlayer + IAudioDevices + IAudioMonitor + IAudio.
// A no-op when not running on Linux, so it is safe in shared startup code.
builder.Services.AddLinuxAudio();

Call it first

AddLinuxAudio() must come before AddAudioServices(), AddSpeechServices() or any AddCloudSpeechToText / AddCloudTextToSpeech overload. Those all use TryAdd, so the first registration wins — and on Linux they register nothing at all.

There is no native speech on Linux

Linux has no OS speech engine to wrap the way iOS, Android and Windows do, so AddSpeechToText() and AddTextToSpeech() register nothing there. Pair AddLinuxAudio() with any cloud provider — they are pure HTTP over the PCM stream and work unchanged:

builder.Services.AddLinuxAudio();
builder.Services.AddAzureSpeech("subscription-key", "region");

Behaviour notes

• No resampler ships in the library. Streams open at exactly the format required — 16 kHz mono for capture, a decoded clip’s own rate for playback. PulseAudio converts server-side; ALSA’s default device converts through its plug plugin.
• Playback decodes MP3 and WAV only. That covers what Azure, OpenAI and ElevenLabs return by default. Any other container throws NotSupportedException naming the format.
• Playback metering works here (unlike Windows/Browser) because the player decodes to PCM in managed code and sees every sample.
• Voice processing is best-effort. PulseAudio publishes echo cancellation as a virtual source from module-echo-cancel; when that module is loaded, AudioProcessingOptions.EchoCancellation selects it, otherwise capture is raw.
• No permission prompt. Microphone access on Linux is a filesystem/group concern, so RequestAccess() returns Available whenever a backend is present.
• ShowOutputPicker() is a no-op — there is no system route picker equivalent to iOS’s AVRoutePickerView; routing lives in the desktop environment’s sound settings. IAudioDevices.Changed fires on PulseAudio/PipeWire only, since ALSA has no change notification.

Runtime dependencies

The distro’s own audio libraries — libpulse-simple.so.0 (usually the libpulse0 package) or libasound.so.2 (usually libasound2). In a container, install one of them explicitly.

Probe at startup when the app must degrade gracefully rather than throw on first use:

if (!LinuxAudioServiceCollectionExtensions.IsLinuxAudioAvailable)
    logger.LogWarning("No Linux audio backend found — voice features disabled");

Platform support

Capability	iOS / Mac Catalyst	Android	Windows	Browser	Linux
IAudioPlayer / IAudioSource	✅	✅	✅	✅	✅
IAudioSource.InputLevelChanged (mic VU)	✅	✅	✅	✅	✅
IAudioPlayer.AudioLevelChanged (playback VU)	✅	✅	❌	❌	✅
IAudioPlayer.Volume (read)	✅	✅	✅	✅	✅
IAudioPlayer.Volume (set)	❌ ¹	✅	✅	✅	✅ ²
IAudioMonitor (live monitor)	✅	✅	❌	❌	✅
IAudioDevices (routes)	✅	✅	❌	❌	✅

¹ Native macOS (not Mac Catalyst) can set the volume via CoreAudio — see the Volume section.

² Linux volume is settable on PulseAudio/PipeWire (the default sink’s volume). On the ALSA fallback IsVolumeControlSupported is false and the setter throws — reading returns the last value set in-process rather than a real device level. Guard the set, exactly as on iOS.