Skip to content
Shiny.NET
Commercial SupportGuaranteed response times on Shiny bug fixes, straight from the maintainers. Plans from US$1,500 a year.See the plans

Audio (Monitor & Devices)

Shiny.Audio is the standalone audio backbone under Shiny.Speech — usable on its own for recording and playback without the speech stack. Alongside capture (IAudioSource) and concurrent playback (IAudioPlayer) — both with normalized VU metering — it provides a live microphone monitor (IAudioMonitor) and audio route enumeration/selection (IAudioDevices), all discoverable through a single IAudio facade.

builder
.UseMauiApp<App>()
.UseShiny(); // required on Android for RECORD_AUDIO permission routing
builder.Services.AddAudioServices(); // IAudioPlayer + IAudioSource + IAudioMonitor + IAudioDevices + IAudioRecorder + IAudioDuplex + IAudio

AddSpeechServices() already registers the capture/playback pieces; call AddAudioServices() when you also want the monitor/devices/facade. The individual registrations still exist (AddAudioSource(), AddAudioPlayer(), AddAudioMonitor(), AddAudioDevices(), AddAudioDuplex()), and every focused interface remains injectable directly — the facade doesn’t replace them.

Inject one service to discover the whole surface. Each member is the same service you could inject directly; lifetimes are preserved (player/monitor/devices singletons, IAudioSource a fresh transient per access).

using Shiny.Audio;
public class AudioViewModel(IAudio audio)
{
Task Play() => audio.Player.PlayAsync("https://example.com/clip.mp3");
Task StartMic() => audio.Monitor.Start();
var Outputs => audio.Devices.GetOutputs();
}

Monitor and Devices are implemented on iOS / Mac Catalyst and Android only, and Duplex on iOS, Mac Catalyst, macOS and Android; accessing them on other platforms throws PlatformNotSupportedException.

Routes the microphone straight to the current audio output in near-real-time — a PA / megaphone. Ideal for “talk into the phone, hear it over a Bluetooth speaker”.

using Shiny.Audio;
public partial class MicViewModel(IAudio audio) : ObservableObject
{
[ObservableProperty] double level; // bind to a VU bar
public async Task Toggle()
{
var monitor = audio.Monitor;
if (monitor.IsMonitoring)
{
await monitor.Stop();
return;
}
if (await monitor.RequestAccess() != AccessState.Available)
return;
monitor.InputLevelChanged += (_, l) =>
MainThread.BeginInvokeOnMainThread(() => Level = l);
// No Processing → routes to a Bluetooth A2DP speaker (see the trade-off below).
await monitor.Start(new AudioMonitorOptions { Gain = 1.0 });
}
}

The output goes to the current route — the **built-in speaker, a wired output, or a connected Bluetooth speaker (A2DP) using the phone’s own mic. Gain (0.0–1.0) is adjustable live; InputLevelChanged emits a normalized level for a meter.

Implementation: iOS/Mac Catalyst use AVAudioEngine (input node → main mixer → output) in a PlayAndRecord session (no DefaultToSpeaker, so it follows a Bluetooth/wired route; Default mode unless processing is requested); the engine rebuilds on route changes so monitoring follows a newly-connected output. Android uses AudioRecord → AudioTrack. The audio session is snapshotted and restored on Stop.

AirPlay (HomePod / Apple TV) is not supported for a live mic — iOS only permits AirPlay for playback, not while recording. Use Bluetooth for a wireless live PA.

If the captured audio feeds a model — speaker recognition/verification, wake words, anything that produces an embedding — capture it with AudioProcessingOptions.Analysis:

var pcm = await source.StartCaptureAsync(AudioProcessingOptions.Analysis, ct);

That is no effects plus AllowBluetooth = false, and both halves matter:

  • AEC / noise suppression / AGC are adaptive and non-linear. They exist to normalize away speaker and channel characteristics — exactly what a speaker embedding encodes. Because they adapt per session, two recordings of one person come out different.
  • A Bluetooth mic runs over HFP at 8 kHz narrowband. Fricatives (/s/, /ʃ/, /f/) live above that, so the bandwidth you actually record depends on what happens to be paired.

On iOS/Mac Catalyst, requesting no effects puts the session in Measurement mode (minimum system input processing) rather than VoiceChat; AllowBluetooth = false additionally drops the Bluetooth options from the session category. Android’s raw mic source and Windows capture never route to a Bluetooth mic implicitly, so Analysis is a no-op there.

None of this applies to transcription — for STT, the effects are usually what you want.

Capture raises InputLevelChanged with a normalized 0.0–1.0 level while it streams PCM — a mic VU meter for a record button, a “we’re hearing you” indicator, or a silence detector. Unlike the playback meter it needs no capability check: it’s computed from the captured PCM itself, so it works on every platform, including the browser.

using Shiny.Audio;
public partial class RecorderViewModel(IAudio audio) : ObservableObject
{
[ObservableProperty] double level; // bind to a VU bar
public async Task Record(CancellationToken ct)
{
var source = audio.Source;
if (await source.RequestAccess() != AccessState.Available)
return;
source.InputLevelChanged += (_, l) =>
MainThread.BeginInvokeOnMainThread(() => Level = l);
var pcm = await source.StartCaptureAsync(cancellationToken: ct);
// consume the 16 kHz / 16-bit / mono stream...
await source.StopCaptureAsync();
Level = 0;
}
}

Events are raised on the capture thread and throttled to roughly 20/sec (peak-held in between), so a bound bar stays smooth without flooding the UI thread. If you’d rather meter the PCM yourself — say you’re buffering it elsewhere — the same mapping is public:

var level = AudioLevel.FromPcm16(buffer.AsSpan(0, bytesRead)); // 0.0 - 1.0

AudioLevel maps linear RMS to dBFS against a -50 dB noise floor. Every meter in the library goes through it, so mic and playback bars read on the same scale.

Its most reliable use is displaying where audio is flowing and reacting to route changes — e.g. “Output: JBL Flip · Bluetooth”. Read CurrentInput / CurrentOutput (each an AudioDevice with a normalized Type) and subscribe to Changed:

using Shiny.Audio;
var devices = audio.Devices;
string input = devices.CurrentInput?.Name ?? "—"; // .Type: BuiltInMic, Bluetooth, …
string output = devices.CurrentOutput?.Name ?? "—"; // .Type: BuiltInSpeaker, BluetoothA2dp, …
devices.Changed += (_, _) => Refresh(); // route added / removed / switched

Each AudioDevice carries Id, Name, Io (input/output), a normalized Type (BuiltInMic, BluetoothA2dp, WiredHeadphones, BuiltInSpeaker, …) and IsCurrent.

Classifying the route — wired, Bluetooth, or built-in

Section titled “Classifying the route — wired, Bluetooth, or built-in”

AudioDeviceType covers wired routes as well as Bluetooth: WiredHeadphones (output only) and WiredHeadset (output plus a mic). Both map on each platform — iOS from AVAudioSession.PortHeadphones / PortHeadsetMic, Android from WiredHeadphones / WiredHeadset.

Rather than matching every enum member by hand, use the classification helpers. Each exists on both AudioDevice and AudioDeviceType:

using Shiny.Audio;
var output = audio.Devices.CurrentOutput;
if (output?.IsWired() == true) { /* 3.5mm jack, Lightning, or USB-C */ }
if (output?.IsBluetooth() == true) { /* HFP/SCO or A2DP */ }
if (output?.IsBuiltIn() == true) { /* phone speaker or earpiece — nothing attached */ }
// "Is audio private to the user?" — e.g. before speaking a TTS reply out loud.
if (output?.IsHeadphones() == true)
await audio.Player.PlayAsync(replyUrl);
// Will capture come from the accessory, or fall back to the phone's own mic?
bool accessoryMic = output?.HasMicrophone() == true;
Helper True for
IsWired() WiredHeadphones, WiredHeadset, Usb
IsBluetooth() Bluetooth (HFP/SCO), BluetoothA2dp
IsBuiltIn() BuiltInMic, BuiltInSpeaker, BuiltInReceiver
IsHeadphones() WiredHeadphones, WiredHeadset, Bluetooth, BluetoothA2dp
HasMicrophone() WiredHeadset, Bluetooth

Combine with Changed to react to plug/unplug — it fires on both platforms (iOS route-change notification, Android device callback).

Selection (GetInputs/GetOutputs + IAudioMonitor.SetInputDevice/SetOutputDevice) is a full selector on Android only:

await audio.Monitor.SetInputDevice(chosenInput); // Android + iOS
await audio.Monitor.SetOutputDevice(chosenOutput); // Android only (best-effort elsewhere)

Concurrent playback (IAudioPlayer / IAudioPlayback)

Section titled “Concurrent playback (IAudioPlayer / IAudioPlayback)”

Clips play concurrently. Starting one does not stop the others, so background music, a sound effect and a voice line can overlap — and each one is still cancellable on its own.

StartAsync returns an IAudioPlayback handle as soon as the clip is playing; PlayAsync is the same thing plus a wait on that handle’s Completion, so it waits for its clip without touching anything else.

using Shiny.Audio;
public class SoundboardViewModel(IAudioPlayer player)
{
IAudioPlayback? music;
public async Task StartMusic(CancellationToken ct)
=> this.music = await player.StartAsync("https://example.com/theme.mp3", ct);
// Overlaps the music rather than interrupting it
public Task Ping(CancellationToken ct)
=> player.PlayAsync(Path.Combine(FileSystem.AppDataDirectory, "ping.mp3"), ct);
public Task StopMusic() => this.music?.StopAsync() ?? Task.CompletedTask; // effects play on
public Task StopEverything() => player.StopAsync();
}
Member Meaning
IAudioPlayer.StartAsync(stream | source, ct) Start a clip, return its IAudioPlayback
IAudioPlayer.PlayAsync(stream | source, ct) StartAsync + await Completion — waits for this clip only
IAudioPlayer.Active Everything currently playing, oldest first
IAudioPlayer.StopAsync() Stops every clip this player started
IAudioPlayback.StopAsync() Stops one clip; the rest keep playing
IAudioPlayback.Completion Completes when that clip ends, stops, or is cancelled
IAudioPlayback.Id / Source / IsPlaying Identity, the URL/path (null for a stream), live state

A CancellationToken passed to StartAsync/PlayAsync stops that clip only, and completes it normally — Completion does not throw OperationCanceledException. IAudioPlayback is IAsyncDisposable, so disposing a handle stops its clip.

AudioLevelChanged stays a single stream per player: while clips overlap it reports the loudest of them, which is what a VU meter should show.

IAudioPlayer exposes the device media volume — the level the hardware buttons control, normalized 0.0–1.0 and independent of any per-request TTS Volume. Reading works on every platform; setting is platform-limited — always guard a set with IsVolumeControlSupported.

using Shiny.Audio;
public partial class VolumeViewModel(IAudio audio) : ObservableObject
{
[ObservableProperty] float volume; // bind to a Slider
public bool CanSet => audio.Player.IsVolumeControlSupported;
public VolumeViewModel(IAudio audio) : this(audio)
{
Volume = audio.Player.Volume; // reading works everywhere
audio.Player.VolumeChanged += (_, v) => // hardware buttons, OS UI, or a successful set
MainThread.BeginInvokeOnMainThread(() => Volume = v);
}
partial void OnVolumeChanged(float value)
{
if (audio.Player.IsVolumeControlSupported) // iOS / Mac Catalyst setter throws
audio.Player.Volume = value;
}
}
Platform Read Set VolumeChanged source Backing API
Android ✅ ✅ system settings observer AudioManager STREAM_MUSIC (integer-stepped → set values quantize)
Windows ✅ ✅ endpoint volume callback WASAPI IAudioEndpointVolume (default render endpoint)
macOS ✅ ✅ † CoreAudio property listener HAL virtual main volume of the default output device
iOS / Mac Catalyst ✅ ❌ KVO on outputVolume AVAudioSession.OutputVolume (read-only)
Browser (WASM) ✅ ✅ echoed on set HTMLAudioElement.volume — app-local, not the OS volume

Full-duplex voice for calls (IAudioDuplex)

Section titled “Full-duplex voice for calls (IAudioDuplex)”

IAudioDuplex runs microphone capture and live PCM playback on one voice-processing session, so the platform’s echo canceller removes what the speaker plays from what the microphone hears. It’s the audio half of a phone / FaceTime-style call, where PCM frames arrive from the network every ~20 ms and must play straight away.

using Shiny;
using Shiny.Audio;
public class CallAudio(IAudioDuplex duplex) // or IAudio.Duplex
{
IAudioDuplexSession? session;
public async Task Start(Func<ReadOnlyMemory<byte>, CancellationToken, ValueTask> send, CancellationToken ct)
{
if (await duplex.RequestAccess() != AccessState.Available)
return;
session = await duplex.StartAsync(new AudioDuplexOptions
{
SampleRate = 16000, // or 8000
Speakerphone = true // loudspeaker rather than the earpiece
}, ct);
// Mic → network: one 20 ms PCM16 mono frame per item, already echo-cancelled.
_ = Task.Run(async () =>
{
await foreach (var frame in session.Captured.ReadAllAsync(ct))
await send(frame, ct);
}, ct);
}
// Network → speaker: any thread, any size, never blocks.
public void OnReceived(ReadOnlyMemory<byte> pcm16) => session?.Play(pcm16.Span);
public void ToggleSpeaker() => session!.Speakerphone = !session.Speakerphone; // live
// A mic from IAudioDevices.GetInputs(), or null for the system default. Live.
public void UseMic(AudioDevice? mic) => session!.InputDevice = mic;
public Task Stop() => session?.StopAsync() ?? Task.CompletedTask; // restores the audio session
}
AudioDuplexOptions Default
SampleRate 16000 8000 or 16000, both directions
Processing AudioProcessingOptions.VoiceChat Echo cancellation, noise suppression, AGC. Null / nothing enabled = no echo cancellation
Speakerphone true Loudspeaker rather than the earpiece. A connected headset still wins
InputDevice null The microphone, one of IAudioDevices.GetInputs(). Null is the system default. Changeable live
FrameDuration 20 ms The size of each Captured item: whole milliseconds, 10–100
PlaybackLatency 60 ms The jitter-buffer target
MaxPlaybackLatency 200 ms The most that may queue; must be greater than PlaybackLatency

Play hands audio to a jitter buffer that the output pulls at the hardware rate:

  • Priming. Nothing plays until about PlaybackLatency is queued, so the first late packet doesn’t starve the output straight away.
  • Underruns. When the queue runs dry, whatever there is plays, then silence, and it primes again before playing more. Lost and late packets come out as silence, never as a stall.
  • Overflow. When a burst takes the queue past MaxPlaybackLatency, the oldest audio is dropped back down to PlaybackLatency, so delay can’t build up over a long call.
  • Buffered reports how much is queued.

The other direction is bounded too: Captured holds about a second of frames, and a reader that falls behind loses the oldest. It completes when the session stops.

AudioDuplexOptions.InputDevice picks the mic the call starts on, and IAudioDuplexSession.InputDevice switches it mid-call without stopping capture or playback:

var inputs = audio.Devices.GetInputs();
session.InputDevice = inputs.FirstOrDefault(d => d.Type == AudioDeviceType.Bluetooth); // null = default
session.InputDeviceChanged += (_, _) =>
{
// The chosen mic was unplugged or disconnected: the call is back on the default mic,
// and session.InputDevice is null. Raised on a platform thread.
};
  • Null is the system default. An output device throws ArgumentException. A mic that isn’t connected falls back to the default, and InputDevice then reads null.
  • A headset’s mic and speaker move together on iOS, Mac Catalyst and Android. Choosing a Bluetooth, wired or USB headset mic routes the call’s audio to that headset as well, and Speakerphone has no effect while it’s chosen. Choosing the built-in mic while a headset is connected takes the call off the headset, to the loudspeaker or the earpiece per Speakerphone. With null, a connected headset wins, as before.
  • InputDeviceChanged fires only when the platform changes the mic underneath you, never for your own sets.
  • How long a switch takes is the platform’s. iOS reroutes the session and the engine rebuilds after the route change, a short blip. macOS restarts its voice-processing engine on the new device, a gap of a few tens of milliseconds; queued playback is kept. Android reroutes the running AudioRecord in place.
Platform How it’s built
iOS / Mac Catalyst AVAudioSession PlayAndRecord in VoiceChat mode (DefaultToSpeaker for speakerphone, HFP Bluetooth when AllowBluetooth), one AVAudioEngine with voice processing on its input node, a tap resampled by AVAudioConverter, and an AVAudioPlayerNode fed from the jitter buffer. The engine is rebuilt after a route change. InputDevice is the session’s preferred input (setPreferredInput)
macOS The same engine, without AVAudioSession. Speakerphone has no effect. InputDevice points the voice-processing unit at that CoreAudio device; only the input changes, since macOS output is a separate device
Android AudioRecord (VOICE_COMMUNICATION) and a streaming AudioTrack (USAGE_VOICE_COMMUNICATION, CONTENT_TYPE_SPEECH) on one audio session, with AcousticEchoCanceler, NoiseSuppressor and AutomaticGainControl when the device has them. MODE_IN_COMMUNICATION and transient audio focus while running; speakerphone through setCommunicationDevice (API 31+) or setSpeakerphoneOn. InputDevice is the record’s setPreferredDevice, and it steers the communication device as described above. Below API 31 the session doesn’t start a Bluetooth SCO link, so a Bluetooth headset mic may not be reachable there. Add MODIFY_AUDIO_SETTINGS to the manifest
Windows, Linux, Browser Not available
  • One session at a time. A second StartAsync throws InvalidOperationException until the first stops. Invalid options throw ArgumentException.
  • Stopping restores what the session changed: the audio session category and mode on Apple; the audio mode, communication device and audio focus on Android. Disposing the session stops it.
  • Echo-cancellation quality belongs to the platform and the device. Without processing, use a headset.

Linux support ships in a separate Shiny.Audio.Linux package. It implements all four services over PulseAudio/PipeWire, falling back to ALSA where no sound server is running — headless servers, minimal containers, Raspberry Pi images.

It is a separate package because it carries a managed MP3 decoder (NLayer): Linux has no system media decoder to hand a stream to, unlike MediaPlayer on Windows/Android or AVAudioPlayer on Apple.

Terminal window
dotnet add package Shiny.Audio.Linux
using Shiny;
// Registers IAudioSource + IAudioPlayer + IAudioDevices + IAudioMonitor + IAudio.
// A no-op when not running on Linux, so it is safe in shared startup code.
builder.Services.AddLinuxAudio();

Linux has no OS speech engine to wrap the way iOS, Android and Windows do, so AddSpeechToText() and AddTextToSpeech() register nothing there. Pair AddLinuxAudio() with any cloud provider — they are pure HTTP over the PCM stream and work unchanged:

builder.Services.AddLinuxAudio();
builder.Services.AddAzureSpeech("subscription-key", "region");
  • No resampler ships in the library. Streams open at exactly the format required — 16 kHz mono for capture, a decoded clip’s own rate for playback. PulseAudio converts server-side; ALSA’s default device converts through its plug plugin.
  • Playback decodes MP3 and WAV only. That covers what Azure, OpenAI and ElevenLabs return by default. Any other container throws NotSupportedException naming the format.
  • Playback metering works here (unlike Windows/Browser) because the player decodes to PCM in managed code and sees every sample.
  • Voice processing is best-effort. PulseAudio publishes echo cancellation as a virtual source from module-echo-cancel; when that module is loaded, AudioProcessingOptions.EchoCancellation selects it, otherwise capture is raw.
  • No permission prompt. Microphone access on Linux is a filesystem/group concern, so RequestAccess() returns Available whenever a backend is present.
  • ShowOutputPicker() is a no-op — there is no system route picker equivalent to iOS’s AVRoutePickerView; routing lives in the desktop environment’s sound settings. IAudioDevices.Changed fires on PulseAudio/PipeWire only, since ALSA has no change notification.

The distro’s own audio libraries — libpulse-simple.so.0 (usually the libpulse0 package) or libasound.so.2 (usually libasound2). In a container, install one of them explicitly.

Probe at startup when the app must degrade gracefully rather than throw on first use:

if (!LinuxAudioServiceCollectionExtensions.IsLinuxAudioAvailable)
logger.LogWarning("No Linux audio backend found — voice features disabled");
Capability iOS / Mac Catalyst Android Windows Browser Linux
IAudioPlayer / IAudioSource ✅ ✅ ✅ ✅ ✅
IAudioSource.InputLevelChanged (mic VU) ✅ ✅ ✅ ✅ ✅
IAudioPlayer.AudioLevelChanged (playback VU) ✅ ✅ ❌ ❌ ✅
IAudioPlayer.Volume (read) ✅ ✅ ✅ ✅ ✅
IAudioPlayer.Volume (set) ❌ ¹ ✅ ✅ ✅ ✅ ²
IAudioMonitor (live monitor) ✅ ✅ ❌ ❌ ✅
IAudioDevices (routes) ✅ ✅ ❌ ❌ ✅
IAudioDuplex (call audio) ✅ ³ ✅ ❌ ❌ ❌

¹ Native macOS (not Mac Catalyst) can set the volume via CoreAudio — see the Volume section.

² Linux volume is settable on PulseAudio/PipeWire (the default sink’s volume). On the ALSA fallback IsVolumeControlSupported is false and the setter throws — reading returns the last value set in-process rather than a real device level. Guard the set, exactly as on iOS.

³ Native macOS too.