Three new Microsoft.Extensions.AI packages turn device services into tools an LLM can call — Shiny.Contacts.Extensions.AI, Shiny.Notifications.Extensions.AI, and Shiny.Locations.Extensions.AI. Opt-in, read-only by default, AOT-safe, and they compose into a single IChatClient.
A camera control sounds simple until you ship one. You want live preview, zoom, torch, lens selection, photo and video capture — fine. Then you want to scan a barcode, box a face, read a receipt, parse a driver’s license, apply a live filter, and have it all run on iOS, Android, Windows, macOS AppKit, and Blazor WebAssembly without rewriting the pipeline five times.
That’s what CameraView is: one control, one API surface, every platform — with a pluggable frame-analysis pipeline bolted on.
Terminal window
dotnetaddpackageShiny.Maui.Controls.Camera
builder
.UseShinyControls()
.UseShinyCamera();
xmlns:cam="http://shiny.net/maui/camera"
<cam:CameraViewx:Name="Camera"
Facing="Back"
ScaleMode="AspectFill"
Filter="None" />
The preview auto-starts (IsActive defaults true) and the control requests camera permission itself — you handle a denial (or any error) through CameraError, and toggle IsActive for lifecycle:
this.Camera.CameraError += (_, e) => status = e.Message; // e.g. "Camera permission denied"
protectedoverridevoidOnDisappearing()
{
base.OnDisappearing();
this.Camera.IsActive =false; // release the camera off-screen
}
// JPEG bytes — the current Filter is baked in, so the photo matches the preview
The same CameraView lights up on five hosts, each over the platform’s native stack:
Host
Backend
Apple (iOS / Mac Catalyst)
AVFoundation
macOS (AppKit)
AVFoundation over AppKit
Android
CameraX (min SDK 23)
Windows
Media Capture
Blazor WebAssembly
getUserMedia / MediaRecorder / BarcodeDetector
Facing picks a lens by position (Back / Front / External); CameraId pins an exact device — which is how you choose between multiple back lenses on a phone or a specific USB webcam on macOS:
Set Filter and the look is applied to the live preview and baked into captured photos, so what you see is what you get:
this.Camera.Filter = CameraFilter.Noir;
Eleven filters ship — Mono, Noir, Sepia, Invert, Vivid, Cool, Warm, Fade, Chrome, Instant, Tonal — plus None. A couple of honest platform caveats: recorded video records the unfiltered feed, the Android live-preview filter needs API 31+ (it uses RenderEffect; captured photos are still filtered on older Android), and Windows has no live filter.
This is where CameraView stops being “a camera” and becomes a platform. Assign an IFrameAnalyzer to Camera.Analyzer and the pipeline streams frames off the UI thread with drop-on-busy back-pressure (only one frame in flight — it never backs up). It’s one analyzer at a time, and you set or swap it freely while the camera is running.
Every analyzer has two channels:
A strongly-typed event carrying the semantic result (the barcodes, the faces, the recognized text, the structured document).
The return value of its analysis: styled OverlayBoxes to draw. A returned set persists until the analyzer returns a different set (replace) or null (clear).
The boxes draw live the whole time the analyzer runs; the typed event is gated on a scan. The analyzer stays quiet until you arm it — Camera.Scan(), or bind a shutter button / Fab to Camera.ScanCommand — and then it delivers its next confirmed detection exactly once. Single-shot is the default, which is usually what you want (“point, tap, get the code”). To keep scanning, give the analyzer an OnDetected handler that returns true:
varbarcode=newBarcodeAnalyzer(); // native — Apple Vision (iOS/macOS) / Android MLKit
barcode.BarcodesDetected += (_, e) =>
status =$"{e.First.Format}: {e.First.Value}"; // e.Barcodes holds every code in the frame
Camera.Analyzer = barcode;
Camera.Scan(); // arm — deliver the next decode, once
// continuous: stay armed as long as the handler returns true
Swapping the analyzer is just an assignment — the running session picks it up:
Camera.Analyzer =newFaceAnalyzer(); // Apple Vision / Android MLKit / Windows.FaceAnalysis
Camera.Analyzer =newMotionAnalyzer(); // pure-managed frame differencing — every platform
Events are marshalled to the UI thread for you, so handlers can touch UI directly.
To switch the analyzer off without dropping it (keeping its bindings and internal state), set FrameAnalyzer.IsEnabled = false — it resumes instantly when re-enabled, and while disabled the camera behaves as if it had no analyzer (so on Android you can record video). That’s distinct from ShowBoundingBox = false (run and deliver, just draw nothing).
Analyzers are BindableObjects, and CameraView’s content property is Analyzer, so you can declare it inline under the one cam: prefix and bind its …Command to your ViewModel — fired on the UI thread with the same args as the event. Bind a shutter button to the camera’s ScanCommand to arm it:
OverlayBox.Rect is normalized (0..1), upright, and mirror-corrected — the overlay maps it into view space, so you never deal in raw pixels. Set the analyzer’s ScanWindow to a normalized rectangle to restrict detection to a band: only detections centered inside it are reported and drawn, and the overlay dims everything outside it and frames a viewfinder reticle. Want custom styling? Each analyzer exposes an OverlayProvider to return exactly the boxes you want (or null for none):
MotionAnalyzer is a nice example of the pipeline’s range: it clusters movement into separate regions, so motion in two spots yields two boxes rather than one box spanning both — handy for a security-cam view. Tune it with PixelThreshold / AreaThreshold, SampleStride, and GridColumns / CellThreshold.
Document analyzers — structured data, not just text
The Shiny.Maui.Controls.Camera.Documents package turns the camera into a scanner that hands you strongly-typed records, not raw strings. Every document type is its own analyzer with its own typed DocumentDetected event, and every payload is a record with nullable fields — only what was actually found is set. Like every analyzer, delivery is arm-gated, so call Camera.Scan() (or bind ScanCommand) to capture the next read.
status =$"Invoice {doc.Number} — total {doc.Total}, {doc.Lines.Count} line(s)";
};
Camera.Analyzer = invoice;
Camera.Scan(); // arm — delivers the next parsed invoice
varlicense=newDriversLicenseAnalyzer(); // PDF417 + AAMVA — deterministic, no ML
license.DocumentDetected += (_, e) =>
status =$"{e.Document.FirstName}{e.Document.LastName} — {e.Document.Number}";
What ships:
InvoiceAnalyzer → Invoice with order lines in .Lines.
ReceiptAnalyzer → Receipt with purchased line items (.Lines), a per-tax breakdown (.Taxes), and subtotal / tip / discount / total, plus best-effort payment method, last-4, currency, date/time.
DriversLicenseAnalyzer → DriversLicense, decoded from the PDF417 barcode on the back and parsed against the AAMVA standard — fully deterministic. Works for US states and the Canadian provinces that emit an AAMVA PDF417 (BC, AB, SK, MB, NS, NB, PEI, NL); dates auto-switch to Canadian CCYYMMDD order and the province surfaces as Jurisdiction. (Ontario and Quebec licences carry no PDF417, so they don’t scan — use a custom OCR parser for those.)
HealthCardAnalyzer → HealthCard, OCR tuned for Canadian cards: it detects the issuing province from on-card keywords and applies that province’s number format — Quebec/RAMQ, Ontario/OHIP, BC PHN, Alberta/AHCIP, etc. — surfacing Province and Issuer.
CreditCardAnalyzer → CreditCard: brand (Visa/Mastercard/Amex/…) and number validity from the IIN prefix + Luhn are deterministic; name/expiry are best-effort OCR. The CVV lives on the back, so a front scan almost always leaves it null.
PassportAnalyzer → Passport, parsed from the MRZ (the two <<< lines, ICAO TD3) — deterministic.
The deterministic ones (driver’s license PDF417/AAMVA, passport MRZ, credit-card IIN/Luhn) are exactly that — no ML guesswork. The rule-based ones (invoice, receipt, health card) are best-effort, and when you need more accuracy you swap in your own parser without writing a new analyzer:
A document analyzer accumulates reads across a few frames before it commits (AccumulationFrames, default 5) — handy when one frame catches half the card and the next catches the rest — and resets after a run of empty frames (ResetAfterEmptyFrames).
Need to scan something we don’t ship — a business card, a shipping label, a lab form? Derive from DocumentAnalyzer<TDocument> and supply an IDocumentParser<TDocument>. The base class runs the shared OCR recognizer, calls your parser, raises the typed event (and command) on the UI thread, draws the boxes, and honours arming / IsEnabled / ShowBoundingBox / OverlayProvider. You write the payload and the parse rules — nothing else:
Because the parser is just an interface, an LLM- or service-backed parser is perfectly fine — the analyzer drops frames while one parse is in flight, so a slow remote call won’t pile up.
Add only the packages you need. MotionAnalyzer is pure-managed and runs everywhere; the barcode / face / OCR / document analyzers ride native ML/vision, so they produce results on the device platforms (not bare net10.0) — barcode and the PDF417-based driver’s license are iOS / macOS / Android.
Preview, zoom, filters (CSS), photo, and video capture work in every browser. Barcode scanning uses the browser’s native BarcodeDetector (Chromium today): assign a BarcodeAnalyzer and await RequestBarcodeAsync() for the next code, or handle BarcodesDetected for every code in a frame while a request is outstanding. On Firefox/Safari OnError fires once and preview continues, so feature-detect if you need universal coverage. Face/motion/OCR/document analyzers are MAUI-native.
Permissions are yours to declare — NSCameraUsageDescription on Apple (omitting it crashes iOS instantly), the CAMERA permission on Android, the webcam capability on Windows. getUserMedia needs a secure context (HTTPS or localhost).
Android: video vs. analyzers — CameraX caps concurrent use-cases, so the camera binds either image analysis (while an analyzer is enabled) or video capture. Clear Analyzer or set IsEnabled = false to record; StartVideoRecordingAsync throws a clear error otherwise.
Don’t gate startup on RequestPermissionAsync() — it routes through the handler and returns false before the view is connected (e.g. in OnAppearing on first show), which looks like a denial. Rely on auto-start + CameraError.
Grab the packages, call .UseShinyCamera(), and you’ve got a real camera — preview to structured documents — on every platform you ship to. Full docs are in the CameraView guide.
One IHealthService for Apple HealthKit and Android Health Connect — 30+ data types across activity, body, vitals, nutrition, reproductive/cycle tracking, and workouts, real-time observation, and a brand-new Microsoft.Extensions.AI tool surface for LLM agents.
Shiny.DocumentDb.Orleans puts the entire Orleans persistence stack — grain storage, reminders, clustering, and grain directory — on one backend-agnostic IDocumentStore. The headline win: query grain state directly, without activating a single grain.
Shiny.Data.Sync is a background-capable, bidirectional record-sync engine. It doesn’t reinvent background execution — it stands on the same playbook as Shiny Jobs and Shiny.Net.Http, moving records where transfers move files and riding the Jobs scheduler for periodic pulls.