Skip to content

Audio input: serve auddmg and audsmx streams through Mac and iPhone microphones - #1282

Open
rastapasta wants to merge 1 commit into
seedds:mainfrom
rastapasta:fix/audio-input-capture
Open

rastapasta wants to merge 1 commit into
seedds:mainfrom
rastapasta:fix/audio-input-capture

Conversation

@rastapasta

Copy link
Copy Markdown
Contributor

What and why

Games opening audsmx:u need captured PCM in their shared input ring. Returning a device list alone leaves OpenInputStream without a producer.

  • Extend CMIF transport with bounded typed payloads, MapAlias buffers, copied handles and owned child interfaces. Duplicate sessions share ownership, and final close is serialized with dispatch.
  • Serve auddmg:u device queries, profiles and microphone attachment, plus owned audsmx:u input streams, state and filled-frame events, gain ramps and format conversion.
  • Publish captured samples before the producer cursor. Preserve the consumer cursor on overflow and drain capture callbacks before releasing stream memory.
  • Capture the default microphone through AudioQueue. Add permission metadata and iPhone audio-session, interruption, background and route-reset handling, plus regression impact mappings.

The named routes currently share the default microphone. Legacy AudioIn, output mixer streams, device-change notifications and dedicated voice processing remain unsupported. Audio reference.

Checked

  • This branch on current main: t_sf (40 checks), t_auddmg (89), t_audio_input_ring (18) and t_audsmx (126), normally and under ASan/UBSan, all pass. Documentation, usage-map and Mac packaging tests pass.
  • iPhone source compile: 264 files, zero failures. Device: Apple M2, macOS 27.0.1, Xcode 27.2. No iPhone hardware run.
  • Earlier local integration: the Mac capture test received 96,480 nonzero samples and checked stop/suspend/resume/reset; the service test consumed 57,600 frames through the microphone and shared ring. Ocarina opened both input streams. These hardware/game checks were not repeated on the rebased branch; game microphone behavior and gameplay remain unvalidated.
  • C lint: all new implementation and test files have no findings. LLVM 23.1.5 reports one existing hle_oe.c/fread sizing finding, reproduced against unchanged upstream. The Python ratchet reports existing findings in dm/games/settings.py:115 and dm/tools/tests/test_split_linkdata.py:39; both files are byte-identical to upstream.
  • regress.py impact origin/main..HEAD completed. The five-game check selected Ocarina (input streams), Animal Crossing (CMIF), Mario Kart World (CMIF/camera), Wonder (playback) and Hades (another playback/API path). It stops at preflight with exit 2: only Ocarina has a local Mac build. No gameplay regression pass is claimed. Metal cache cleanup was skipped to preserve the ongoing shader build.
  • The full host/graphics suites passed on the earlier combined local changes; this rebased branch received the focused checks above.

Front ends

Front-ends: new: Include NSMicrophoneUsageDescription, compile capture_apple.m and link AVFAudio. iPhone front ends must forward audio-session, suspension and route-reset events as described in docs/frontends.md.

Docs

  • Updated existing reference documentation and regenerated the symptom index.
  • No shader-chain change.
  • No game files, keys or extracted data included. No new Markdown files.

Add typed CMIF requests, owned child interfaces and copied-handle decoding, then use them for audio-device queries and mixer input streams. Capture default-device PCM through AudioQueue, publish it through the shared input ring, and drain callbacks before releasing stream state. Include conversion, gain, notification and lifetime tests.

Configure microphone permission metadata and iPhone session, interruption and route handling. Add regression impact coverage for the shared transport and capture backend.

Front-ends: new: Include NSMicrophoneUsageDescription, compile capture_apple.m and link AVFAudio. iPhone front ends must forward session, suspension and route-reset events as described in docs/frontends.md.
@github-actions github-actions Bot added host The Horizon host: SVCs, services, threads, memory (dm/host) packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers ios The iPhone and iPad app (dm/ios) mac The Mac front end and app (dm/mac, front_mac.m) tools Tools and scripts (dm/tools, scripts) docs Documentation labels Nov 5, 2026
@rastapasta
rastapasta force-pushed the fix/audio-input-capture branch from b8cca05 to 6e6bb41 Compare November 5, 2026 06:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation host The Horizon host: SVCs, services, threads, memory (dm/host) ios The iPhone and iPad app (dm/ios) mac The Mac front end and app (dm/mac, front_mac.m) packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers tools Tools and scripts (dm/tools, scripts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant