Skip to content

fix: native panel size on non-16-aligned panels, bitrate changes freezing the Mac app, and bitrate overflow - #112

Merged
tranvuongquocdat merged 3 commits into
tranvuongquocdat:mainfrom
hectorviov:fix/native-size-and-bitrate-freeze
Nov 7, 2026
Merged

tranvuongquocdat merged 3 commits into
tranvuongquocdat:mainfrom
hectorviov:fix/native-size-and-bitrate-freeze

Conversation

@hectorviov

@hectorviov hectorviov commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #110. Fixes #111.

Three fixes, one commit each, so they can be reviewed or reverted separately. The third came out of testing the second.

1. Stream at the exact panel size when its sides aren't multiples of 16 (#110)

On panels with a side that isn't a multiple of 16, the stream never reached native size: a Xiaomi Pad 6 (2880×1800) got 2864×1792 and upscaled it back, and the 2340×1080 LG in #88 got 2336×1072. Two places rounded down to 16 on their own.

  • Android, CodecCapabilities.maxStreamSize: tries the native panel size first, aligned only to the decoder's own widthAlignment/heightAlignment. areSizeAndRateSupported already counts macroblocks rounded up, so 1800 or 1080 needs no flooring. Over-budget panels still shrink stepwise on 16-aligned sizes exactly as before, so [Bug] My Android screen definition is always set to 1968 × 1216 #78 behavior is unchanged. The search is split out as fitStreamSize(...) with an injected supports check so it can be unit tested. Exception handling is unchanged.
  • Mac, CodecLimits.clamp: when the capture has the box's aspect ratio (HiDPI 5760×3600 into a 2880×1800 limit), it returns the box as is (kept even for 4:2:0) instead of 2880×1792. Every other fit still floors to 16, including the 2880×1792 limit an older client sends, which still gives 2864×1792.

HEVC handles sides that aren't multiples of 16 through the conformance window. HEVC streams without a client limit already go out at native sizes like 2880×1800.

2. Changing the bitrate while streaming froze the app (#111)

Every slider step called VideoEncoder.updateSettings on the main thread, which invalidated and recreated the compression session while the capture queue kept encoding into it. That produced -12903, wedged VTEncoderXPCService, and froze both threads (spindump in #111).

  • One queue for the session: all access to compressionSession now runs on a private serial queue. encode() is submitted there from any capture queue, and updateSettings() returns at once and applies the change between frames.
  • Live bitrate changes: a bitrate-only change sets kVTCompressionPropertyKey_AverageBitRate on the live session instead of rebuilding it, so the stream continues with no gap. If that call fails, it falls back to a rebuild. Quality and Gaming Boost changes still rebuild, now serialized with encoding.
  • Slider bursts collapse: while an apply is queued, newer values only replace it.

3. Cap the average bitrate at what VideoToolbox accepts

kVTCompressionPropertyKey_AverageBitRate is read as a signed 32-bit value. Probing a live HEVC session on macOS 27.3 / M5 Pro:

Set (Mbps) VTSessionSetProperty Read back
1540 noErr 1,540,000,000
2147 noErr 2,147,000,000
2148, 2190, 4000, 4294 -12900 2,147,000,000 (previous value kept)
4680 noErr 385,032,704
5000 noErr 705,032,704

The slider goes to 5000 and the session setup ignored the status, so the top half of the slider either did nothing or set a far lower bitrate than shown. The bitrate is now capped at Int32.max bps (about 2147 Mbps), split out as VideoEncoder.averageBitRateBps so it can be tested. With the cap, those slider steps also stay on the live-update path instead of falling back to a rebuild.

Verification

  • Mac: swift test, 85/85 (10 new). The universal release build CI uses also compiles.
    • CodecLimitsTests, 6 new: Pad 6 HiDPI → 2880×1800, 2340×1080 HiDPI → 2340×1080, portrait transposition, even rounding, an older client's 2880×1792 limit still giving 2864×1792, aspect mismatch still 16-aligned.
    • VideoEncoderSettingsTests, 2 new, skipped without a hardware HEVC encoder:
      • The blocking test: 1,000 updateSettings calls blocked the caller for 4.3 s on the old code and now return immediately.
      • A streaming-while-dragging stress test: it also passes on the old code, because the freeze needs real capture timing, so treat it as a smoke test.
      • Two bitrate-cap tests: 5000 and 3000 Mbps → Int32.max; 2147, 1540, the 60 Mbps floor and Gaming Boost are unchanged.
  • Android: testDebugUnitTest, all passing (6 new in FitStreamSizeTest, using a stand-in for areSizeAndRateSupported that counts blocks per second). I couldn't run ktlint locally, so I'm relying on CI for it.
  • End-to-end, fix 1, on a Xiaomi Pad 6 (Android 16, OMX.qcom.video.decoder.hevc), USB, 120 fps, debug client against the released 1.1.0 Mac app:
    • Before: Client decoder limit table: 30fps=2880x1792, 60fps=2880x1792, 120fps=2880x1792 and Encode size 5760x3600 -> 2864x1792.
    • After: Client decoder limit table: 30fps=2880x1800, 60fps=2880x1800, 120fps=2880x1800 and Sent display config: 2880x1800. Stable stream at about 95–105 fps, dropped: 0, overlay 2880x1800 (desktop 1440x900).
    • That run used a 2880×1800 capture, which passes the clamp unchanged. I haven't run the Mac half (5760×3600 HiDPI capture) end to end; unit tests cover it.
  • End-to-end, fix 2, in the app (ad-hoc build of this branch with fixes 1 and 2, before fix 3), streaming 2880×1800 HEVC at 120 fps over USB to the Pad 6:
    • The slider was dragged across its whole range (20 ↔ 5000 Mbps) several times, with quality and Gaming Boost toggles mixed in.
    • It produced 76 live bitrate updates, with no freeze and no new .hang report. 1.1.0 froze twice on the same action.
    • The only rebuilds during pure bitrate drags happened at 2190, 2490, 2630, 3620, 4000 and 4160 Mbps. That's the -12900 range above, and it's what led to fix 3.
  • Fix 3 in the app: I didn't drag the slider again with fix 3. The VideoToolbox probe above checks the exact call it changes, and the cap is unit-tested.

🤖 Generated with Claude Code

hectorviov and others added 2 commits November 6, 2026 13:48
…f 16

A 2880x1800 panel was streamed at 2864x1792 and upscaled back on the
tablet (a 2340x1080 phone at 2336x1072), so text was slightly soft.

- Android: maxStreamSize tries the native panel size first, aligned
  only to the decoder's own widthAlignment/heightAlignment.
  areSizeAndRateSupported already counts macroblocks rounded up, so
  1800 or 1080 needs no flooring to 16. Over-budget panels still
  shrink on 16-aligned sizes as before. The search is split out as
  fitStreamSize so it can be unit tested.
- Mac: CodecLimits.clamp returns the client's box as is when the
  capture has the box's aspect ratio (HiDPI 5760x3600 into 2880x1800),
  instead of flooring it to 2880x1792. Other fits are unchanged.

Verified on a Xiaomi Pad 6 against the 1.1.0 Mac app: the limit table
is now 2880x1800 at 30/60/120 fps and the stream decodes at 2880x1800
at about 100 fps. Mac: 81/81 tests (6 new). Android: 6 new unit tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every bitrate slider step called VideoEncoder.updateSettings on the main
thread, which invalidated and recreated the compression session while
the capture queue kept encoding into it. VideoToolbox reported -12903
(invalid session), its encoder XPC service hung, and the app froze with
both threads blocked in mach_msg (spindump) until it died.

- All session access now runs on a private serial queue: encode() is
  submitted there, and updateSettings() returns at once and applies the
  change between frames.
- A bitrate-only change sets kVTCompressionPropertyKey_AverageBitRate
  on the live session instead of rebuilding it, so the stream continues
  with no gap. Quality and Gaming Boost changes still rebuild, now
  serialized with encoding.
- Slider bursts collapse: while an apply is queued, newer values only
  replace it.

Tests: 2 new in VideoEncoderSettingsTests (skipped without a hardware
HEVC encoder). updateSettings x1000 took 4.3 s on the caller before and
now returns immediately; streaming while dragging keeps producing
frames. Full suite 77/77.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 6, 2026

Copy link
Copy Markdown

@hectorviov is attempting to deploy a commit to the tranvuongquocdat2-7001's projects Team on Vercel.

A member of the Team first needs to authorize it.

kVTCompressionPropertyKey_AverageBitRate is read as a signed 32-bit
value. Probing a live HEVC session: 2147 Mbps is accepted, 2148-4294
Mbps return -12900 and keep the previous value, and higher values wrap
(4680 Mbps reads back as 385 Mbps, 5000 as 705). The slider goes to
5000, and the session setup ignored the status, so the slider's top
half silently did nothing or set a far lower bitrate.

Cap at Int32.max bps (about 2147 Mbps), split out as
VideoEncoder.averageBitRateBps so it can be tested. With the cap, those
slider steps update the live session instead of falling back to a
rebuild.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hectorviov hectorviov changed the title fix: native panel size on non-16-aligned panels, and bitrate changes freezing the Mac app fix: native panel size on non-16-aligned panels, bitrate changes freezing the Mac app, and bitrate overflow Nov 6, 2026
@hectorviov

Copy link
Copy Markdown
Contributor Author

Tested the bitrate fix in the real app, and pushed a third commit that the test turned up.

  • Slider freeze ([Bug] Mac app freezes when the bitrate is changed while streaming #111): with an ad-hoc build of this branch, I dragged the slider 20 ↔ 5000 Mbps repeatedly while streaming 2880×1800 at 120 fps. That gave 76 live bitrate updates, with no freeze and no hang report. 1.1.0 froze twice on the same action.
  • New: bitrate overflow (9b3ed78). VideoToolbox reads AverageBitRate as a signed 32-bit value: 2148–4294 Mbps are rejected (-12900), and higher values wrap (5000 Mbps becomes 705 Mbps). The bitrate is now capped at Int32.max bps. Details and the probe results are in the description.

swift test passes 85/85.

@tranvuongquocdat

Copy link
Copy Markdown
Owner

Merged, thanks for the thorough work on all three — the spindump analysis on #111 especially. This goes out in 1.1.1.

@tranvuongquocdat
tranvuongquocdat merged commit f5bab86 into tranvuongquocdat:main Nov 7, 2026
2 of 3 checks passed
SirRiddle pushed a commit to SirRiddle/SideScreen that referenced this pull request Nov 11, 2026
…urry streams (tranvuongquocdat#78) (tranvuongquocdat#88)

Two causes behind the same blurry, undersized stream:

- The single type-11 limit is the smaller of the panel and the 60 fps
  decode budget, so the Mac could not tell them apart and halved its
  area for 120 Hz streams even when the decoder had budget to spare
  (MatePad Air: 2800x1840 became 1968x1280).
- Budget tablets pair a Level-4-ish HEVC decoder with a far stronger
  H.264 one, so HEVC shrank the picture to a quarter (Honor Pad X9:
  1248x752 of 2000x1200).

Android now sends wire type 14 after type 11: the sustainable size
measured at 30, 60 and 120 fps, every byte >= 0x80 so a Mac that
predates it skips the payload byte-wise (verified against 0.11.3).
The Mac uses the entry at the lowest rate at or above its stream rate
and only scales by area beyond 120 fps; without a table the old
scaling is unchanged. The client also advertises AVC-only when its
H.264 decoder sustains at least 1.2x the HEVC area at 60 fps.

Tests: 7 new CodecLimits cases (Mac), 5 DecoderLimitWire cases
(Android). E2E on an LG V510N against both 0.11.3 and this build.

(cherry picked from commit 853ed08)

Fork conflict resolution: MainActivity keeps the fork's decoder-init
structure with upstream's reworded toast; CodecLimitsTests restores the
tranvuongquocdat#78 table tests dropped from the tranvuongquocdat#112 pick (test set now matches upstream).
SirRiddle added a commit to SirRiddle/SideScreen that referenced this pull request Nov 11, 2026
- Upstream 0.11.3..1.1.1 (synced through 73b8067): tranvuongquocdat#102 adb reverse per
  device, tranvuongquocdat#103/tranvuongquocdat#113/tranvuongquocdat#98 Android crash fixes, tranvuongquocdat#112 exact panel size + encoder
  freeze + bitrate overflow, tranvuongquocdat#88 per-rate decoder table (type 14), tranvuongquocdat#92
  per-tablet profiles (type 18), tranvuongquocdat#81 build scripts, tranvuongquocdat#93 CameraX/ML Kit,
  tranvuongquocdat#106 logo, Sparkle (fork feed/key), SDK 36 + R8. Skipped items are listed
  in CLAUDE.md.
- Fork fixes for bugs open upstream: tranvuongquocdat#69 admission before swap, tranvuongquocdat#70 USB
  checklist lifecycle, tranvuongquocdat#72 dead-link watchdog, tranvuongquocdat#73, tranvuongquocdat#74, tranvuongquocdat#118 port,
  tranvuongquocdat#89 backup exclusions, tranvuongquocdat#91/tranvuongquocdat#100 adb discovery + path setting, tranvuongquocdat#108
  stable display identity, tranvuongquocdat#76 non-blocking reachability, tranvuongquocdat#80 device
  parsing/hints.
- Low latency at 120 Hz (default on), bitrate Max = 2147 Mbps.
- Stable signing: self-signed Mac identity (permissions survive updates),
  Sparkle on the fork feed, Android release key; signed test DMG per CI run.
- VERSION 1.2.0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants