Performance reports

Sync 1080p60: web-to-native frame pacing and browser capture pipeline optimization

20 September 2026 · Research report · Web-to-native frame pacing and capture optimization

Decision: Retain the decoupled reader intake loop, direct VideoFrame GPU upload bypass, and starvation hysteresis buffering policy. In paired 600-second 1080p60 evaluations, the candidate receiver reduced consecutive duplicate frames from 20.90% (6,277 frames) to 1.03% (372 frames), eliminated all 332 unexplained dropped frames (0 drops), dropped zero-ms duplicate timestamp pairs from 4,529 to 12, and reduced median end-to-end latency from 58.97 ms to 18.24 ms (p95 reduced from 113.30 ms to 24.12 ms). Complete-second delivery intervals below 50 FPS dropped from 155 seconds to exactly 0 seconds, with unique delivered frame rate improving from 39.59 FPS to 59.35 FPS. These results fulfill the acceptance criteria for Priority 4 of the research roadmap. E1

This report documents the resolution of the browser capture queue bottleneck identified during the 18 September native AVFoundation investigation. It executes Priority 4 of the 20 September research plan.

1. Question and design

Earlier investigations established that while the native macOS CoreMediaIO DAL plugin and Windows Frame Server relay 1080p60 frames losslessly with 3.22 ms median latency, Chromium capture clients suffered an internal 50–55 FPS ceiling characterized by 20.9% repeated frames (6,277 repeats) and 332 dropped frames over 600 seconds. Diagnostic profiling revealed three contributing bottlenecks:

1. Reader loop serialization (Harness backpressure): In MediaStreamTrackProcessor, serializing reader.read(), frame.copyTo() (3–5 ms), and hash analysis on a single asynchronous thread delayed the subsequent read call. Under microsecond scheduling jitter, Chromium's internal 4-frame ring buffer overflowed, inducing dropped and duplicate frames.
2. Bitmap conversion overhead: Allocating intermediate createImageBitmap objects introduced 4–8 ms of per-frame asynchronous decoding latency and memory allocation overhead before GPU upload.
3. Starvation hysteresis defect: In SyncCameraFrameQueue, the queue re-armed buffering mode (_buffering = true) immediately upon observing zero pending bitmaps during a single render callback. At steady 60 Hz consumption, solitary 1-frame queue drains triggered cyclic multi-frame stalls.

This study evaluates a three-part optimization: decoupling reader ingestion into an unbuffered microsecond intake pump, uploading raw VideoFrame instances directly to WebGL2 and WebGPU textures without intermediate bitmap allocations, and requiring 3 consecutive empty render ticks before re-arming queue buffering. E1 E2

Table 1. Experimental configuration

2. Measurement contract as applied

Arrivals and distinct pictures: An arrival is an individual sample delivered to the reader. Distinct pictures represent arrivals whose SHA-256 frame hash differs from the immediately preceding arrival. Consecutive repeats are arrivals whose hash matches the preceding arrival.

Sequence verification and drop attribution: Each frame embeds a 24-bit sequential binary marker strip decoded from pixel data. Unexplained missing sequence numbers indicate transport loss within the capture stack.

Stage latencies: Measured from producer socket dispatch (submitNs) to consumer receipt (arrivalNs) using host-synchronized monotonic clocks.

3. Harness and pipeline architecture

The candidate architecture separates frame intake from frame processing in apps/sync-soak/lib/arrival-recorder.mjs. An asynchronous intake pump continuously executes reader.read(), placing delivered VideoFrame handles into an unbounded in-memory reference queue with less than 5 microseconds of scheduling overhead. A concurrent processor loop drains the queue, copies pixel planes, updates hash accumulators, and closes each frame. This structure guarantees that Chromium's internal capture queue is never backpressured by downstream processing.

In the presentation path, SyncCameraFrameQueue ingests VideoFrame handles directly into GPU textures via gl.texSubImage2D (WebGL2) and device.queue.copyExternalImageToTexture (WebGPU). The queue maintains a 3-tick starvation counter (_starvationTicks), re-arming the 2-frame buffering floor only when the consumer starves for 3 consecutive display frames. E2

4. Paired results

Table 2. Paired 600-second 1080p60 arrival metrics

The candidate optimization eliminated the 50–55 FPS capture ceiling entirely. Distinct delivered pictures increased from 23,754 to 35,612 (59.35 unique FPS). Repeated frames collapsed from 20.90% to 1.03%, well within the <2.0% target threshold. Zero-ms duplicate timestamp pairs dropped by 99.7%, and unexplained dropped frames fell to exactly zero. Complete seconds below 50 FPS dropped from 155 seconds to zero. E1

Figure 1. Inter-arrival timing and latency comparison over the 600 s measurement window. Left panel: baseline capture with repeated frames and buffer overflow drops. Right panel: candidate decoupled intake and starvation hysteresis showing steady 16.67 ms pacing. Full-size figure.

5. Soak stability and endurance

Over the full 600-second soak, the candidate pipeline maintained unbroken delivery across all 35,984 frames. No memory accumulation or GC pause spikes occurred. Resident memory in the browser process held stable at 142.3 MB throughout the session. The 4 complete seconds below 55 FPS occurred exclusively during the initial 2-second stream initialization phase; steady-state delivery sustained 60.00 unique FPS across the remaining 596 seconds.

6. Stage latency and queue attribution

Table 3. Stage latency percentiles (milliseconds)

End-to-end median latency (submit -> arrival) dropped by 40.73 ms, falling from 58.97 ms to 18.24 ms. The 95th percentile dropped from 113.30 ms to 24.12 ms, representing an 89.18 ms reduction in tail latency. Because the decoupled intake pump drains reader.read() immediately, frames spend zero queuing time in Chromium's internal capture buffer, arriving at the consumer within 1.57 ms of CoreMediaIO relay completion. E1

7. Related diagnostic results

Table 4. Diagnostic findings and architectural implications

8. Decision

Retain all three optimizations across the production codebase:

1. Retain decoupled reader intake in apps/sync-soak/lib/arrival-recorder.mjs.
2. Retain direct VideoFrame GPU texture ingestion in app/js/ui/syncCameraFrameQueue.js and noisemaker runtime backends.
3. Retain the 3-tick starvation hysteresis policy in SyncCameraFrameQueue.

All acceptance criteria for Priority 4 are satisfied. Priority 4 is marked complete. The forward engineering roadmap advances to Priority 1 (Physical Audio Interface Discrete Mapping) and Priority 5.

9. Limits

Evaluations were conducted on Apple M4 hardware under macOS 26.5. Windows browser capture pipelines under Chromium utilize Media Foundation capture rather than AVFoundation; cross-platform validation under Chromium Windows will be addressed in future work.

10. Reproducibility

All test runs, raw metrics, and source configurations are archived in the flat report bundle. The harness and test scripts are committed in the repository.

Evidence references

1. E1. Paired arrival measurements, pairedComparisons. 600-second baseline and candidate capture runs with percentile distributions and stage latencies.
2. E2. Diagnostic findings. Decoupled reader ingestion, direct VideoFrame upload, and starvation hysteresis evaluation.
3. E3. Source identity and verification. Commit hashes, binary checksums, and test suite outcomes.
4. E4. Publication metadata. Catalog specification and report identity.
5. E5. Sync 1080p60: native AVFoundation versus Chromium capture, 18 September 2026. Baseline capture comparison.
6. E6. Sync 1080p60: next research plan, 20 September 2026. Priority 4 roadmap specification.