Performance reports

Sync 1080p60: next research plan

11 September 2026 · Research plan · Evidence from Apple M2 / 24 GiB and Apple M4 / 32 GiB

Recommended first experiment: instrument the camera extension's consume cycle, the stage between the daemon's CMIO sink enqueue and the extension's relay of the sample, before changing it. Locate the mechanism there before proposing event-driven consumption or any other change.

This document updates the 10 September 2026 plan after its recommended first experiment (section 4A) ran. This document outlines proposed research. The measurements below come from earlier experiments. No performance tests ran for this publication.

1. Objective and constraints

The objective remains sustained 1920×1080 at 60 unique delivered frames per second, without performance degradation over time. The implementation must serve web and desktop builds. An average of 50 unique FPS is an accepted progress checkpoint. It does not establish completion or resolve the recurring low-rate intervals.

Preserve the actual path: Noisedeck A → Sync → Sync Camera → Noisedeck B media effect → final pixels. Use testPattern() for reproducible identity checks. Check representative shader content before release. Defer 4K work.

The local research baseline uses Sync d167ee4fefb12ad714f8b20023f4d2ddaed0e840 and Noisedeck b7c0cb0ac3e89ea5aa5552e35dde642f6dbbad09, unchanged from the 10 September 2026 plan. The producer-comparison leg (section 4A) used the same baseline throughout. These revisions identify the research baseline. They do not identify the latest deployed versions: Sync has since shipped 0.2.66, built from sync 9df4c3b7b38a62909cad36e36eed362288e85787. Keep the research baseline fixed for the extension consume-cycle trace below unless a discriminating change requires otherwise.

2. Evidence that determines the next action

Observation Supported conclusion Limit
Five-second trace: 300 camera arrivals, 300 uploads, 229 distinct timestamps read. The receiver can upload the preceding camera frame despite receiving about 60 frames/s. Timestamp selection does not measure final pixels. E1
Camera arrival intervals ranged from 9.042 to 24.200 ms. Slot replacement averaged 0.023 ms after enqueue. Arrival timing deserves investigation before compositor handoff optimization. The trace does not locate jitter before browser arrival. E1
Fixed delay: web final delivery rose from 45.595 to 51.642 FPS. Desktop fell from 57.426 to 48.394 FPS. Reject the fixed delay as a shared change. Check both builds early. Short sequential comparisons do not estimate long-session performance. E2 E3
Web minimum intervals remained near 42 FPS with the fixed delay. An average gain did not remove the spikes. One web pair. E2
Camera queue accepted 59.940 frames/s while the receiver observed 49.011 unique final FPS. Successful native enqueue does not establish delivery. The observer cost about 9.7 ms per capture and could affect the result. E4
A 30-second profile attributed 6.699 seconds to sender getBufferSubData. Sender and receiver GC samples totaled about 16 and 46 ms. Sender readback remains a candidate boundary to measure. API samples include waiting or preemption. This does not exclude GC during particular stalls. E5
Media pooling reduced sampled component allocations from 35.48 MB to 0.20 MB. All four delivery comparisons were lower. Do not retain pooling solely because an allocation benchmark improves. These comparisons do not prove why delivery fell. The unshipped patch was removed. E6

Historical soaks also found declining delivery under camera backpressure. Browser context restarts recovered delivery. Eleven daemon restarts did not. The reset boundary remains unresolved. A new context also creates a new client, so those observations do not isolate browser state. E7

The eight-hour Node camera arm sent without pacing. Its accepted counter measured camera queue enqueue. It cannot decide browser versus publisher degradation. E7

Treat short-term frame selection, recurring spikes, and multi-hour decline as separate measurements. One explanation might connect them, but current evidence does not establish that connection.

Camera arrivals and selected frame timestamps

Fixed-delay desktop comparison

The recommended first experiment above (section 4A) ran on 11 September 2026 and adds this evidence, published separately as a full report (E10, linked below):

Observation Supported conclusion Limit
A paced Node producer (submit-interval sd 0.005 to 0.012 ms, absolute monotonic deadlines) still arrived at a Chrome MediaStreamTrackProcessor reader and a native AVFoundation client with interval sd 2.476 to 3.264 ms, p99 22.740 to 24.400 ms, max 24.369 to 31.100 ms, in two runs of reversed order. The browser sender does not create the body of the arrival jitter this plan's evidence (E1) found. One host (Apple M4 / 32 GiB), not the M2 / 24 GiB evidence host; two runs per cell; the two receivers ran in separate sessions. E10
By frame identity, 78 to 81 percent of that spread's variance entered between the daemon's CMIO sink stamp and the extension's relay of the sample; the stage's latency was bimodal (0.258 to 7.073 ms, sd 1.579 to 1.762 ms) with a plateau about 4 ms wide, matching the extension's empty-queue wait. The extension's consume cycle is the leading candidate boundary. The sink stamp's clock provenance is recorded as unverified; the discriminating run (a sink overrun with the daemon's dropped counter above zero) did not occur. E10
Send-interval gaps above 25 ms appeared in every browser-sender window (33 to 48 per 20 s window), each matched within 60 ms by a downstream arrival gap above 25 ms at both receivers (48 of 48, 33 of 33, 47 of 47, and 45 of 45). Section 4B's entry condition, a controlled trace identifying sender stalls that precede downstream gaps, is met. The delay is localized only to between the SDK's frame timestamp and its socket.send call; GPU fence readiness, buffer copy, encoding, and submission are not yet separated. E10
The Chrome reader re-delivered the same picture within 1 ms after most sender-side gaps (44 to 51 per 20 s window); a native AVFoundation client on the same sender, same daemon, and same extension never did. A receiver-side or capture-path re-delivery effect exists, separate from arrival-interval jitter. Not exclusive to the browser sender: three same-hash re-deliveries appeared in one native-producer warmup window. Frame selection and uploads were not measured, so section 4C's entry condition is not met. E10

Stage-timing diagram: producer submit, daemon sink stamp, extension relay stamp, and receiver arrival, interior 5 second window

Arrival-interval histograms per arm and native-to-AVFoundation stage-latency histograms

3. Measurement contract

Use these definitions in every comparison:

Keep the same observer, source, resolution, engine snapshot, and settings within each pair. Record actual browser versions. The retained desktop fixture uses the application source entry, not a packaged release. Its Chromium version differs from the web fixture. Cross-build differences therefore do not isolate the desktop shell.

Use traces without the pixel observer to locate timing boundaries. Use final pixels to evaluate delivery. Preserve both results. Never substitute callback counts, render entries, or queue acceptance for unique delivery.

The existing pooled WebGL2 observer adds measurable work. It has no drain operation. Record pending captures at both endpoints and abandoned captures at stop. Bound capture-window counts with those values. Keep completed-read counts separate from capture counts. Reject an acceptance result if observer errors or missing records prevent valid accounting.

Full-texture checks and deliberate one-pixel failure controls must pass before timing. Equal acquisition and final totals do not prove identity equality for each capture. Exact paired claims require a bounded record that joins actual identities from both observations.

Record every complete second, partial endpoints, zero-delivery seconds, and missing intervals. Retain median, p95, p99, maximum marker age, and consecutive intervals below 50 FPS. Record source pressure and native failures separately.

Sample AC power, thermal state when available, and competing CPU/GPU activity. Do not run builds or unrelated profiling during a delivery comparison. Do not stop unrelated processes. Use "GPU contention on the test host" as a hypothesis until controlled evidence supports it.

Two tools are now available for locating a boundary without a full delivery claim: an identity-bearing paced producer (apps/sync-soak/bin/paced-producer.mjs) that submits frames on absolute monotonic deadlines with a 24-bit sequence marker, and a receiver-agnostic per-arm probe (apps/sync-soak/bin/producer-probe.mjs) that can drive either a Chrome MediaStreamTrackProcessor reader or a native AVFoundation client, recording arrival timing and, where identity exists, per-stage latency by frame. Arrival timing at a reader is not unique delivered FPS. Do not substitute one for the other. Record the cross-process clock join bound for every joined stage; do not read a latency finer than that bound.

4. Research order

A. Locate jitter before browser frame selection, answered 11 September 2026

Original question: does the browser sender create the arrival jitter, or does jitter persist with a paced native producer?

Answer. A paced Node producer submitting on absolute monotonic deadlines still arrived at both a Chrome reader and a native AVFoundation client with the interval spread this plan's evidence (E1) had attributed to the browser sender. By frame identity, 78 to 81 percent of that spread's variance enters between the daemon's CMIO sink stamp and the extension's relay of the sample; the stage's latency is bimodal, with a plateau about 4 ms wide matching the extension's empty-queue wait. The browser sender adds a separate tail on top of this. Full numbers and method are in E10, linked below.

Decision reached. "Jitter persists" fired. Per this section's original decision rule, the next step is to instrument native submission, extension consumption, and browser arrival. Native submission, the daemon's sink stamp, and both receivers are now instrumented (E10). The extension's consume cycle is not: no record exists of each consume call's issue time, completion time, nil-or-not result, hasMore flag, or relay time, and no extension CPU or power sample exists. Section A2 below specifies that experiment.

Also decided: "paced production reduces arrival jitter" fired, in both orders and at both receivers, which by the same decision rule opens section 4B below. Its entry condition is now met.

A2. Instrument the extension's consume cycle

Question: is the stage between the daemon's CMIO sink enqueue and the extension's relay of the sample the mechanism behind the residual arrival jitter, and does the extension's fixed 4 ms empty-queue wait explain its bimodal shape?

Entry condition: met. Section A's paced-producer comparison places 78 to 81 percent of the native receiver's arrival-interval variance in this stage, with a latency distribution (0.258 to 7.073 ms, sd 1.579 to 1.762 ms) that a uniform 0 to 4 ms wait alone would not produce.

  1. Build an instrumented extension, the same source as the installed build or a recorded rebuild, that logs per consume call the issue time, the completion time, nil or non-nil, the hasMore flag, and the relay time, all on the host clock, with bounded storage.
  2. Join each record by CMIO sequence number or by the producer's marker to the daemon's pts stamp and to the receiver's arrival, at either receiver from section A.
  3. Record extension CPU and power at 60 fps and at idle, with the present 4 ms wait unchanged. Do not shorten the wait in this step.
  4. Record measured clock anchors between the daemon, the extension, and the receiver process, as in section A.

This settles which population, short or plateau, each frame falls in, whether the framework parks an empty consume call or returns nil immediately, and the pts stamp's clock provenance, enqueue-time versus a CMIO relay re-stamp, which is unverified by record today.

Candidate change, only after the trace supports it: event-driven consumption of the sink queue, re-arming consume without the timed gap, or another framework-side wake-up, should collapse the pts-to-relay latency from two populations to one if the plateau is the poll's own round trip. This requires rebuilding and re-approving the system extension, and carries a CPU-spin risk if the framework returns nil immediately rather than parking the call. A one-frame jitter buffer, relaying on a fixed timer, is not a candidate: it adds 16.7 ms of marker age, this plan already rejected a fixed delay for delivery reasons (E2, E3), and it would reproduce a two-clock beat of the kind already found on this host.

Do not shorten the 4 ms poll on this plan's own rule without evidence of missed arrival deadlines and idle CPU and power measurements. Leg A recorded zero seconds below 50 or 55 unique frames per second in every native arm, so that evidence does not yet exist; do not treat the plateau's size alone as sufficient justification.

Validating measurement, baseline then candidate, one reverse-order confirmation: the pts-to-relay histogram, which must collapse to one mode, AVFoundation and Chrome-reader arrival-interval sd, p95, p99, and max, complete-second delivery counts, marker loss and duplicates, the daemon's dropped counter, extension CPU and power at 60 fps and at idle. Reject on any increase in marker loss, any CPU spin at idle, or no change in the histogram.

Ordering. Run this before section 4B's sender-stall separation. It completes the unmeasured part of section A's original decision rule, it does not depend on the browser sender, and it settles the pts provenance that section A's whole stage picture rests on. Both are one-hour screens (section 5) and must run serially on the host.

Deliverable: a consume-cycle timing diagram, the pts-to-relay histogram, source receipts for the instrumented extension, and a decision naming whether event-driven consumption is justified.

B. Reduce measured sender stalls

Entry condition: met 11 September 2026. A controlled trace (E10) found send-interval gaps above 25 ms in every browser-sender window, 33 to 48 per 20 s window, each matched within 60 ms by a downstream arrival gap above 25 ms at both receivers. The delay sits between the SDK's frame timestamp and its socket.send call: stamp-to-send latency was p50 20.300 ms overall but about 37 ms for the frame that ends a gap, against gaps of about 33 ms. It is not yet separated into GPU fence readiness, CPU buffer copy, frame encoding, socket submission, and socket pressure.

Separate GPU fence readiness, CPU buffer copy, frame encoding, socket submission, and socket pressure. Compare a quiet host with a recorded, controlled GPU load. Preserve source content within each pair. Measure any tracing overhead separately.

Select one change from the measured bottleneck. Candidate changes include bounded export-buffer ownership, fewer full-frame copies, or less work while a frame awaits admission. Existing fences and queues must retain their ownership guarantees. A larger queue needs explicit latency and retained-memory evidence.

Do not repeat the rejected read-ahead change without a new causal result. Lower elapsed read time previously coincided with lower final delivery. E5

Decision: continue only if actual delivery or low-rate intervals improve without a regression in the other build. A faster isolated call is insufficient.

C. Evaluate receiver freshness only after validating the signal

Entry condition: repeated frame selection remains after A, and a public signal can guide a bounded scheduling change. Not yet met: leg A did not measure uploads or selected frame timestamps. It supplies one diagnostic input only: the Chrome reader, not a native AVFoundation client on the same sender, same daemon, and same extension, re-delivered the same picture within 1 ms after most sender-side gaps, 44 to 51 per 20 s window, and this signature is not exclusive to the browser sender; three same-hash re-deliveries appeared in one native-producer warmup window. Whether this arises in Chrome's capture service, its track, or the CMIO client path is undecided (E10).

The latest signal checks remain diagnostic:

These results support an arrival hint with a race during upload. They do not establish an exact identity signal or desktop behavior. The analysis aligns indices using the modal counter offset. It does not assume an absolute counter value.

First repeat signal validation in desktop. Test source replacement, counter resets, stale metadata, and frames arriving during upload. Do not skip an upload solely because a public counter or timestamp repeats.

Only then evaluate one bounded candidate at MediaInputManager.startUpdateLoop and stopUpdateLoop. Preserve the effective baseline cadence when frames are fresh. Measure added age, timer lateness, polling cost, and objects created by browser queries. Reject unconditional render delays and private renderer monkeypatches.

Use stable callbacks and bounded state. Test cancellation, stale callbacks, stop/restart during upload, source replacement, and exact restoration. If correct timing requires an upstream renderer hook, document that interface request. Do not fork or patch the captured engine.

Decision: reject on any clear desktop regression, stale-frame error, unbounded wait, or increased low-rate intervals. Stop after one implementation and one reverse-order check unless new evidence identifies a specific repair.

D. Test WebAssembly and workers against a measured CPU cost

WebAssembly is a candidate for a small CPU kernel, not a replacement architecture by default. Its documented use cases include image and video processing alongside JavaScript. WebAssembly FAQ

First account for bytes copied and time spent per frame. Name the work a proposed kernel would remove. Test existing JavaScript, pooled JavaScript, and Wasm SIMD with identical inputs. Include copies into linear memory and JavaScript boundary costs. Preallocate memory before measurement.

A Wasm kernel that leaves the same GPU readback and camera upload calls intact does not remove those synchronization boundaries. This is an architectural inference, not a measured rejection of Wasm.

Evaluate a worker separately from the kernel. Test a bounded transferable-buffer ring before adding shared memory. Specify ownership, backpressure, shutdown, and recovery. Check API support and isolation requirements in both actual builds before implementation. Do not assume that a shared buffer can serve as a BYOB destination.

Decision: stop if boundary copies consume the kernel saving, if memory grows, or if delivery regresses. Retain a CPU improvement only after the complete camera path confirms its practical benefit. Do not port the application to Wasm without that evidence.

E. Consider fewer transfers if the existing path remains limited

After A–D identify a persistent transfer cost, compare these architecture options on paper before prototyping one:

Option Question to answer first Required limit
GPU video import in the WebGPU receiver Can the renderer consume camera frames with fewer conversions or CPU copies? Verify color, lifetime, backend support, and the actual upstream interface.
Direct browser-to-browser receive Does it remove a measured camera detour for internal routing? It does not qualify the virtual-camera path or external receivers.
Compression or a lower-bandwidth format Does reduced transport cost exceed encode/decode cost at 1080p60? Preserve required pixel and color semantics. Report any loss explicitly.
Upstream renderer/export support Which public hook or ownership contract removes the measured wait? Request the smallest justified interface. Keep upstream alignment.

These are conditional research leads. No capability or performance claim applies until both builds exercise the proposed path. Native platform camera adapters remain valid research targets. Electron-only bypasses do not satisfy the shared implementation requirement.

5. Short decision cycles

Use one candidate at a time. Analysts can review evidence concurrently. Run camera timing tests serially on the host.

Stage Work Exit
Correctness Focused lifecycle tests, source checks, full-texture controls. Reject invalid pixels, ownership faults, or incomplete cleanup.
Initial screen Baseline then candidate, 20–30 seconds each, in web and desktop. Reject a clear delivery or tail regression in either build.
Confirmation One reverse-order pair in each build that passed. Retain a repeatable gain. Stop if signs disagree or the result remains inconclusive.
Content check Three-minute windows with deterministic and representative shader content. Confirm the gain without changing resolution, effects, or observer to favor the candidate.
Endurance Thirty minutes, then three hours in both builds. Escalate duration only while delivery, age, and resources remain stable.
Release verification Final integrated sources, required CI, and actual supported packages. Publish only the measured claim supported by the final artifact.

A screening decision should normally fit within one focused working hour, including preparation. Stop at the time limit if fixture repair dominates. Record the missing evidence instead of extending the experiment indefinitely.

Do not impose a 50 FPS minimum on every screening interval. The accepted checkpoint concerns average delivery. Report all low-rate intervals and reject candidates that worsen them. A small repeatable gain can ship as incremental progress when correctness and the other build remain intact.

An uncertain result is inconclusive. Do not claim equality, improvement, or causation. Do not run many permutations until one favors the candidate.

6. Long-session degradation and completion

Preserve the full-session clock and every interval. Compare matched workload segments. The earlier checkpoint errors came from unmatched cuts. E7

For each long run, retain five-minute delivery, marker-age, pressure, queue, resident-memory, and resource-count summaries. Compare the first and last matched hours. Preserve intervening drops and recoveries. Estimate trends with uncertainty that accounts for serial dependence between neighboring intervals.

If decline reappears, change one lifecycle boundary after a recorded baseline segment. Distinguish browser context, logical client, socket, and daemon. Verify which identities actually changed. If the protocol cannot separate two boundaries, state the confound. Do not build periodic restarts into the product as a performance fix.

Use an eight-hour run only after a candidate passes the shorter stages or a reset-boundary question requires that duration. Confirm real pacing before launch. Do not restart a progressing soak because an observation call times out.

Completion requires sustained delivery near the measured 60 FPS source cadence, correct pixels, and no degradation over time in both builds. Establish clock and observer uncertainty before interpreting a small departure from 60. The 50 FPS checkpoint, a high average with recurring 40 FPS intervals, and enqueue-only counters cannot prove completion.

This plan sets no launch tolerance for missed frames or latency. Record those distributions without replacing the original target. Physical platform and packaged-build claims require matching evidence. macOS source-entry results cannot certify Windows camera behavior.

Leg A's eight 20-second windows, inside six minutes of wall time on one host, do not bear on this section. They cannot resolve the reset-boundary question or multi-hour decline. E10

7. Retention and reporting

Keep the fixed-delay, texture-reuse, read-ahead, and media-pooling candidates rejected unless new evidence changes their premise. Do not repeat a rejected candidate under a different label.

Each future cycle should retain one local record containing:

  1. The hypothesis and the single changed variable.
  2. Source and runtime hashes, environment, workload, and test order.
  3. Raw measurements, clock bounds, missing data, and observer overhead.
  4. Delivery and latency distributions for both builds.
  5. A stage-timing diagram and a complete-second delivery plot.
  6. The decision, cleanup result, and next action.

Preserve failed attempts. Correct stale conclusions by adding the superseding result. Keep retained evidence immutable.

If implementation resumes, update the existing known-issue record at a meaningful decision. Publish a measured performance report with each release. Keep raw data and generated diagrams outside SCM.

The boundary-location tools built for section 4A are committed to the scaffold repository: apps/sync-soak/bin/paced-producer.mjs, bin/producer-probe.mjs, lib/marker.mjs, and lib/arrival-recorder.mjs. The native AVFoundation consumer and the analysis scripts used for E10 remain outside version control, per the constraint above; rebuild them from the retained sources named in E10's own report before extending this plan's next experiment.

Evidence references

E8 and E9 contain selected numeric observations from retained traces. Each upload span contains exactly one traced GetCurrentFrame. Both traces contained 300 uploads. Source stability, trace completeness, cleanup, and restoration checks passed. Full raw traces remain private. The publication checks started no camera session.

E10 is a separate published report, not a same-bundle evidence file. Its own manifest and checksums govern its retained inputs; this document cites its numbers rather than repeating its raw records.

Public evidence summaries omit local paths, process identifiers, raw logs, and connection details. Source digests identify the retained inputs.