# Sync 1080p60: next research plan

10 September 2026 · Research plan · Evidence from Apple M2 / 24 GiB

**Recommended first experiment:** compare the browser sender with a verified, paced native producer through the same camera path. Locate arrival jitter before changing receiver scheduling.

This document outlines proposed research. The measurements below come from earlier experiments. No performance tests ran for this publication.

## 1. Objective and constraints

The objective remains sustained 1920×1080 at 60 unique delivered frames per second, without performance degradation over time. The implementation must serve web and desktop builds. An average of 50 unique FPS is an accepted progress checkpoint. It does not establish completion or resolve the recurring low-rate intervals.

Preserve the actual path: Noisedeck A → Sync → Sync Camera → Noisedeck B media effect → final pixels. Use `testPattern()` for reproducible identity checks. Check representative shader content before release. Defer 4K work.

- Work against current upstream versions. Preserve unrelated work.
- Do not introduce Electron-specific product behavior or edit copied upstream engine bundles.
- Keep raw measurements and generated reports outside Git. Publish reviewed reports in this archive.
- Complete short comparisons before a long soak. Commit only a retained, reviewed improvement.

The local research baseline uses Sync `d167ee4fefb12ad714f8b20023f4d2ddaed0e840` and Noisedeck `b7c0cb0ac3e89ea5aa5552e35dde642f6dbbad09`. These revisions identify the research baseline. They do not identify the latest deployed versions.

## 2. Evidence that determines the next action

| Observation | Supported conclusion | Limit |
|---|---|---|
| Five-second trace: 300 camera arrivals, 300 uploads, 229 distinct timestamps read. | The receiver can upload the preceding camera frame despite receiving about 60 frames/s. | Timestamp selection does not measure final pixels. [E1] |
| Camera arrival intervals ranged from 9.042 to 24.200 ms. Slot replacement averaged 0.023 ms after enqueue. | Arrival timing deserves investigation before compositor handoff optimization. | The trace does not locate jitter before browser arrival. [E1] |
| Fixed delay: web final delivery rose from 45.595 to 51.642 FPS. Desktop fell from 57.426 to 48.394 FPS. | Reject the fixed delay as a shared change. Check both builds early. | Short sequential comparisons do not estimate long-session performance. [E2] [E3] |
| Web minimum intervals remained near 42 FPS with the fixed delay. | An average gain did not remove the spikes. | One web pair. [E2] |
| Camera queue accepted 59.940 frames/s while the receiver observed 49.011 unique final FPS. | Successful native enqueue does not establish delivery. | The observer cost about 9.7 ms per capture and could affect the result. [E4] |
| A 30-second profile attributed 6.699 seconds to sender `getBufferSubData`. Sender and receiver GC samples totaled about 16 and 46 ms. | Sender readback remains a candidate boundary to measure. | API samples include waiting or preemption. This does not exclude GC during particular stalls. [E5] |
| Media pooling reduced sampled component allocations from 35.48 MB to 0.20 MB. All four delivery comparisons were lower. | Do not retain pooling solely because an allocation benchmark improves. | These comparisons do not prove why delivery fell. The unshipped patch was removed. [E6] |

Historical soaks also found declining delivery under camera backpressure. Browser context restarts recovered delivery. Eleven daemon restarts did not. The reset boundary remains unresolved. A new context also creates a new client, so those observations do not isolate browser state. [E7]

The eight-hour Node camera arm sent without pacing. Its accepted counter measured camera queue enqueue. It cannot decide browser versus publisher degradation. [E7]

Treat short-term frame selection, recurring spikes, and multi-hour decline as separate measurements. One explanation might connect them, but current evidence does not establish that connection.

![Camera arrivals and selected frame timestamps](arrival-timing.png)

![Fixed-delay desktop comparison](desktop-comparison.png)

## 3. Measurement contract

Use these definitions in every comparison:

- **Unique delivered FPS:** distinct, ordered identities decoded from final rendered pixels, divided by the measured interval.
- **Camera acceptance:** successful enqueue into the native camera queue. Report relay and browser arrival separately.
- **Marker age:** time from marker insertion to observed capture. It does not measure physical display latency.
- **API elapsed time:** wall time within an API call. Do not label it CPU or GPU execution time.

Keep the same observer, source, resolution, engine snapshot, and settings within each pair. Record actual browser versions. The retained desktop fixture uses the application source entry, not a packaged release. Its Chromium version differs from the web fixture. Cross-build differences therefore do not isolate the desktop shell.

Use traces without the pixel observer to locate timing boundaries. Use final pixels to evaluate delivery. Preserve both results. Never substitute callback counts, render entries, or queue acceptance for unique delivery.

The existing pooled WebGL2 observer adds measurable work. It has no drain operation. Record pending captures at both endpoints and abandoned captures at stop. Bound capture-window counts with those values. Keep completed-read counts separate from capture counts. Reject an acceptance result if observer errors or missing records prevent valid accounting.

Full-texture checks and deliberate one-pixel failure controls must pass before timing. Equal acquisition and final totals do not prove identity equality for each capture. Exact paired claims require a bounded record that joins actual identities from both observations.

Record every complete second, partial endpoints, zero-delivery seconds, and missing intervals. Retain median, p95, p99, maximum marker age, and consecutive intervals below 50 FPS. Record source pressure and native failures separately.

Sample AC power, thermal state when available, and competing CPU/GPU activity. Do not run builds or unrelated profiling during a delivery comparison. Do not stop unrelated processes. Use “GPU contention on the test host” as a hypothesis until controlled evidence supports it.

## 4. Research order

### A. Locate jitter before browser frame selection

**Question:** does the browser sender create the arrival jitter, or does jitter persist with a paced native producer?

Use the existing native-emitter source as a starting point. Rebuild it against the tested native implementation. The retained emitter binary predates that implementation. Do not use it as a current-native control.

1. Check a 60 FPS producer using monotonic, absolute deadlines and actual submission timestamps.
2. Preserve frame dimensions, format, marker sequence, and camera settings between producers.
3. Compare browser and native producers with the same receiver and installed extension.
4. Record browser arrivals, frame-slot replacements, uploads, and selected frame timestamps.
5. Repeat the short comparison in desktop after the web measurement.

Start with five seconds of warmup and 20 seconds of measurement per arm. Trace the same five-second interior window. Reverse the order only if the first result distinguishes the producers or appears sensitive to host load.

**Decision:** if paced production reduces arrival jitter and repeated selection, investigate sender export, pacing, and socket pressure next. If jitter persists, instrument native submission, extension consumption, and browser arrival. If the difference is inconclusive, preserve the result and proceed to one targeted stage trace. Do not sweep timing constants.

The producer comparison changes browser export, transport ingress, and pacing together. It narrows a boundary but cannot identify one operation as the cause. Check that both producers exercise the same camera publishing implementation and format contract.

Native inspection points are `camera_publisher.cpp`, `cmio_camera_sink.mm`, and `camera_extension/main.mm`. The extension's empty-queue wait is a possible source of jitter. Do not shorten it without evidence of missed arrival deadlines and measurements of idle CPU and power.

Check the running extension's provenance before attributing behavior to local source. Record enqueue, consume, and relay timestamps with bounded storage. Join events by identity where possible. Use measured clock anchors across processes. Do not equate browser media timestamps with native sample presentation timestamps.

**Deliverable:** one stage-timing diagram, source receipts, and a decision naming the next boundary to change.

### B. Reduce measured sender stalls

**Entry condition:** A implicates the sender, or a controlled trace identifies sender stalls that precede downstream gaps.

Separate GPU fence readiness, CPU buffer copy, frame encoding, socket submission, and socket pressure. Compare a quiet host with a recorded, controlled GPU load. Preserve source content within each pair. Measure any tracing overhead separately.

Select one change from the measured bottleneck. Candidate changes include bounded export-buffer ownership, fewer full-frame copies, or less work while a frame awaits admission. Existing fences and queues must retain their ownership guarantees. A larger queue needs explicit latency and retained-memory evidence.

Do not repeat the rejected read-ahead change without a new causal result. Lower elapsed read time previously coincided with lower final delivery. [E5]

**Decision:** continue only if actual delivery or low-rate intervals improve without a regression in the other build. A faster isolated call is insufficient.

### C. Evaluate receiver freshness only after validating the signal

**Entry condition:** repeated frame selection remains after A, and a public signal can guide a bounded scheduling change.

The latest signal checks remain diagnostic:

- `video.currentTime` matched 290 of 300 uploaded timestamps in its dedicated trace. It misclassified eight repeat transitions. [E8]
- A separate counter trace contained 300 uploads. The before/after received-frame counts enclosed every inferred frame index.
- The counter stayed unchanged across 277 uploads. All 277 matched the inferred index. The before-count alone misclassified eleven repeat transitions. [E9]

These results support an arrival hint with a race during upload. They do not establish an exact identity signal or desktop behavior. The analysis aligns indices using the modal counter offset. It does not assume an absolute counter value.

First repeat signal validation in desktop. Test source replacement, counter resets, stale metadata, and frames arriving during upload. Do not skip an upload solely because a public counter or timestamp repeats.

Only then evaluate one bounded candidate at `MediaInputManager.startUpdateLoop` and `stopUpdateLoop`. Preserve the effective baseline cadence when frames are fresh. Measure added age, timer lateness, polling cost, and objects created by browser queries. Reject unconditional render delays and private renderer monkeypatches.

Use stable callbacks and bounded state. Test cancellation, stale callbacks, stop/restart during upload, source replacement, and exact restoration. If correct timing requires an upstream renderer hook, document that interface request. Do not fork or patch the captured engine.

**Decision:** reject on any clear desktop regression, stale-frame error, unbounded wait, or increased low-rate intervals. Stop after one implementation and one reverse-order check unless new evidence identifies a specific repair.

### D. Test WebAssembly and workers against a measured CPU cost

WebAssembly is a candidate for a small CPU kernel, not a replacement architecture by default. Its documented use cases include image and video processing alongside JavaScript. [WebAssembly FAQ](https://webassembly.org/docs/faq/)

First account for bytes copied and time spent per frame. Name the work a proposed kernel would remove. Test existing JavaScript, pooled JavaScript, and Wasm SIMD with identical inputs. Include copies into linear memory and JavaScript boundary costs. Preallocate memory before measurement.

A Wasm kernel that leaves the same GPU readback and camera upload calls intact does not remove those synchronization boundaries. This is an architectural inference, not a measured rejection of Wasm.

Evaluate a worker separately from the kernel. Test a bounded transferable-buffer ring before adding shared memory. Specify ownership, backpressure, shutdown, and recovery. Check API support and isolation requirements in both actual builds before implementation. Do not assume that a shared buffer can serve as a BYOB destination.

**Decision:** stop if boundary copies consume the kernel saving, if memory grows, or if delivery regresses. Retain a CPU improvement only after the complete camera path confirms its practical benefit. Do not port the application to Wasm without that evidence.

### E. Consider fewer transfers if the existing path remains limited

After A–D identify a persistent transfer cost, compare these architecture options on paper before prototyping one:

| Option | Question to answer first | Required limit |
|---|---|---|
| GPU video import in the WebGPU receiver | Can the renderer consume camera frames with fewer conversions or CPU copies? | Verify color, lifetime, backend support, and the actual upstream interface. |
| Direct browser-to-browser receive | Does it remove a measured camera detour for internal routing? | It does not qualify the virtual-camera path or external receivers. |
| Compression or a lower-bandwidth format | Does reduced transport cost exceed encode/decode cost at 1080p60? | Preserve required pixel and color semantics. Report any loss explicitly. |
| Upstream renderer/export support | Which public hook or ownership contract removes the measured wait? | Request the smallest justified interface. Keep upstream alignment. |

These are conditional research leads. No capability or performance claim applies until both builds exercise the proposed path. Native platform camera adapters remain valid research targets. Electron-only bypasses do not satisfy the shared implementation requirement.

## 5. Short decision cycles

Use one candidate at a time. Analysts can review evidence concurrently. Run camera timing tests serially on the host.

| Stage | Work | Exit |
|---|---|---|
| Correctness | Focused lifecycle tests, source checks, full-texture controls. | Reject invalid pixels, ownership faults, or incomplete cleanup. |
| Initial screen | Baseline then candidate, 20–30 seconds each, in web and desktop. | Reject a clear delivery or tail regression in either build. |
| Confirmation | One reverse-order pair in each build that passed. | Retain a repeatable gain. Stop if signs disagree or the result remains inconclusive. |
| Content check | Three-minute windows with deterministic and representative shader content. | Confirm the gain without changing resolution, effects, or observer to favor the candidate. |
| Endurance | Thirty minutes, then three hours in both builds. | Escalate duration only while delivery, age, and resources remain stable. |
| Release verification | Final integrated sources, required CI, and actual supported packages. | Publish only the measured claim supported by the final artifact. |

A screening decision should normally fit within one focused working hour, including preparation. Stop at the time limit if fixture repair dominates. Record the missing evidence instead of extending the experiment indefinitely.

Do not impose a 50 FPS minimum on every screening interval. The accepted checkpoint concerns average delivery. Report all low-rate intervals and reject candidates that worsen them. A small repeatable gain can ship as incremental progress when correctness and the other build remain intact.

An uncertain result is inconclusive. Do not claim equality, improvement, or causation. Do not run many permutations until one favors the candidate.

## 6. Long-session degradation and completion

Preserve the full-session clock and every interval. Compare matched workload segments. The earlier checkpoint errors came from unmatched cuts. [E7]

For each long run, retain five-minute delivery, marker-age, pressure, queue, resident-memory, and resource-count summaries. Compare the first and last matched hours. Preserve intervening drops and recoveries. Estimate trends with uncertainty that accounts for serial dependence between neighboring intervals.

If decline reappears, change one lifecycle boundary after a recorded baseline segment. Distinguish browser context, logical client, socket, and daemon. Verify which identities actually changed. If the protocol cannot separate two boundaries, state the confound. Do not build periodic restarts into the product as a performance fix.

Use an eight-hour run only after a candidate passes the shorter stages or a reset-boundary question requires that duration. Confirm real pacing before launch. Do not restart a progressing soak because an observation call times out.

Completion requires sustained delivery near the measured 60 FPS source cadence, correct pixels, and no degradation over time in both builds. Establish clock and observer uncertainty before interpreting a small departure from 60. The 50 FPS checkpoint, a high average with recurring 40 FPS intervals, and enqueue-only counters cannot prove completion.

This plan sets no launch tolerance for missed frames or latency. Record those distributions without replacing the original target. Physical platform and packaged-build claims require matching evidence. macOS source-entry results cannot certify Windows camera behavior.

## 7. Retention and reporting

Keep the fixed-delay, texture-reuse, read-ahead, and media-pooling candidates rejected unless new evidence changes their premise. Do not repeat a rejected candidate under a different label.

Each future cycle should retain one local record containing:

1. The hypothesis and the single changed variable.
2. Source and runtime hashes, environment, workload, and test order.
3. Raw measurements, clock bounds, missing data, and observer overhead.
4. Delivery and latency distributions for both builds.
5. A stage-timing diagram and a complete-second delivery plot.
6. The decision, cleanup result, and next action.

Preserve failed attempts. Correct stale conclusions by adding the superseding result. Keep retained evidence immutable.

If implementation resumes, update the existing known-issue record at a meaningful decision. Publish a measured performance report with each release. Keep raw data and generated diagrams outside SCM.

## Evidence references

- [E1: Camera arrival and texture-read timing][E1]
- [E2: Web final-pixel fixed-delay comparison][E2]
- [E3: Desktop fixed-delay rejection][E3]
- [E4: Camera enqueue and browser output][E4]
- [E5: Production camera CPU profile and rejected read-ahead result][E5]
- [E6: Media allocation reuse measurements][E6]
- [E7: Corrected round-two handoff][E7]
- [E8: Public currentTime measurement][E8]
- [E9: Public received-frame counter measurement][E9]

E8 and E9 contain selected numeric observations from retained traces. Each upload span contains exactly one traced `GetCurrentFrame`. Both traces contained 300 uploads. Source stability, trace completeness, cleanup, and restoration checks passed. Full raw traces remain private. The publication checks started no camera session.

[E1]: evidence-e1.txt
[E2]: evidence-e2.txt
[E3]: evidence-e3.txt
[E4]: evidence-e4.txt
[E5]: evidence-e5.txt
[E6]: evidence-e6.txt
[E7]: evidence-e7.json
[E8]: evidence-e8.json
[E9]: evidence-e9.json

Public evidence summaries omit local paths, process identifiers, raw logs, and connection details. Source digests identify the retained inputs.