Chasing Zero Audio Latency in the Browser: Web Audio API and Precision Scheduling

In vertical scrolling rhythm games (VSRGs), a few milliseconds matter. A single visual frame slip or a slight audio buffer delay turns an accurate hit into an early or late judgement.

When building Rhythmio as a browser game, the first question we got was obvious: Can a web browser match the audio timing of native desktop rhythm games?

The answer is no if you rely on standard HTML5 audio, but yes if you build directly on the Web Audio API hardware clock. Here is how we designed our audio pipeline to eliminate latency, drift, and garbage collection stutter.

Why HTML5 <audio> Fails for Rhythm Games

The quickest way to play sound on the web is the <audio> tag or HTMLAudioElement.play(). That works for podcasts and background music, but breaks down in rhythm games for three reasons:

1. Unpredictable start latency: Calling audio.play() goes through asynchronous browser handshakes. Depending on the operating system and CPU load, the delay between calling the method and hearing the sound can swing between 30ms and 150ms.
2. Coarse time resolution: The currentTime property updates only every 16ms to 66ms (around 15Hz to 60Hz). That is far too coarse for millisecond judgements.
3. Unlinked clocks: The JavaScript main thread and the browser audio renderer run on separate clocks. If the main thread drops a frame, the audio keeps playing, and your notes slide out of sync with the music.

Hardware Clock Scheduling with Web Audio API

To get sub-millisecond precision, we built Rhythmio’s audio subsystem on AudioContext.

The key property is AudioContext.currentTime. Unlike performance.now(), which measures CPU elapsed time on the main thread, AudioContext.currentTime reads elapsed time directly from the audio hardware sample counter.

// Lookahead scheduling pattern in Rhythmio audio engine
const scheduleAheadTime = 0.1; // 100ms lookahead window
let nextNoteIndex = 0;

function scheduler() {
  const currentAudioTime = audioContext.currentTime;
  
  while (nextNoteIndex < chart.notes.length) {
    const note = chart.notes[nextNoteIndex];
    const noteTargetTime = songStartTime + (note.time / 1000);
    
    // If the note falls within our lookahead window, schedule it on the audio thread
    if (noteTargetTime < currentAudioTime + scheduleAheadTime) {
      scheduleHitSound(note.soundId, noteTargetTime);
      nextNoteIndex++;
    } else {
      break;
    }
  }
}

Instead of checking whether to play a sound on each animation frame, we schedule sound events 100ms ahead on the hardware audio thread. Even if a script creates a temporary spike on the main thread, the audio output stays smooth and on beat.

Pre-Decoding and Zero In-Game Allocations

In dense 4K and 7K charts, note streams can exceed 30 keypresses per second. Allocating a new AudioBufferSourceNode or callback on every hit will eventually trigger a V8 garbage collection pause, causing a micro-freeze right when you need precision.

To keep input response flat and avoid GC pauses:

1. Pre-decoded PCM audio: When loading a track, we fetch all hit sound samples (clicks, snares, claps) into array buffers and decode them upfront with audioContext.decodeAudioData().
2. Node pooling: We reuse source nodes through a ring buffer. No memory is allocated while notes are flying.
3. Audio worklets: When supported, an AudioWorkletNode runs custom processing on the real-time audio thread, completely independent of main thread execution.

Syncing Display Refresh with the Audio Clock

Even with a hardware clock, differences between display refresh rates (60Hz, 144Hz, 240Hz, or 360Hz) and audio sample rates (44.1kHz or 48kHz) cause minor drift over a three-minute song.

We treat the audio hardware clock as the single reference for game time:

• On every requestAnimationFrame, note positions are calculated directly from elapsed audio time: $$\text{Progress} = (\text{audioContext.currentTime} - \text{songStartTime}) \times \text{ScrollSpeed}$$
• If a Bluetooth headset connects or the user switches audio devices mid-song, causing a sudden jump in audio timestamps, the renderer interpolates over three frames rather than teleporting notes across the screen.

Wrapping Up

By pairing Web Audio hardware scheduling with pre-decoded sample pools and an audio-driven render loop, we achieved the responsive timing players expect from installed desktop rhythm games.

In the next post, we will walk through the math behind our ±16ms MAX judgement window and how accuracy is calculated.

WRITTEN BY

Rhythmio Team

The Rhythmio team — patch notes, technical architecture and design notes.