Deterministic Time Axis in Real-Time PCM Processing

t = n / fₛ

Here, n is the number of processed samples per channel and fₛ is the sampling rate. For example, in a 16 kHz stream, 16,000 samples equal exactly one second:

t = 16000 / 16000 = 1 second

In a deterministic PCM pipeline, the same input must produce the same frames, segment boundaries, and timestamps on every execution. The fundamental requirement is that the primary time state be a processed-sample counter rather than a wall clock.

Separating samples, bytes, and channel count

In 16-bit signed little-endian PCM, one sample per channel occupies two bytes:

bytesPerSample = 2

At 16 kHz, a 10 ms mono frame contains 160 samples and 320 bytes:

samplesPerFrame = 16000 × 10 / 1000 = 160
monoFrameBytes = 160 × 2 = 320

In stereo interleaved PCM, the data volume for the same duration is 640 bytes:

stereoFrameBytes = 160 × 2 channels × 2 bytes = 640

Nevertheless, the time axis does not double. A stereo frame contains 320 total 16-bit values in the order L₀, R₀, L₁, R₁, ..., but each channel has only 160 time positions. Therefore, the counter must be updated only by the number of valid samples per channel:

processedSamples += validSamplesPerChannel

A 640-byte stereo frame contains 640 / 2 = 320 16-bit values and thus 320 / 2 = 160 time samples. The operation processedSamples += 320 incorrectly records 10 ms of data as 20 ms. When channels are separated, the counter must likewise be incremented only once for each stereo frame.

Producing media time from sample positions

A system clock measures processing time, not media time. One hour of audio read from a file may be processed in minutes; network or queue delays may cause one second of audio to be processed later. Therefore, System.nanoTime(), DateTime.UtcNow, or similar wall-clock measurements must not determine a segment's position within the audio.

Frame sample boundaries are derived directly from the counter:

frameStart = processedSamples
frameEnd   = frameStart + validSamples
processedSamples = frameEnd

The millisecond representation should be calculated only when it is exported:

startMs = frameStart × 1000 / sampleRate
endMs   = frameEnd × 1000 / sampleRate

For 16 kHz, the simplification timeMs = sampleIndex / 16 applies; general code should nevertheless use sampleRate explicitly. The primary counter must be a long. At 16 kHz, a 32-bit counter can overflow after approximately 37.3 hours. The multiplication must also use long arithmetic:

final long timeMs = (long) sampleIndex * 1000L / sampleRate;

Integer division rounds down. When start and end times are calculated using the same conversion from absolute sample positions, no artificial gaps or overlaps arise between adjacent segments. Total sample positions should be converted instead of rounding and summing individual frame durations.

Partial reads and the valid length of the final frame

A read call must not be assumed to fill the requested buffer completely. For example, a 320-byte mono frame may arrive in two reads of 120 and 200 bytes. Reading must continue until the buffer is full or the actual end of file is reached:

offset = 0

while offset < frameBytes:
    n = read(buffer, offset, frameBytes - offset)

    if n < 0:
        break

    offset += n

In general stream abstractions, a read operation returning zero must additionally be assessed as a potential infinite-loop risk. The number of valid samples per channel is:

validSamples = bytesRead / (bytesPerSample × channelCount)

For stereo 16-bit PCM, this is bytesRead / 4. However, the following condition must first be verified:

bytesRead mod (bytesPerSample × channelCount) = 0

In a stereo 16-bit stream, one time sample occupies four bytes. Two or three bytes remaining at the end of a file indicate an incomplete channel sample; this data must not be silently padded or combined with previous buffer content.

The final frame may be shorter than a full frame. For example, a buffer capacity may be 160 samples while only 73 samples are valid at the end of the file. The algorithm must treat only those 73 samples as actual media data. Safe options are:

1. Process only valid samples if the algorithm supports them.
2. Explicitly zero-pad the final frame.
3. Manage remaining samples with a defined termination policy in algorithms that accept only full frames.

When zero padding is used, media and model-input lengths must be separated:

mediaSamples = 73
processingSamples = 160

The time counter advances only by mediaSamples. Otherwise, up to one frame of artificial duration is appended to the end of every file.

Half-open segment intervals and PCM layout

Segments should be represented with half-open sample intervals:

[startSample, endSample)

When adjacent intervals are [A, B) and [B, C), there are neither duplicate samples nor gaps. The same contract applies directly to PCM byte positions:

byteStart = sampleStart × bytesPerSample × channelCount
byteEnd   = sampleEnd   × bytesPerSample × channelCount

This representation is compatible with file seeking, HTTP ranges, NIO buffer boundaries, and array indices.

When reconstructing a 16-bit little-endian sample, the low byte must be masked:

final int sample = (buffer[index + 1] << 8) | (buffer[index] & 0xFF);

Because byte is signed in Java, failing to mask the low byte can cause a sign-extension error. In a stereo interleaved sample array, time order is preserved as follows:

left[n]  = stereo[2n]
right[n] = stereo[2n + 1]

Here, stereo is a 16-bit sample array; in a byte array, indices are additionally scaled by bytesPerSample.

Frame duration, state thresholds, and performance measurement

When VAD thresholds are defined by frame counts, they depend on frame duration. For example, in 10 ms frames, confirmFrames = 4 means 40 ms and silenceFrames = 30 means 300 ms. If frame duration is increased to 20 ms without changing the counters, these thresholds become 80 ms and 600 ms, respectively.

Where possible, thresholds should be defined in time or samples and converted to frame counts:

confirmFrames =
    ceil(confirmMs × sampleRate /
         (1000 × samplesPerFrame))

Frame duration affects VAD decision frequency, speaker-change latency, segment-termination resolution, minimum detectable speech duration, buffer size, and processing cost per call.

Media duration and processing duration must be measured separately:

mediaDuration = processedSamples / sampleRate
processingDuration = monotonicClockEnd - monotonicClockStart
RTF = processingDuration / mediaDuration

If 100 seconds of audio are processed in 12 seconds, RTF = 0.12. Media duration must be derived from the sample counter, while processing duration must be derived from a monotonic clock; wall-clock changes must not affect performance measurement.

A shared time contract

VAD, speaker selection, segment merging, ASR, JSON output, and the player may use different time representations. In a reliable design, their common primary unit is the sample index per channel. Milliseconds, PCM byte positions, frame numbers, and player ranges are derived from this unit.

For example, the byte equivalent of a sample offset in an interleaved stream is:

byteOffset = sampleOffset × channelCount × bytesPerSample

In a channel-separated mono file, it is:

monoByteOffset = sampleOffset × bytesPerSample

Time remains unchanged; only the physical data layout changes. Partial reads, channel count, final-frame padding, and processing delay must not alter the fundamental sample counter. A timestamp is the externally presented representation of the number of actual processed samples.