# Deterministic Time Axis in PCM Processing

> In PCM processing, media time is derived from the number of samples processed per channel rather than the wall clock. The same counter keeps stereo frames, partial reads, and segment boundaries on one time axis.

- Author: Muhammet Ali Köker
- Language: en
- Canonical: https://alikoker.com.tr/en/deterministic-time-axis-in-real-time-pcm-processing
- Translation: https://alikoker.com.tr/pcm-islemede-deterministik-zaman-ekseni
- Published: 2026-07-29T12:00:00+03:00
- Modified: 2026-09-02T13:23:29+03:00
- Verified: 2026-08-07T11:00:00+03:00
- Type: article

```text
t = n / fₛ
```

Here, `n` is the number of processed samples per channel and `fₛ` is the sampling rate. For example, in a 16 kHz stream, 16,000 samples equal exactly one second:

```text
t = 16000 / 16000 = 1 second
```

In a deterministic [PCM](/en/wiki/pcm) pipeline, the same input must produce the same frames, segment boundaries, and timestamps on every execution. The fundamental requirement is that the primary time state be a processed-sample counter rather than a wall clock.

This problem grew out of time-axis errors I encountered while segmenting PCM streams in real-time speech-processing systems. When partial reads, stereo interleaving, and model padding share the same counter, deviations that appear small in milliseconds can corrupt segment boundaries over long streams. I therefore use the sample-counter approach here to separate media time completely from wall-clock time.

## Separating samples, bytes, and channels

In 16-bit signed little-endian PCM, one sample per channel occupies two bytes:

```text
bytesPerSample = 2
```

At 16 kHz, a 10 ms mono frame contains 160 samples and 320 bytes:

```text
samplesPerFrame = 16000 × 10 / 1000 = 160
monoFrameBytes = 160 × 2 = 320
```

In stereo interleaved PCM, the data volume for the same duration is 640 bytes:

```text
stereoFrameBytes = 160 × 2 channels × 2 bytes = 640
```

Nevertheless, the time axis does not double. A stereo frame contains 320 total 16-bit values in the order `L₀, R₀, L₁, R₁, ...`, but each channel has only 160 time positions. Therefore, the counter must be updated only by the number of valid samples per channel:

```text
processedSamples += validSamplesPerChannel
```

A 640-byte stereo frame contains `640 / 2 = 320` 16-bit values and thus `320 / 2 = 160` time samples. The operation `processedSamples += 320` incorrectly records 10 ms of data as 20 ms. When channels are separated, the counter must likewise be incremented only once for each stereo frame.

## Producing media time from the sample counter

A system clock measures processing time. It does not identify a position in the media. One hour of audio read from a file may be processed in minutes. Network or queue delays may also cause one second of audio to be processed later. Therefore, `System.nanoTime()`, `DateTime.UtcNow`, or similar wall-clock measurements must not determine a segment's position within the audio.

[Frame](/en/wiki/frame) sample boundaries are derived directly from the counter:

```text
frameStart = processedSamples
frameEnd = frameStart + validSamples
processedSamples = frameEnd
```

The millisecond representation should be calculated only when it is exported:

```text
startMs = frameStart × 1000 / sampleRate
endMs = frameEnd × 1000 / sampleRate
```

For 16 kHz, the simplification `timeMs = sampleIndex / 16` applies; general code should nevertheless use `sampleRate` explicitly. The primary counter must be a `long`. At 16 kHz, a 32-bit counter can overflow after approximately 37.3 hours. The multiplication must also use `long` arithmetic:

```java
final long timeMs = (long) sampleIndex * 1000L / sampleRate;
```

Integer division rounds down. When start and end times are calculated using the same conversion from absolute sample positions, no artificial gaps or overlaps arise between adjacent segments. Total sample positions should be converted instead of rounding and summing individual frame durations.

## Partial reads and the final frame

A `read` call must not be assumed to fill the requested buffer completely. For example, a 320-byte mono frame may arrive in two reads of 120 and 200 bytes. Reading must continue until the buffer is full or the actual end of file is reached:

```text
offset = 0

while offset < frameBytes:
 n = read(buffer, offset, frameBytes - offset)

 if n < 0:
 break

 offset += n
```

In general stream abstractions, a `read` operation returning zero must additionally be assessed as a potential infinite-loop risk. The number of valid samples per channel is:

```text
validSamples = bytesRead / (bytesPerSample × channelCount)
```

For stereo 16-bit PCM, this is `bytesRead / 4`. However, the following condition must first be verified:

```text
bytesRead mod (bytesPerSample × channelCount) = 0
```

In a stereo 16-bit stream, one time sample occupies four bytes. Two or three bytes remaining at the end of a file indicate an incomplete channel sample; this data must not be silently padded or combined with previous buffer content.

The final frame may be shorter than a full frame. For example, a buffer capacity may be 160 samples while only 73 samples are valid at the end of the file. The algorithm must treat only those 73 samples as actual media data. Safe options are:

1. Process only valid samples if the algorithm supports them.
2. Explicitly zero-pad the final frame.
3. Manage remaining samples with a defined termination policy in algorithms that accept only full frames.

When zero padding is used, media and model-input lengths must be separated:

```text
mediaSamples = 73
processingSamples = 160
```

The time counter advances only by `mediaSamples`. Otherwise, up to one frame of artificial duration is appended to the end of every file.

## Half-open intervals and PCM layout

Segments should be represented with half-open sample intervals:

```text
[startSample, endSample)
```

When adjacent intervals are `[A, B)` and `[B, C)`, there are neither duplicate samples nor gaps. The same contract applies directly to PCM byte positions:

```text
byteStart = sampleStart × bytesPerSample × channelCount
byteEnd = sampleEnd × bytesPerSample × channelCount
```

This representation is compatible with file seeking, HTTP ranges, NIO buffer boundaries, and array indices.

When reconstructing a 16-bit little-endian sample, the low byte must be masked:

```java
final int sample = (buffer[index + 1] << 8) | (buffer[index] & 0xFF);
```

Because `byte` is signed in Java, failing to mask the low byte can cause a sign-extension error. In a stereo interleaved sample array, time order is preserved as follows:

```text
left[n] = stereo[2n]
right[n] = stereo[2n + 1]
```

Here, `stereo` is a 16-bit sample array; in a byte array, indices are additionally scaled by `bytesPerSample`.

## Frame duration and performance measurement

When [VAD](/en/wiki/voice-activity-detection) thresholds are defined by frame counts, they depend on frame duration. For example, in 10 ms frames, `confirmFrames = 4` means 40 ms and `silenceFrames = 30` means 300 ms. If frame duration is increased to 20 ms without changing the counters, these thresholds become 80 ms and 600 ms, respectively.

Where possible, thresholds should be defined in time or samples and converted to frame counts:

```text
confirmFrames =
 ceil(confirmMs × sampleRate /
 (1000 × samplesPerFrame))
```

Frame duration affects VAD decision frequency, speaker-change latency, segment-termination resolution, minimum detectable speech duration, buffer size, and processing cost per call.

Media duration and processing duration must be measured separately:

```text
mediaDuration = processedSamples / sampleRate
processingDuration = monotonicClockEnd - monotonicClockStart
RTF = processingDuration / mediaDuration
```

If 100 seconds of audio are processed in 12 seconds, `RTF = 0.12`. Media duration must be derived from the sample counter, while processing duration must be derived from a monotonic clock; wall-clock changes must not affect performance measurement.

## A shared time contract

VAD, speaker selection, segment merging, [ASR](/en/wiki/automatic-speech-recognition), JSON output, and the player may use different time representations. In a reliable design, their common primary unit is the sample index per channel. Milliseconds, PCM byte positions, frame numbers, and player ranges are derived from this unit.

For example, the byte equivalent of a sample offset in an interleaved stream is:

```text
byteOffset = sampleOffset × channelCount × bytesPerSample
```

In a channel-separated mono file, it is:

```text
monoByteOffset = sampleOffset × bytesPerSample
```

Time remains unchanged; only the physical data layout changes. Partial reads, channel count, final-frame padding, and processing delay must not alter the fundamental sample counter. A timestamp is the externally presented representation of the number of actual processed samples.

## Separating the Standard from the Derived Time Contract

G.711 defines PCM sample coding, while RTP defines how media timestamps relate to the media clock. The rule in this article—derive media time from processed sample count rather than wall-clock time—is not a sentence copied from either standard; it is an engineering consequence of carrying the sample/time relationship into a real-time processor.

At a fixed sample rate, the ideal media duration of `N` samples is `N / f_s`. That makes media position deterministic with respect to the stream: thread scheduling, I/O stalls, or wall-clock jumps do not move the media timeline. User-perceived processing latency, however, must still be measured against wall-clock time. “Media time” and “processing latency” should therefore not be derived from the same clock; keeping those two axes separate is the key design boundary.

## References

- **[1]** Henning Schulzrinne; Stephen L. Casner; Ron Frederick; Van Jacobson. (2003). RTP: A Transport Protocol for Real-Time Applications. RFC Editor. [doi:10.17487/RFC3550](https://doi.org/10.17487/RFC3550)
- **[2]** International Telecommunication Union. (1988). ITU-T Recommendation G.711 - Pulse Code Modulation (PCM) of Voice Frequencies. ITU-T. [URL](https://www.itu.int/rec/T-REC-G.711)

## Cite This Work

Köker, M. A. (2026). Deterministic Time Axis in PCM Processing. alikoker.com.tr. https://alikoker.com.tr/en/deterministic-time-axis-in-real-time-pcm-processing

- BibTeX: https://alikoker.com.tr/en/deterministic-time-axis-in-real-time-pcm-processing.bib
- RIS: https://alikoker.com.tr/en/deterministic-time-axis-in-real-time-pcm-processing.ris
- CSL-JSON: https://alikoker.com.tr/en/deterministic-time-axis-in-real-time-pcm-processing.csl.json