Storage Durability Boundaries: fsync, Directory fsync, Journaling and Device Caches
Why atomic visibility and durability are different guarantees, and how fsync, directory metadata, journaling, write caches and NFS alter crash-consistency boundaries.
A file becoming visible under its final name is not the same guarantee as finding that name and content intact after power loss. Atomic rename can make a namespace transition indivisible to readers, while fsync attempts to push dirty file data and required metadata through the persistence path. Crash-consistency design should not collapse those properties into one generic idea of a "safe write."
Visibility and durability
A common publication sequence writes data to a temporary file on the same filesystem, flushes/fsyncs that file and renames it to the final path. Readers then avoid observing a partially written target. The rename itself modifies a directory entry, however. If the namespace change must survive power loss, the durability boundary of the parent directory also matters; directory fsync is the usual mechanism on Linux/Unix systems.
A simplified sequence is:
write temp
fsync(temp)
rename(temp, final)
fsync(parent_directory)On POSIX systems, fsync() on a file descriptor and fsync() on a directory descriptor address different persistence objects. Filesystem and remote-storage implementations differ in their error and support behavior. A successful test on one kernel, mount configuration, device cache, or NFS server does not establish the same guarantee for all others.
The exact guarantee still depends on filesystem, mount policy, kernel and storage-device flush semantics. One application call cannot by itself prove the physical behavior of the entire storage chain.
What journaling does and does not guarantee
Journaling filesystems record metadata or data updates in an ordered form to simplify structural recovery after a crash. "The filesystem is journaled" does not mean that application data is durable at the moment the application expects. Policies such as ordered or writeback modes can change how metadata and user data reach stable storage.
An application protocol should therefore define explicit durability points rather than relying on a vague assumption about the filesystem crash model. Transaction logs, index snapshots, model artifacts and expensive processing results are typical cases where the distinction matters.
Device caches and the flush chain
After a write, data may pass from user space to the kernel page cache, block layer, controller cache and drive cache. "Completed" can mean something different at each layer. A power-protected cache and a volatile write-back cache do not provide the same guarantee. Practical durability analysis therefore includes storage-device cache and flush behavior, not just the application API.
NFS and remote-filesystem boundaries
Local-filesystem intuition should not be transferred blindly to NFS. Client caching, server COMMIT behavior, protocol version and the server's stable-storage policy can change the interval between visibility and durability. One client observing a new file does not prove that every other client has the same view or that the data is protected against loss.
For that reason, atomic publication in file processing over NFS should not be reduced to rename atomicity. Writer/reader protocol, temporary-file placement, retries and idempotency all matter.
Why crash testing matters
Durability code cannot be validated adequately by clean shutdown tests. Process termination, reboot, network loss and controlled fault injection are more relevant. Recovery checks should verify not only that a file exists, but whether the old or new version is valid, whether metadata is structurally consistent and whether rerunning the operation is safe.
A durability test must distinguish failure points. Forced application termination, operating-system failure, and sudden loss of device power can discard different buffers. Writing and closing the temporary file, synchronizing its data, renaming it to the target, and synchronizing the parent directory provide distinct fault-injection boundaries. After restart, validation must cover not merely filename existence but whether the published version is complete and internally consistent.
Durable writing is a cross-layer protocol rather than a single API call. Application, kernel, filesystem, network protocol and physical storage participate in the same persistence contract. Once that boundary is explicit, atomicity, visibility and durability can be designed independently instead of being treated as synonyms.
Acceptance Condition After Failure
A durable-publication experiment defines success by the version a reader may observe after restart. Either the old version or the complete new version may be acceptable, while a partial new version may not be. This condition is tested separately for process crash, operating-system crash and power loss. Rename preconditions, such as placing the temporary file on the same filesystem as the target, are part of the experimental record.
On Linux, a successful file fsync() does not by itself make the containing directory entry durable; the directory may require a separate synchronization. Write, synchronization and close failures must not be converted into success. NFS and other storage architectures require separate assessment of protocol and server behavior rather than assuming guarantees from a local-filesystem experiment.
References
