[colug-432] Troubleshooting Suggestions: File Truncation Issue
jeff.frontz at gmail.com
Tue Jan 12 00:49:19 EST 2016
I'm not sure of the latest implementation of tmpfs, but at least some of
them still have the possibility of actual physical I/O (see
https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt ). In any
case, you can reduce the likelihood, but the underlying problem still
exists -- trying to enforce/rely on temporal ordering across the driver(s)
and kernel for an ordering that generic Unix/Linux offers no guarantee.
File I/O in the kernel has buffer management, write/read
scheduling/optimization, journaling, etc. and can lag behind the progress
of the client process-- so much so that, as in this case, the client can
sometimes "complete" its I/O and ask for the file's status before the
kernel has actually completed processing of the original I/O; the stat()
request asks "what is the true state of the file?" (which is potentially at
a time when the I/O is still pending in the kernel); the kernel replies
only with that it knows for sure at the time of the request. For a
pathological case, imagine that the filesystem journaling repeatedly
encounters a retryable disk write error-- it will eventually complete, but
until it does, the state of the file is what the kernel has already been
successfully written/journaled; that is, while this retry is happening, the
state of the file[system] is still what it was before the
still-being-journaled operation started.
To put it in transaction processing terms -- the file I/O is one
transaction; the request for file size (via stat()) is a second
transaction. There is no guarantee/feedback from the kernel about when the
first transaction is complete; the only feedback is that the transaction
has been queued. The probability of completion of the first transaction
quickly approaches one as time passes, but there is never a 100% guarantee
that the first transaction has completed by the time the second transaction
Note that this problem would be more likely to manifest itself in a
multi-processor environment (where the kernel I/O was happening on one core
and the user/client process was running on another core -- thus allowing
the client to make progress while the kernel was still working).
On Mon, Jan 11, 2016 at 8:14 PM, Brian <bnmille at gmail.com> wrote:
> If Jeff is correct, you might get around the issue by doing the writes to
> a tmpfs RAM disk, rather than to a physical disk.
> On Jan 10, 2016 10:32 AM, "Jeff Frontz" <jeff.frontz at gmail.com> wrote:
>> On Sat, Jan 9, 2016 at 11:13 PM, William E. T. <linux.hacker at gmail.com>
>>> 1. Opens a new file
>>> 2. Writes to the file
>>> 3. Closes the file
>>> 4. Creates a link to its real name
>>> 5. Removes the original file
>>> 6. stats the file to find a zero size
>> Without knowing the details of the particular kernel or filesystem/RAID
>> drivers involved, there is the potential for a race condition between 2/3
>> and 6.
>> On Unix/Linux, there is no guarantee that data has ever made it to the
>> "disk" (which could include the various levels of drivers/caches in between
>> your program and the actual hardware).
>> From the close(2) man page notes:
>> " A successful close does not guarantee that the data has been
>> successfully saved to disk, as the kernel defers writes. It is not common
>> for a filesystem to flush the buffers when the stream is closed. If you
>> need to be sure that the data is physically stored use fsync(2). (It will
>> depend on the disk hardware at this point.)"
>> The only guarantee of a non-zero size would be if you called fstat (note
>> the "f") on the still-open file descriptor (used in 1/2/3 above).
>> You can throw in a call to sync(2) to after the close(), though depending
>> on how your I/O stack is composed even that might not be perfect.
>> colug-432 mailing list
>> colug-432 at colug.net
> colug-432 mailing list
> colug-432 at colug.net
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the colug-432