[colug-432] Troubleshooting Suggestions: File Truncation Issue

[Jeff Frontz]

I'm not sure of the latest implementation of tmpfs, but at least some of
them still have the possibility of actual physical I/O (see
https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt ). In any
case, you can reduce the likelihood, but the underlying problem still
exists -- trying to enforce/rely on temporal ordering across the driver(s)
and kernel for an ordering that generic Unix/Linux offers no guarantee.

File I/O in the kernel has buffer management, write/read
scheduling/optimization, journaling, etc. and can lag behind the progress
of the client process-- so much so that, as in this case, the client can
sometimes "complete" its I/O and ask for the file's status before the
kernel has actually completed processing of the original I/O; the stat()
request asks "what is the true state of the file?" (which is potentially at
a time when the I/O is still pending in the kernel); the kernel replies
only with that it knows for sure at the time of the request.   For a
pathological case, imagine that the filesystem journaling repeatedly
encounters a retryable disk write error-- it will eventually complete, but
until it does, the state of the file is what the kernel has already been
successfully written/journaled; that is, while this retry is happening, the
state of the file[system] is still what it was before the
still-being-journaled operation started.

To put it in transaction processing terms -- the file I/O is one
transaction; the request for file size (via stat()) is a second
transaction.  There is no guarantee/feedback from the kernel about when the
first transaction is complete; the only feedback is that the transaction
has been queued.  The probability of completion of the first transaction
quickly approaches one as time passes, but there is never a 100% guarantee
that the first transaction has completed by the time the second transaction
is processed.

Note that this problem would be more likely to manifest itself in a
multi-processor environment (where the kernel I/O was happening on one core
and the user/client process was running on another core -- thus allowing
the client to make progress while the kernel was still working).

Jeff


On Mon, Jan 11, 2016 at 8:14 PM, Brian <bnmille at gmail.com> wrote:

> If Jeff is correct, you might get around the issue by doing the writes to
> a tmpfs RAM disk, rather than to a physical disk.
> On Jan 10, 2016 10:32 AM, "Jeff Frontz" <jeff.frontz at gmail.com> wrote:
>
>>
>> On Sat, Jan 9, 2016 at 11:13 PM, William E. T. <linux.hacker at gmail.com>
>> wrote:
>>
>>> 1.   Opens a new file
>>> 2.  Writes to the file
>>> 3.  Closes the file
>>> 4.  Creates a link to its real name
>>> 5.  Removes the original file
>>> 6.  stats the file to find a zero size
>>>
>>
>> Without knowing the details of the particular kernel or filesystem/RAID
>> drivers involved, there  is the potential for a race condition between 2/3
>> and 6.
>>
>> On Unix/Linux, there is no guarantee that data has ever made it to the
>> "disk" (which could include the various levels of drivers/caches in between
>> your program and the actual hardware).
>>
>> From the close(2) man page notes:
>>
>> " A  successful  close  does  not  guarantee that the data has been
>> successfully saved to disk, as the kernel defers writes. It is not common
>> for a filesystem to flush the buffers when the stream is closed. If you
>> need to be sure that the data is physically stored use fsync(2).  (It will
>> depend on the disk hardware at this point.)"
>>
>>
>>
>> The only guarantee of a non-zero size would be if you called fstat (note
>> the "f") on the still-open file descriptor (used in 1/2/3 above).
>>
>> You can throw in a call to sync(2) to after the close(), though depending
>> on how your I/O stack is composed even that might not be perfect.
>>
>> Jeff
>>
>>
>>
>> _______________________________________________
>> colug-432 mailing list
>> colug-432 at colug.net
>> http://lists.colug.net/mailman/listinfo/colug-432
>>
>>
> _______________________________________________
> colug-432 mailing list
> colug-432 at colug.net
> http://lists.colug.net/mailman/listinfo/colug-432
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.colug.net/pipermail/colug-432/attachments/20160112/e0c15c4d/attachment.html