<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style id="owaParaStyle" type="text/css">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
P {margin-top:0;margin-bottom:0;}</style>
</head>
<body ocsi="0" fpstyle="1">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">Sorry about the drip-feeding of updates...<br>
<br>
I found that replacing stat by _stat64 in the Open function enables the read to continue. My program successfully compresses the big tiff. The resulting file is not viewable by Irfanview, but I am able to uncompress it again. It's possible that Irfanview
cannot handle tiff files of this size. I still need to figure out a way to check that all the files contain the correct data.<br>
<br>
The stat() function does not appear anywhere else in itkTIFFImageIO.cxx, so checking for big tiff is needed only in Open().<br>
<br>
Gib<br>
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<hr tabindex="-1">
<div style="direction: ltr;" id="divRpF553436"><font color="#000000" size="2" face="Tahoma"><b>From:</b> Community [community-bounces@itk.org] on behalf of Gib Bogle [g.bogle@auckland.ac.nz]<br>
<b>Sent:</b> Friday, 7 March 2014 5:09 p.m.<br>
<b>To:</b> Matt McCormick<br>
<b>Cc:</b> community@itk.org<br>
<b>Subject:</b> Re: [ITK Community] Windows TIFF size limit<br>
</font><br>
</div>
<div></div>
<div>
<div style="direction:ltr; font-family:Tahoma; color:#000000; font-size:10pt">The current failure when trying to read the big tiff file has me puzzled.
<br>
<br>
int TIFFReaderInternal::Open(const char *filename)<br>
{<br>
printf("TIFFReaderInternal::Open\n");<br>
this->Clean();<br>
printf("did Clean!\n");<br>
printf("filename: %s\n",filename);<br>
struct stat fs;<br>
if ( stat(filename, &fs) )<br>
{<br>
printf("stat(filename,%fs) is not 0\n");<br>
return 0;<br>
}<br>
...<br>
<br>
(my printf statements, obviously.) The call to stat() is returning a non-zero value, and the open fails here. If I comment out the block, the read is successful, and my test of compressing the image again seems to be OK, but again the file generated cannot
be opened (for the same reason, I presume). My problem is I do not know about stat. It looks as if the wrong info is written to the stat structure when the file is created. I'm guessing that something funny might happen to st_size when the file is very
big. <br>
<br>
In fact Wikipedia says:<br>
<br>
<p>The <a href="http://en.wikipedia.org/wiki/C_POSIX_library" title="C POSIX library" target="_blank">
C POSIX library</a> header <sys/stat.h>, found on <a href="http://en.wikipedia.org/wiki/POSIX" title="POSIX" target="_blank">
POSIX</a> and other <a href="http://en.wikipedia.org/wiki/Unix-like" title="Unix-like" target="_blank">
Unix-like</a> operating systems, declares the <code>stat()</code> functions, as well as related function called
<code>fstat()</code> and <code>lstat()</code>. The functions take a <i>struct stat</i> buffer argument, which is used to return the file attributes. On success, the functions return zero, and on error, −1 is returned and
<a href="http://en.wikipedia.org/wiki/Errno" title="Errno" class="mw-redirect" target="_blank">
errno</a> is set appropriately.</p>
<p>The stat() and lstat() functions take a <a href="http://en.wikipedia.org/wiki/Filename" title="Filename" target="_blank">
filename</a> argument. If the file name is a <a href="http://en.wikipedia.org/wiki/Symbolic_link" title="Symbolic link" target="_blank">
symbolic link</a>, stat() returns attributes of the eventual target of the link, while lstat() returns attributes of the link itself. The fstat() function takes a
<a href="http://en.wikipedia.org/wiki/File_descriptor" title="File descriptor" target="_blank">
file descriptor</a> argument instead, and returns attributes of the file that it identifies.</p>
<p>The family of functions was extended to implement <a href="http://en.wikipedia.org/wiki/Large_file_support" title="Large file support" target="_blank">
large file support</a>. Functions named stat64(), lstat64() and fstat64() return attributes in a
<i>struct stat64</i> structure, which represents file sizes with a 64-bit type, allowing the functions to work on files 2 GiB and larger. When the _FILE_OFFSET_BITS
<a href="http://en.wikipedia.org/wiki/C_macro" title="C macro" class="mw-redirect" target="_blank">
macro</a> is defined to 64, these 64-bit functions are available under the original names.</p>
<br>
So it looks as if stat64() should be used for the big files, not stat(). I guess this requires detection of a big file, and probably implies changes in a few places.<br>
<br>
Gib<br>
<div style="font-family:Times New Roman; color:#000000; font-size:16px">
<hr tabindex="-1">
<div id="divRpF369136" style="direction:ltr"><font color="#000000" size="2" face="Tahoma"><b>From:</b> Matt McCormick [matt.mccormick@kitware.com]<br>
<b>Sent:</b> Friday, 7 March 2014 4:44 p.m.<br>
<b>To:</b> Gib Bogle<br>
<b>Cc:</b> Brian Helba; community@itk.org<br>
<b>Subject:</b> Re: [ITK Community] Windows TIFF size limit<br>
</font><br>
</div>
<div></div>
<div>
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Thu, Mar 6, 2014 at 10:25 PM, Gib Bogle <span dir="ltr">
<<a href="mailto:g.bogle@auckland.ac.nz" target="_blank">g.bogle@auckland.ac.nz</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left-width:1px; border-left-color:rgb(204,204,204); border-left-style:solid; padding-left:1ex">
<div>
<div style="direction:ltr; font-size:10pt; font-family:Tahoma">Thanks. I have tracked down the reason for the crash reading this compressed tiff in Windows. The compression is DEFLATE (AKA ZIP). The function TIFFReaderInternal::CanRead() in itkTIFFImageIO.cxx
checks for COMPRESSION_NONE, COMPRESSION_PACKBITS and COMPRESSION_LZW, although COMPRESSION_DEFLATE is covered elsewhere in the code. I added this line:<br>
|| this->m_compression == COMPRESSION_DEFLATE<br>
and now the read proceeds successfully, as far as I can tell, and my program creates an uncompressed 4.5GB file. This is progress!<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Well done! Any patches [1] are greatly appreciated :-)</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left-width:1px; border-left-color:rgb(204,204,204); border-left-style:solid; padding-left:1ex">
<div>
<div style="direction:ltr; font-size:10pt; font-family:Tahoma"><br>
Unfortunately the file that is created does not identify itself as a valid TIFF.
<br>
Irfanview says: Decode error! Invalid or unsupported TIF file. <br>
When I try to read it with ITK, the message is:<br>
File: ...itkImageFileReader.hxx<br>
Line: 143<br>
Description: Could not create IO object for file preeq_u.tif<br>
Tried to create one of the following:<br>
JPEGImageIO<br>
...<br>
TIFFImageIO<br>
...<br>
You probably failed to set a file suffix, or set the suffix to an unsupported type.<br>
<br>
Well, the suffix is obviously correct, so there is another problem with the file.<br>
<br>
The saga continues...<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Keep up the good fight :-),</div>
<div>Matt</div>
<div><br>
</div>
<div><br>
</div>
<div> [1] <a href="http://insightsoftwareconsortium.github.io/ITKBarCamp-doc/CommunitySoftwareProcess/SubmitAPatchToGerrit/index.html" target="_blank">http://insightsoftwareconsortium.github.io/ITKBarCamp-doc/CommunitySoftwareProcess/SubmitAPatchToGerrit/index.html</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>