<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi,<div><br></div><div>OK, I recall this problem a little more now.</div><div><br></div><div>I think there is a problem internally with HDF5 library internally.</div><div><br></div><div>But here is another work around you may like better. Each time you set a file name the itk::ImageFileReader tries all registered ImageIO to see which one can read the file. One of these happens to be HDF5 which has this problem. You can by pass this by explicitly setting the ImageIO to the file reader, with the ImageFileReader::SetImageIO() method. In your case that should be itk::NiftiImageIO.</div><div><br></div><div>This should have additional performance benefits too, due to not searching for the correct ImageIO.</div><div><br></div><div>Brad</div><div><br><div><div>On Nov 14, 2013, at 11:41 AM, Alexander Schmidt-Richberg <<a href="mailto:schmidtrichberg@me.com">schmidtrichberg@me.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div><div>Hi Brad</div><div><br></div><div>Thanks for the quick reply. I'm not explicitly using any streaming, just the common ImageFileReader. I was intending to circumvent allocating new memory space for each image but I can do without ;)</div><div><br></div><div>The error message is attached below. </div><div><br></div><div>Cheers</div><div>Alex</div><div><br></div><div><div>HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:</div><div> #000: ../ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5F.c line 794 in H5Fis_hdf5(): unable to open file</div><div> major: Low-level I/O</div><div> minor: Unable to initialize object</div><div> #001: ../ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5FD.c line 1086 in H5FD_open(): open failed</div><div> major: Virtual File Layer</div><div> minor: Unable to initialize object</div><div> #002: ../ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5FDsec2.c line 348 in H5FD_sec2_open(): unable to open file: name = '../ADNI_128_S_0167_MR_MPR-R__GradWarp__B1_Correction__N3__Scaled_Br_20080603173812534_S50416_I108232.nii.gz', errno = 24, error message = 'Too many open files', flags = 0, o_flags = 0</div><div> major: File accessability</div><div> minor: Unable to open file</div><div>terminate called after throwing an instance of 'itk::ImageFileReaderException'</div><div> what(): ../ITK/Modules/IO/ImageBase/include/itkImageFileReader.hxx:143:</div><div> Could not create IO object for file ../ADNI_128_S_0167_MR_MPR-R__GradWarp__B1_Correction__N3__Scaled_Br_20080603173812534_S50416_I108232.nii.gz</div><div>The file couldn't be opened for reading. </div><div>Filename: ../ADNI_128_S_0167_MR_MPR-R__GradWarp__B1_Correction__N3__Scaled_Br_20080603173812534_S50416_I108232.nii.gz</div><div><br></div><br>Am 14. November 2013 um 17:33 schrieb Bradley Lowekamp <<a href="mailto:blowekamp@mail.nih.gov">blowekamp@mail.nih.gov</a>>:<br><br></div><div><blockquote type="cite"><div class="msg-quote" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; " data-mce-style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">The issue below is with HDF5 and streaming. I am not sure that you have the same one, it may similar be in nifti with files left open changing files and keeping the reader.<div><br></div><div>Are you performing streaming?</div><div>What is the error message you are getting?</div><div><br></div><div>I think your solution of creating a new ImageFilterReader for each file is best. That's the way I have done it.</div><div><br></div><div>Brad</div><div><br></div><div><br></div><div><br><div><div>On Nov 14, 2013, at 11:22 AM, Alexander Schmidt-Richberg <<a href="mailto:schmidtrichberg@me.com" data-mce-href="mailto:schmidtrichberg@me.com">schmidtrichberg@me.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div><div>Hi all,</div><div><br></div><div>I wonder if this problem has ever been solved/fixed.</div><div>I'm experiencing (what I think is) exactly the same weird bug when reading a large number of nifti images.</div><div><br></div><div>The first option in the code below (sampleRegion = true) produces the same message after approx. 200 images, the second one (= false) works fine... Instantiating a new reader in every loop also works. </div><div><br></div><div>Running the current git version.</div><div><br></div><div>Cheers</div><div>Alex</div><div><br></div><div>for( unsigned int subject = 0; subject < numSubjects; ++subject )</div><div>{</div><div> imageReader->SetFileName( filenames[subject].c_str() );</div><div> imageReader->Update();</div><div><br></div><div> if( sampleRegion )</div><div> {</div><div> IteratorType it( imageReader->GetOutput(), samplingRegion );</div><div> for( it.GoToBegin(); !it.IsAtEnd(); ++it )</div><div> {</div><div> FeatureValueType value = it.Get(); // ...</div><div> }</div><div> }</div><div> else</div><div> {</div><div> IteratorType it( imageReader->GetOutput(), imageReader->GetOutput()->GetRequestedRegion() );</div><div> for( it.GoToBegin(); !it.IsAtEnd(); ++it )</div><div> {</div><div> FeatureValueType value = it.Get(); // ...</div><div> }</div><div> }</div><div>}</div><div><br></div><br>Am 08. März 2012 um 17:00 schrieb Bradley Lowekamp <<a href="mailto:blowekamp@mail.nih.gov" data-mce-href="mailto:blowekamp@mail.nih.gov">blowekamp@mail.nih.gov</a>>:<br><br></div><div><blockquote type="cite"><div class="msg-quote" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; " data-mce-style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hello Matthias,<div><br></div><div>You are correct that the documentation for the ImageFileReader::EnlargeOutputRequestedRegion should be updated. However only certain ImageIO file formats support the "stream" or roi reading. The one derived from the StreamingImageIO base along with nift, and maybe others support this functionality. You'd have to look at ImageIO to know for sure.</div><div><br></div><div>Your usage of the ImageFileReader->ExtactImageFilter was scenario to motivate for the recent optimization of making the Extract filter run InPlace. That is very nice usage!</div><div><br></div><div>It looks like you have encountered a bug. Most likely some one is not closing a file, and you are running out of file descriptors for the process.</div><div><br></div><div><br></div><div>Here is the procedure for reporting a bug:</div><div><a href="http://www.itk.org/Wiki/ITK/Procedure_for_Contributing_Bug_Fixes" data-mce-href="http://www.itk.org/Wiki/ITK/Procedure_for_Contributing_Bug_Fixes">http://www.itk.org/Wiki/ITK/Procedure_for_Contributing_Bug_Fixes</a></div><div><br></div><div>If you have a "working" bit of code which exhibits the bug, that would help us reproduce the issue and should be included. This will help expedite the solution. If you have a patch to fix it even better.</div><div><br></div><div>I am suspicious that the new HDF5 ImageIO is not closing the file in the CanRead method.</div><div><br></div><div>Brad</div><div><br><div><div>On Mar 8, 2012, at 10:31 AM, Matthias Schneider wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">Hi.<br><br>I'm trying to extract subregions of 2D image files. Ideally, I'd like to <br>read the required data only.<br>In the ITK doc I came across the comment in<br>itk::ImageFileReader::EnlargeOutputRequestedRegion() saying that:<br>"ImageFileReader cannot currently read a portion of an image (since the <br>ImageIO objects cannot read a portion of an image)"<br><br>I'm not quite sure if this still holds. After a quick experiment, I was <br>pretty much confident that the readers are in fact able to read portions:<br><br>const std::string fname = "/path/to/my/image.nii"; // [2048x2048 px]<br>typedef itk::Image<Pixel8uType, 2> ImageType;<br>typedef itk::ImageFileReader<ImageType> ReaderType;<br><br>ImageType::RegionType roi;<br>roi.SetSize(0, 512);<br>roi.SetSize(1, 512);<br><br>for (int i = 0; i < N; ++i) {<br> std::cout << i << std::endl;<br> ReaderType::Pointer reader = ReaderType::New();<br> reader->SetFileName(fname);<br> reader->GetOutput()->SetRequestedRegion(roi);<br> reader->Update();<br>}<br><br>At first glance, this works perfectly fine (I checked the buffered data <br>chunk) and runs much faster than reading the entire slice. Choosing N <br>very large (say 1024), the program crashes since the reader seems to not <br>close the file descriptors properly. I get the following error:<br><br>HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:<br> #000: [...]/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5F.c line <br>794 in H5Fis_hdf5(): unable to open file<br> major: Low-level I/O<br> minor: Unable to initialize object<br> #001: [...]/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5FD.c line <br>1086 in H5FD_open(): open failed<br> major: Virtual File Layer<br> minor: Unable to initialize object<br> #002: [...]/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5FDsec2.c <br>line 348 in H5FD_sec2_open(): unable to open file: name = <br>'/path/to/my/image.nii', errno = 24, error message = 'Too many open <br>files', flags = 0, o_flags = 0<br> major: File accessability<br> minor: Unable to open file<br>terminate called after throwing an instance of <br>'itk::ImageFileReaderException'<br> what(): <br>[...]/ITK/Modules/IO/ImageBase/include/itkImageFileReader.hxx:143:<br> Could not create IO object for file /path/to/my/image.nii<br>The file couldn't be opened for reading.<br>Filename: /path/to/my/image.nii<br><br><br>I also tried to explicitly use the ExtractImageFilter:<br><br>[... same as above ...]<br>typedef itk::ExtractImageFilter<ImageType, ImageType> ExtractFilterType;<br><br>for (int i = 0; i < N; ++i) {<br> std::cout << i << std::endl;<br> ReaderType::Pointer reader = ReaderType::New();<br> reader->SetFileName(fname);<br> // reader->Update();<br><br> ExtractFilterType::Pointer extractor = ExtractFilterType::New();<br> extractor->SetInput(reader->GetOutput());<br> extractor->SetExtractionRegion(roi);<br> extractor->InPlaceOn(true);<br> extractor->Update();<br>}<br><br>Without the reader->Update() the program still crashes when choosing N <br>sufficiently large (~1024). Updating the reader first of course reads <br>the entire slice (what I really want to avoid).<br>So does this mean that the comment in the manual still holds and the <br>ImageIO is not able to correctly handle this case even though the <br>buffered data chunks are correctly imported?! This confuses me a bit or <br>maybe I'm missing something else?!<br><br>I'm happy about any comments or suggestions.<br><br>Thanks,<br>Matthias<br>_____________________________________<br>Powered by <a href="http://www.kitware.com/" data-mce-href="http://www.kitware.com/">www.kitware.com</a><br><br>Visit other Kitware open-source projects at<br><a href="http://www.kitware.com/opensource/opensource.html" data-mce-href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><br><br>Kitware offers ITK Training Courses, for more information visit:<br><a href="http://www.kitware.com/products/protraining.html" data-mce-href="http://www.kitware.com/products/protraining.html">http://www.kitware.com/products/protraining.html</a><br><br>Please keep messages on-topic and check the ITK FAQ at:<br><a href="http://www.itk.org/Wiki/ITK_FAQ">http://www.itk.org/Wiki/ITK_FAQ</a><br><br>Follow this link to subscribe/unsubscribe:<br>http://www.itk.org/mailman/listinfo/insight-users<br></blockquote></div><br><div><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; " data-mce-style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div style="margin: 0px; " data-mce-style="margin: 0px;"><span style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;" data-mce-style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;">========================================================</span></div><div style="margin: 0px; " data-mce-style="margin: 0px;"><span style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;" data-mce-style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;">Bradley Lowekamp<span class="Apple-converted-space"> </span><span class="Apple-converted-space"> </span></span></div><div style="margin: 0px; " data-mce-style="margin: 0px;"><span style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;" data-mce-style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;">Medical Science and Computing for</span></div><div style="margin: 0px; " data-mce-style="margin: 0px;"><span style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;" data-mce-style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;">Office of High Performance Computing and Communications</span></div><div style="margin: 0px; " data-mce-style="margin: 0px;"><span style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;" data-mce-style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;">National Library of Medicine<span class="Apple-converted-space"> </span></span></div><div style="margin: 0px; " data-mce-style="margin: 0px;"><span style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;" data-mce-style="font-style: normal; font-variant: normal; font-weight: normal; font-size: small; line-height: normal; font-family: Helvetica;"><a href="mailto:blowekamp@mail.nih.gov" data-mce-href="mailto:blowekamp@mail.nih.gov">blowekamp@mail.nih.gov</a></span></div></div><br class="Apple-interchange-newline"></div><br></div><div class="_stretch">_____________________________________<br> Powered by <a href="http://www.kitware.com/" data-mce-href="http://www.kitware.com/">www.kitware.com</a><br> <br> Visit other Kitware open-source projects at<br> <a href="http://www.kitware.com/opensource/opensource.html" data-mce-href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><br> <br> Kitware offers ITK Training Courses, for more information visit:<br> <a href="http://www.kitware.com/products/protraining.html" data-mce-href="http://www.kitware.com/products/protraining.html">http://www.kitware.com/products/protraining.html</a><br> <br> Please keep messages on-topic and check the ITK FAQ at:<br> <a href="http://www.itk.org/Wiki/ITK_FAQ" data-mce-href="http://www.itk.org/Wiki/ITK_FAQ">http://www.itk.org/Wiki/ITK_FAQ</a><br> <br> Follow this link to subscribe/unsubscribe:<br> <a href="http://www.itk.org/mailman/listinfo/insight-users" data-mce-href="http://www.itk.org/mailman/listinfo/insight-users">http://www.itk.org/mailman/listinfo/insight-users</a><br></div></div></blockquote></div></blockquote></div><br></div></div></blockquote></div></div></blockquote></div><br></div></body></html>