[Insight-users] [ome-users] bioformats and ITK

Mark Hiner hiner at wisc.edu
Wed May 22 15:00:28 EDT 2013 

Hi Joe,

 Just wanted to send you an update... I eventually realized it wasn't a
Debug/Release issue, but rather a classpath separator issue. The Linux/OSX
separator was hard-coded in the SCIFIO ImageIO.

 So the scifio-imageio
<https://github.com/scifio/scifio-imageio>repository has been updated
to choose the correct separator for OSX or
Windows. I also filed this bug fix
<http://review.source.kitware.com/#/c/11375/>updating ITK to download the
updated scifio-imageio commit.

 Also, while looking through the code, I had forgotten that the JVM
classpath (where those jars need to be) isn't pulled from the PATH... it's
taken from a special environment variable: SCIFIO_PATH. If you set
SCIFIO_PATH to the directory containing the itk-bridge jar and
loci_tools.jar (build/lib/jars) it should work without the update.
(*should*work..)

 Note that the SCIFIO_PATH is prepended onto each jar dependency.

Anyway, I hope this resolves everything for you. If you run into more
trouble, let me know.

Thanks,
Mark


On Fri, May 17, 2013 at 3:39 PM, Mark Hiner <hiner at wisc.edu> wrote:

> Joe,
>
>  That's great! It's almost certainly an issue of the jars being put in the
> wrong place because of the Debug/Release addition on Windows.
>
>  I confirmed that it doesn't work on Windows :-p I think it's in the
> DownloadSCIFIO.cmake.in which downloads to ${jarsDownloadDirectory}...
> but the jars are installed from
> ${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/${jarsDownloadDirectory} which I'm
> assuming points to lib/Debug/jars, which doesn't exist.
>
> Unfortunately I don't have time to verify that fixes things today. I'll
> try to test it on the weekend or Monday.
>
>
> On Fri, May 17, 2013 at 2:29 PM, Joe Ping-Lin Hsiao <phsiao at cs.unc.edu>wrote:
>
>> Mark,
>>
>> Just want to let you know that I repeated the same steps on my macbook
>> with Mac OSX 10.8.3, and program works fine.
>>
>> Joe
>>
>>
>> On Fri, May 17, 2013 at 10:45 AM, Mark Hiner <hinerm at gmail.com> wrote:
>>
>>> Joe,
>>>
>>>  That's odd.. not sure why manually adding them to the classpath didn't
>>> work. I will test locally on a Windows 7 box and let you know if I can
>>> reproduce and/or resolve the issue.
>>>
>>> Thanks for reporting it, hopefully we can get it working for you soon.
>>>
>>> - Mark
>>>
>>>
>>> On Thu, May 16, 2013 at 4:30 PM, Joe Ping-Lin Hsiao <phsiao at cs.unc.edu>wrote:
>>>
>>>> Mark,
>>>>
>>>> Yes, I was using the command "ITKIOSCIFIOTestDriver
>>>> itkSCIFIOImageIOTest inputfile outputfile".
>>>>
>>>> I am on Windows 7. The two jars (loci_tools.jar and
>>>> scifio-itk-bridge-1.0.0.jar) are in ITK/lib/jars.
>>>>
>>>> I manually added the path to classpath, and my classpath is now:
>>>> CLASSPATH=.;C:\Program Files
>>>> (x86)\Java\jre6\lib\ext\QTJava.zip;D:\Joe\dev\ITK\lib\jars
>>>>
>>>> But I still have no luck.
>>>>
>>>> After running the command, I first get a dialog of
>>>>  "Could not find the main class: loci.scifio.itk.SCIFIOITKBridge.
>>>> Program will exit."
>>>> Please see attached image.
>>>>
>>>> And there are also error messages in the console:
>>>>
>>>> D:\Joe\dev\ITK\bin\Release>ITKIOSCIFIOTestDriver itkSCIFIOImageIOTest
>>>> C:\Users\phsiao\Desktop\itk-scifio-build\Release\
>>>> chZT.lsm C:\Users\phsiao\Desktop\itk-scifio-build\Release\abc.tif
>>>> reader->GetUseStreaming(): 1
>>>> done checking streaming usage
>>>> itk::ExceptionObject (0000000000A4EF10)
>>>> Location: "unknown"
>>>> File: .\itkSCIFIOImageIO.cxx
>>>> Line: 172
>>>> Description: itk::ERROR: SCIFIOImageIO(0000000002A5C080): SCIFIOImageIO
>>>> exited abnormally. java.lang.NoClassDefFoundErr
>>>> r: loci/scifio/itk/SCIFIOITKBridge
>>>> Caused by: java.lang.ClassNotFoundException:
>>>> loci.scifio.itk.SCIFIOITKBridge
>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>>>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>>> Exception in thread "main"
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, May 16, 2013 at 4:46 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>
>>>>> Hi Joe,
>>>>>
>>>>>  Hmm.. well that is the correct package specification. What was the
>>>>> exact command you used? I'm assuming it was something like
>>>>>
>>>>> "ITKIOSCIFIOTestDriver itkSCIFIOImageIOTest inputfile outputfile"
>>>>>
>>>>> ?
>>>>>
>>>>> Anyway it sounds like the scifio-itk-bridge jar isn't on your
>>>>> classpath. I just tested a clean build off ITK.git master (commit
>>>>> 2e056e0cee1271d3080a9bfc4f7353c1d9165ef9) and it worked for me... but
>>>>> that's on OSX.
>>>>>
>>>>> What's in your build/lib/jars directory? It should have 2 things:
>>>>> loci_tools.jar
>>>>> scifio-itk-bridge-1.0.0.jar
>>>>>
>>>>> It's possible that they aren't being added to the classpath correctly
>>>>> though, if on Windows there's another directory level (e.g. is it
>>>>> build/Release/lib/jars ?). So if they are in a different directory, let me
>>>>> know.
>>>>>
>>>>> If the jars are there, try manually adding them to your classpath for
>>>>> now and let me know how that goes. If they're not then it's a problem with
>>>>> the download script.
>>>>>
>>>>> You can download the jars manually from:
>>>>>
>>>>> http://cvs.openmicroscopy.org.uk/snapshots/bioformats/4.4.5/loci_tools.jar
>>>>>
>>>>> http://jenkins.imagej.net/view/SCIFIO/job/SCIFIOITKBridge/lastSuccessfulBuild/artifact/target/scifio-itk-bridge-1.0.0.jar
>>>>>
>>>>> That's all I can think of at the moment...
>>>>>
>>>>> - Mark
>>>>>
>>>>>
>>>>> On Thu, May 16, 2013 at 1:02 PM, Joe Ping-Lin Hsiao <phsiao at cs.unc.edu
>>>>> > wrote:
>>>>>
>>>>>> Hi Mark,
>>>>>>
>>>>>> Glad to see SCIFIO is now becoming ITK's remote module. Thanks for
>>>>>> your continuous working.
>>>>>> I did a clean git clone of the latest ITK, and configure it with
>>>>>>
>>>>>>    Fetch_SCIFIO
>>>>>>    BUILD_TESTING
>>>>>>
>>>>>> set to ON. Then I run ITKIOSCIFIOTestDriver.exe on an image file, but
>>>>>> I got the error message
>>>>>>
>>>>>> "Could not find the main class: loci.scifio.itk.SCIFIOITKBridge.
>>>>>> Program will exit."
>>>>>>
>>>>>> Do I miss setting some path, or it still requires Bio-Formats?
>>>>>>
>>>>>> Thanks,
>>>>>> Joe
>>>>>>
>>>>>>
>>>>>> On Fri, Apr 26, 2013 at 2:43 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>>>
>>>>>>> Hi again Joe,
>>>>>>>
>>>>>>>  I noticed a couple things that may help you: in the
>>>>>>> itkVectorImageSCIFIOImageIOTest you sent me (and in all the test classes I
>>>>>>> mentioned in my last e-mail) the PixelType and Dimension variables must be
>>>>>>> set carefully.
>>>>>>>
>>>>>>>  For example, with the code in the state you sent me, PixelType is
>>>>>>> an unsigned short (which is 16 bits) and the dimension is 3.
>>>>>>>
>>>>>>>  These settings would be incorrect for the 2chZT.lsm sample data, as
>>>>>>> it has an 8 bit unsigned pixel type (so PixelType should be defined as
>>>>>>> unsigned char). Also if you're using the SCIFIOImageIO you can typically
>>>>>>> set dimensions to 5.
>>>>>>>
>>>>>>>  I don't know the specifications of your .oif dataset, but the
>>>>>>> PixelType and dimensions may need to be adjusted for it.
>>>>>>>
>>>>>>>   Sorry if you already knew this and were updating these values!
>>>>>>>
>>>>>>>   Anyway, I've been working on improving the stability of the
>>>>>>> SCIFIOImageIO plugin. Using the itkSCIFIOImageIOTest I mentioned
>>>>>>> previously, I confirmed that I was able to read and write the 2chsZT.lsm
>>>>>>> file and the .oif test images I have access to.
>>>>>>>
>>>>>>>   These changes aren't in an ITK patch set yet because they include
>>>>>>> changes to both the c++ and java code, which will may require some
>>>>>>> restructuring of SCIFIOImageIO.
>>>>>>>
>>>>>>> However, if you're eager to test these changes, you could build off
>>>>>>> my development branches:
>>>>>>> 1. Clone my ITK repo. <https://github.com/hinerm/ITK> The default
>>>>>>> branch is what you want here.
>>>>>>> 2. Build ITK as normal (with tests enabled)
>>>>>>> 3. Clone my Bio-Formats repo <https://github.com/hinerm/bioformats>.
>>>>>>> 4. Check out the itk-bridge-fixes branch in Bio-Formats
>>>>>>> 5. From the top level of Bio-Formats, run "ant tools"
>>>>>>> 6. Copy the loci_tools.jar from ${bio-formats}/artifacts to
>>>>>>> ${ITK_build}/lib/jars/
>>>>>>>    (this will overwrite the loci_tools.jar that was downloaded
>>>>>>> automatically when ITK built)
>>>>>>>
>>>>>>>  At this point you can link in the ImageToVTKFilter you were using
>>>>>>> or run the SCIFIO ImageIO test programs included in ITK.
>>>>>>>
>>>>>>>  Whether you build these manually or not, I'll try to get a new
>>>>>>> official patch set released soon.
>>>>>>>
>>>>>>>  Let me know if you run into any problems.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Mark
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Apr 25, 2013 at 4:12 PM, Mark Hiner <hinerm at gmail.com>wrote:
>>>>>>>
>>>>>>>> Hi Joe,
>>>>>>>>
>>>>>>>>
>>>>>>>> Here are my steps to compile the program, in case I missed
>>>>>>>>> something:
>>>>>>>>> 1. Copy the git commend from the 'Patch set 3' review page, and
>>>>>>>>> paste it under the master ITK source, which downloaded a bunch of files and
>>>>>>>>> switched me to the branch 'SCIFIOImageIO'.
>>>>>>>>> 2. Compile the new ITK source in CMake, with Module_ITKVtkGlue,
>>>>>>>>> ITK_USE_REVIEW, and BUILD_SHARED_LIBS checked.
>>>>>>>>> 3. Link the program with the ITK with SCIFIOImageIO.
>>>>>>>>>
>>>>>>>>
>>>>>>>> This looks good. Note that a Patch set 4 is now out, and I'm
>>>>>>>> working on Patch 5. I'm hoping to include any necessary fixes for your
>>>>>>>> issues in patch 5.
>>>>>>>>
>>>>>>>> So, to clarify here - you're end goal is to convert your .oif
>>>>>>>> dataset to .ome.tiff? Is that right? Are you specifically wanting to
>>>>>>>> extract a subset of the image or anything? If you're just trying to do a
>>>>>>>> simple conversion it may be better to use the tests that are part of the
>>>>>>>> ITK SCIFIOImageIO module. You can take a look at them at:
>>>>>>>> $ITK/Modules/IO/SCIFIO/test
>>>>>>>>
>>>>>>>> To build them, just turn the BUILD_TESTING flag on when configuring
>>>>>>>> cmake. The ITKIOSCIFIOTestDriver is built to $build/bin
>>>>>>>>
>>>>>>>> I've been using the itkSCIFIOImageIOTest to test image conversion,
>>>>>>>> for example.
>>>>>>>>
>>>>>>>> The hanging happens after the line printing "post-update".
>>>>>>>>>
>>>>>>>>
>>>>>>>> Looking at the script that suggests it read your dataset
>>>>>>>> successfully but failed to write.
>>>>>>>>
>>>>>>>> Using the ImageIOTest mentioned above I tried converting the
>>>>>>>> 2chsZT.lsm to an .ome.tiff. It got through half the planes and then crashed
>>>>>>>> (which sounds like an improvement over your report, so I'm taking that as a
>>>>>>>> small victory).
>>>>>>>>
>>>>>>>> There are some oddities in the ITK <-> Bio-Formats axis
>>>>>>>> conversion.. it looks like only 1 of the 2 channels was detected for some
>>>>>>>> reason. So I'll fix that in the next patch set.
>>>>>>>>
>>>>>>>> I got similar results with an .oif file locally.
>>>>>>>>
>>>>>>>> So I apologize, as I don't feel I've helped you much yet. I just
>>>>>>>> wanted you to know that I'm still looking at these issues. Once I'm to the
>>>>>>>> point that I can convert that 2chsZT.lsm and local .oif files I'll try to
>>>>>>>> finalize another patch and see if that fixes things for you.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Mark
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Apr 24, 2013 at 2:26 PM, Joe Ping-Lin Hsiao <
>>>>>>>> phsiao at cs.unc.edu> wrote:
>>>>>>>>
>>>>>>>>> Hi Mark,
>>>>>>>>>
>>>>>>>>>     Thanks for the explanation. I have tried switching to
>>>>>>>>> SCIFIOImageIO in the cpp code, but unfortunately the program still hangs
>>>>>>>>> during writing files out (with a .oif input). I have attached the source
>>>>>>>>> code. What I did was just replacing itk::BioFormatsImageIO with
>>>>>>>>> itk::SCIFIOImageIO and a change of the included header file.
>>>>>>>>>
>>>>>>>>> Here are my steps to compile the program, in case I missed
>>>>>>>>> something:
>>>>>>>>> 1. Copy the git commend from the 'Patch set 3' review page, and
>>>>>>>>> paste it under the master ITK source, which downloaded a bunch of files and
>>>>>>>>> switched me to the branch 'SCIFIOImageIO'.
>>>>>>>>> 2. Compile the new ITK source in CMake, with Module_ITKVtkGlue,
>>>>>>>>> ITK_USE_REVIEW, and BUILD_SHARED_LIBS checked.
>>>>>>>>> 3. Link the program with the ITK with SCIFIOImageIO.
>>>>>>>>>
>>>>>>>>> The hanging happens after the line printing "post-update".
>>>>>>>>> By the way, I am using VS 2008 sp1 on Windows 7 64-bit.
>>>>>>>>>
>>>>>>>>> I also tested the latest bf-itk-pipe code. The hanging happens
>>>>>>>>> after the line printing "pre-update" in this case.
>>>>>>>>> Note that I had to link bf-itk-pipe against the SCIFIOImageIO ITK
>>>>>>>>> otherwise there'd be a link error if linked against the master ITK.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Joe
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Apr 23, 2013 at 4:11 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>
>>>>>>>>>> Hi Joe,
>>>>>>>>>>
>>>>>>>>>>  First of all, in case you are using the bf-itk-pipes plugin, you
>>>>>>>>>> should know that we currently have a patch under review<http://review.source.kitware.com/9450>by the ITK developers. This patch adds a true Bio-Formats (SCIFIO) ITK
>>>>>>>>>> ImageIO, which will be available by default in ITK.
>>>>>>>>>>
>>>>>>>>>>  This means that the old bf-itk-pipes plugin won't be supported,
>>>>>>>>>> as all development efforts are going towards the SCIFIOImageIO.
>>>>>>>>>>
>>>>>>>>>>  As this code is still under review, there are issues that I'm
>>>>>>>>>> currently working on resolving.
>>>>>>>>>>
>>>>>>>>>>  So first I'd ask that you check out the latest patch code using
>>>>>>>>>> the links on that review site and rework the splitting code to use
>>>>>>>>>> SCIFIOImageIO<http://review.source.kitware.com/#/c/9450/3/Modules/IO/SCIFIO/include/itkSCIFIOImageIO.h>instead of the BioFormatsImageIO. You can also look at the test
>>>>>>>>>> directory<http://review.source.kitware.com/#/c/9450/3/Modules/IO/SCIFIO/test/itkSCIFIOImageIOTest.cxx>for more sample code.
>>>>>>>>>>
>>>>>>>>>>  If you have any trouble updating your code, please let me know
>>>>>>>>>> and I'll be happy to help.
>>>>>>>>>>
>>>>>>>>>>  As to your actual issue of the .oif image hanging, I actually
>>>>>>>>>> just fixed a bug in the last week regarding hanging when plane size is
>>>>>>>>>> miscalculated by the plugin, so with any luck that was your issue. :)
>>>>>>>>>>
>>>>>>>>>>  If the latest SCIFIOImageIO still gives you problems with your
>>>>>>>>>> .oif dataset though let me know and I'll investigate.
>>>>>>>>>>
>>>>>>>>>>  As for the 2chZT data, that sounds unrelated to the hanging so
>>>>>>>>>> I'll take a look at it locally when I get a chance.
>>>>>>>>>>
>>>>>>>>>>  Thanks for using our software and reporting these issues!
>>>>>>>>>>
>>>>>>>>>> - Mark
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Apr 23, 2013 at 2:40 PM, Joe Ping-Lin Hsiao <
>>>>>>>>>> phsiao at cs.unc.edu> wrote:
>>>>>>>>>>
>>>>>>>>>>> Mark,
>>>>>>>>>>>      I tested your code with the .lsm download from
>>>>>>>>>>> http://www.loci.wisc.edu/files/software/data/2chZT.zip. The
>>>>>>>>>>> program just crashed with the error message
>>>>>>>>>>>
>>>>>>>>>>> "This application has requested the Runtime to terminate it in
>>>>>>>>>>> an unusual way.
>>>>>>>>>>> Please contact the application's support team for more
>>>>>>>>>>> information."
>>>>>>>>>>>
>>>>>>>>>>> If I tested with my own 5D .oif image collection, the execution
>>>>>>>>>>> hangs at 'extractor->Update()' and it does not even use any CPU. Any idea
>>>>>>>>>>> how to fix it?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Joe
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Oct 19, 2012 at 12:41 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>
>>>>>>>>>>>>  I updated your test<https://gist.github.com/3900888/81064cf96fb06b4575a64e81481fda0936cd9056>to use an ExtractImageFilter in the IO pipeline. When a Reader's output is
>>>>>>>>>>>> piped to this Filter, the reader->Update() call only reads the extracted
>>>>>>>>>>>> region.
>>>>>>>>>>>>
>>>>>>>>>>>>  When I tested on the
>>>>>>>>>>>> ConfocalMemCitrine516LeftEar18hpfAug17_2012_Fish1.lsm test image you
>>>>>>>>>>>> uploaded, it correctly splits out the stacks for each time point. And when
>>>>>>>>>>>> I tested on a 9GB image it processed each time point without reading the
>>>>>>>>>>>> entire file.
>>>>>>>>>>>>
>>>>>>>>>>>>  So I'm hoping this solves your issue. If not, just let me know
>>>>>>>>>>>> how it's failing for you and I'll try again!
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Mark
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Oct 17, 2012 at 2:34 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> One thought is that, as the itkBioFormatsImageIO is a
>>>>>>>>>>>>> StreamingImageIOBase<http://www.itk.org/Doxygen320/html/classitk_1_1StreamingImageIOBase.html>,
>>>>>>>>>>>>> you could just set the desired region (as you were) and use *
>>>>>>>>>>>>> its* Read and Write methods instead of wrapping it in a
>>>>>>>>>>>>> Reader or a Writer.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'll try to put a code example up using this approach. It
>>>>>>>>>>>>> would be a good test case for the bf-itk-pipe plug-in anyway.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Oct 17, 2012 at 1:56 PM, Mark Hiner <hinerm at gmail.com>wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  Just wanted to let you know that I'm still looking at your
>>>>>>>>>>>>>> issue today. I updated the gist<https://gist.github.com/3900888/7aab8fddb38af87ea98cd49dd921524ace6430c8>to reflect the topics discussed in this e-mail, using the region setting
>>>>>>>>>>>>>> method calls instead of pipelining, and added some (hopefully) helpful
>>>>>>>>>>>>>> print statements to see what's going on with the regions.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  From what I can tell there are two separate issues
>>>>>>>>>>>>>> preventing your code from writing the desired regions.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  First of all, the IORegion needs to be set on the Writer,
>>>>>>>>>>>>>> for example:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>     writer->SetImageIO(io);
>>>>>>>>>>>>>>     writer->SetInput(reader->GetOutput());
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>     writer->SetIORegion(region);
>>>>>>>>>>>>>>     writer->Write();
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  In the original code you sent to me, that wasn't happening
>>>>>>>>>>>>>> and it was defaulting to the first 3D region and thus the first time point.
>>>>>>>>>>>>>> I mentioned this, without providing a code example, in a past e-mail - so
>>>>>>>>>>>>>> apologies if you've already updated your code!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  Note that according to the itkImageFileWriter API<http://www.itk.org/Doxygen/html/classitk_1_1ImageFileWriter.html#ab2cba65f716aa75ec962171ee0b46fe3>Write() is a special form of Update() that respects the IORegion. However,
>>>>>>>>>>>>>> in practice, for me it still just writes the largest possible region
>>>>>>>>>>>>>> instead of what was specified. So that could suggest a bug in the
>>>>>>>>>>>>>> bf-itk-pipes plug-in.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  The second issue is that even after manually specifying the
>>>>>>>>>>>>>> IO regions, the Reader output still has the largest possible dimensions
>>>>>>>>>>>>>> (T=2) so the current gist test ends up creating 2 full copies of the image.
>>>>>>>>>>>>>> I think this was always happening at the Reader, but then it was truncated
>>>>>>>>>>>>>> to the first time point you saw by the Writer. Anyway, this implies that
>>>>>>>>>>>>>> the full image is still being read, despite the region specifications.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  I think this is a side effect of setting using the
>>>>>>>>>>>>>> LargestPossibleRegion in the Update() method, as mentioned in the itkProcessObject
>>>>>>>>>>>>>> specification<http://www.itk.org/Doxygen/html/classitk_1_1ProcessObject.html#a4041fb21e9105500eee311e265691bd5>
>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  Right now I'm going through the examples from this ITK
>>>>>>>>>>>>>> Users thread<http://itk-insight-users.2283740.n2.nabble.com/Memory-limits-of-ITKImage-object-td5670422.html>about streaming for a better pipelining solution. I'm also testing to see
>>>>>>>>>>>>>> what can be done at the ImageIO level, as when Read and Write are hit in
>>>>>>>>>>>>>> the itkBioFormatsImageIO, GetIORegion() is already returning the largest
>>>>>>>>>>>>>> possible region, instead of the specified region.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  I'll keep you updated.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 1:37 PM, Kishore Mosaliganti <
>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  Dear Mark,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for the clarification. My original problem remains
>>>>>>>>>>>>>>> unresolved. For example, in the LSM gist on lines 104-107:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     ReaderType::Pointer reader = ReaderType::New();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     reader->SetFileName(argv[1]);
>>>>>>>>>>>>>>>     reader->SetImageIO(io);
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     reader->Update();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Here, calling reader->Update() loads the entire image into
>>>>>>>>>>>>>>> memory. If you have a very large LSM file (several gigabytes large), then
>>>>>>>>>>>>>>> this code will fail since such an image cannot get into the memory.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hence, I am trying to send the read region directly into the
>>>>>>>>>>>>>>> reader using
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> io->SetRegion(region);
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and then
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> reader->SetImageIO(io);
>>>>>>>>>>>>>>> reader->Update();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> But this step is not giving me two different timepoints. It
>>>>>>>>>>>>>>> is writing out the same timepoint.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 2:13 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello again,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  I created a gist for your LSM test and updated it with my
>>>>>>>>>>>>>>>> most recent code:
>>>>>>>>>>>>>>>> https://gist.github.com/3900888
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  That example is currently working for me, in that it
>>>>>>>>>>>>>>>> writes out 2 images of 72 planes - one for T=0 and one for T=1. Since
>>>>>>>>>>>>>>>> everything in ITK seems to be pipeline-oriented, I am not sure there is a
>>>>>>>>>>>>>>>> solution to splitting out channels just using Setters like we were both
>>>>>>>>>>>>>>>> trying.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  Instead, I followed this ROI Filter example<http://www.vtk.org/Wiki/ITK/Examples/ImageProcessing/RegionOfInterestImageFilter>and used the logic from my last e-mail.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  I do need to correct one thing I said in my last e-mail:
>>>>>>>>>>>>>>>> SizeT should always be 1, as it is a relative offset to the Index value.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  When I ran that test, both images went to grayscale.. I'm
>>>>>>>>>>>>>>>> going to see if that's an error in the bf-itk-pipes logic, or just
>>>>>>>>>>>>>>>> something that needs to be set up differently in using the ITK API.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  Let me know if you run into any more problems with this
>>>>>>>>>>>>>>>> code, or if you just want to discuss any part of it.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks again for using our software :)
>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 11:00 AM, Mark Hiner <
>>>>>>>>>>>>>>>> hiner at wisc.edu> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  I am sorry I wasn't able to get to your issue yesterday.
>>>>>>>>>>>>>>>>> Looking at it today, I think there are a couple of things going on here.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  First of all: in your test class
>>>>>>>>>>>>>>>>> - I consolidated a BioFormatsImageIO and the ImageIOBase.
>>>>>>>>>>>>>>>>> - In the region updating loop, the wrong region (dim-3
>>>>>>>>>>>>>>>>> instead of dim-2) was being set, and only the index is set. I suspect both
>>>>>>>>>>>>>>>>> the index and size need to be set.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  I attached an updated version of your test, where in the
>>>>>>>>>>>>>>>>> first iteration it sets IndexT = 0, sizeT = 1. The second iteration sets
>>>>>>>>>>>>>>>>> IndexT = 1, sizeT = 2.  I believe those regions will split the time points
>>>>>>>>>>>>>>>>> as you wanted.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But there is a second issue: although the Reader IORegion
>>>>>>>>>>>>>>>>> is correct (I think) the Writer is not. It is defaulting to the ITK base 3D
>>>>>>>>>>>>>>>>> region. When I tried setting the Writer's region to our desired region, it
>>>>>>>>>>>>>>>>> throws an exception because the 5D region can't be contained by what it
>>>>>>>>>>>>>>>>> thinks is the largest possible (3D) region. I am refreshing myself on the
>>>>>>>>>>>>>>>>> ITK API and once we figure out how to set that region, I think it will
>>>>>>>>>>>>>>>>> write the images correctly (or we can move on to the next problem).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> In your test I changed the output format to ome.tiff
>>>>>>>>>>>>>>>>> because I know that will use the itkBioFormatsImageIO writer code, which I
>>>>>>>>>>>>>>>>> know is capable of handling 5D writing.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Also, in testing I found a couple of bugs in the
>>>>>>>>>>>>>>>>> ITKPipesBridge code. I started a new branch to fix these and any other
>>>>>>>>>>>>>>>>> issues that may be discovered. If you want to use the itkBioFormatsImageIO
>>>>>>>>>>>>>>>>> writer code you'll have to use this new branch for now. You can find it
>>>>>>>>>>>>>>>>> here:
>>>>>>>>>>>>>>>>> https://github.com/hinerm/bioformats/commits/bf-itk-fixes
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> As always, let me know if this isn't fully addressing your
>>>>>>>>>>>>>>>>> issue. Otherwise, I'll let you know as soon as I have the Writer region
>>>>>>>>>>>>>>>>> setting properly.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 11:28 AM, Kishore Mosaliganti <
>>>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Mark,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thank you for the email and invaluable help. The
>>>>>>>>>>>>>>>>>> bioformats-itk plugin is wonderful. I am attaching my script here to help
>>>>>>>>>>>>>>>>>> you debug. Let me know if you have any questions on my script. I am
>>>>>>>>>>>>>>>>>> starting with a 5D (XYZTC) image and trying to write out all the 3D
>>>>>>>>>>>>>>>>>> timepoints for the first channel.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 12:19 PM, Mark Hiner <
>>>>>>>>>>>>>>>>>> hinerm at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>  I helped develop the bf-itk-pipe plugin and am
>>>>>>>>>>>>>>>>>>> investigating your issue right now.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>  What you're doing seems reasonable, so I'm trying to
>>>>>>>>>>>>>>>>>>> determine if there's a bug in the plugin. My goal is to have some working
>>>>>>>>>>>>>>>>>>> example code for you (and a fix if necessary) by the end of the day.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thank you for using our plugin! I hope we can get it
>>>>>>>>>>>>>>>>>>> working for you soon.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 10:54 AM, Kishore Mosaliganti <
>>>>>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Dear ITK and OME users,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> To further elaborate, I figured that this has nothing
>>>>>>>>>>>>>>>>>>>> to do with the itkStreamingImageFilter.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Instead, the itkImageFileReader seems to be extracting
>>>>>>>>>>>>>>>>>>>> the same image region although I update the requested region in the
>>>>>>>>>>>>>>>>>>>> BioformatsImageIO class :;
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>     itk::BioFormatsImageIO::Pointer io =
>>>>>>>>>>>>>>>>>>>> itk::BioFormatsImageIO::New();
>>>>>>>>>>>>>>>>>>>>     io->SetIORegion( region ); // doesn't make a
>>>>>>>>>>>>>>>>>>>> difference in terms of image data extracted. Its always from the beginning.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>     ReaderType::Pointer reader = ReaderType::New();
>>>>>>>>>>>>>>>>>>>>     reader->SetFileName(argv[1]);
>>>>>>>>>>>>>>>>>>>>     reader->SetImageIO(io);
>>>>>>>>>>>>>>>>>>>>     reader->Update();
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> So, my general question is whether the
>>>>>>>>>>>>>>>>>>>> BioformatsImageIO expects the whole LSM image to be loaded into memory
>>>>>>>>>>>>>>>>>>>> before writing it out? I would like to specify small image regions since my
>>>>>>>>>>>>>>>>>>>> LSM is too large to be fully loaded into memory. How else can I stream data
>>>>>>>>>>>>>>>>>>>> from large LSM?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 8:53 AM, Kishore Mosaliganti <
>>>>>>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I am trying to use bioformats and itk. I downloaded
>>>>>>>>>>>>>>>>>>>>> and compiled bf-itk-pipe using cmake by linking against ITK 3.2:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> https://github.com/openmicroscopy/bioformats/tree/develop/components/native/bf-itk-pipe
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I tested out itkBFImageInfo and
>>>>>>>>>>>>>>>>>>>>> ./itkRGBBioFormatsImageIOTest on a few simple LSM images. It works great
>>>>>>>>>>>>>>>>>>>>> and writes out all the associated metadata and pixel data for the first
>>>>>>>>>>>>>>>>>>>>> timepoint.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> My LSM microscopy image is 5D. It is
>>>>>>>>>>>>>>>>>>>>> X-Y-Z-Time-Channel. I am interested in extracting individual timepoints
>>>>>>>>>>>>>>>>>>>>> from the LSM file for processing. The filter seems to be using
>>>>>>>>>>>>>>>>>>>>> itkStreamingImageFilter. I set the NumberOfStreams to 10 but that still
>>>>>>>>>>>>>>>>>>>>> seems to still write out only the first image.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> How do I set the output requestion region for an
>>>>>>>>>>>>>>>>>>>>> individual timepoint?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>> ome-users mailing list
>>>>>>>>>>>>>>>>>>>> ome-users at lists.openmicroscopy.org.uk
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> ome-users mailing list
>>>>>>>>>>>> ome-users at lists.openmicroscopy.org.uk
>>>>>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-users/attachments/20130522/8d28880d/attachment.htm>

More information about the Insight-users mailing list