[ITK] convert .lif file with bfconvert

Tue Jun 3 02:56:55 EDT 2014

Hi Mark

> export JAVA_FLAGS=-Xmx3400m
> /SCIFIOTestDriver itkSCIFIOImageIOTest bfconvert_output/output_series_0.ome.tiff test.ome.tiff -w -d 5

This worked for me as well. Thanks a lot.
So I can convert my multi-channell-z-series images contained in my
.lif files with bfconvert into ome.tiff and read these with itk thanks
to the SCIFIO module.
If I see this correctly this reads the image into an object of type
"itk::Image< unsigned short, 5 >".
I was trying to read it into "itk::Image< <itk::Vector<unsigned short,
3>>, 3 >" and to use VectorIndexSelectionCastImageFilter
to get to the single channells.
Could you give me a hint on how to get the single channells out of
"itk::Image< unsigned short, 5 >"?

Thanks,
Michael

On 2 June 2014 15:27, Mark Hiner <hiner at wisc.edu> wrote:
> Hi Michael,
>
>
>> So far I've used "File > Open". Sorry.
>
>
> No need to apologize! This is a common misunderstanding. When you use "File
>> Open", your .ome.tiff gets picked up by the ImageJ TIFF reader instead of
> Bio-Formats. So that's why metadata is missing. This is actually something
> we're improving with the ImageJ2 release this week: "File > Open" will be
> able to work with Bio-Formats automatically.
>
>
>> I just tried "File > Import > Bio-Formats" and it works as well.
>
>
> This made me suspect that there was something going wrong on my end. I
> should have asked which version of Bio-Formats you were using bfconvert
> from.. I was using it from v5.0.1. I updated to v5.0.2 this morning and
> tried bfconvert, and now my output .ome.tiffs open just fine in Fiji's
> Bio-Formats Importer. So I'm sorry for any misguidance there.
>
> Anyway, as for the itkSCIFIOImageIO use, I was still getting
> OutOfMemoryErrors. So you'll want to increase the amount of memory Java can
> use. By default, the ImageIO creates a JVM with 256m. You can override this
> by setting a "JAVA_FLAGS" environment variable. Once I went over 3g, I was
> actually able to convert your data appropriately, finally! To increase
> available memory, I used the command:
>
> export JAVA_FLAGS=-Xmx3400m
>
> Also! I forgot there were some important considerations with other flags to
> use with the itkSCIFIOImageIO:
>
> use "-d 5". I know your data is 3D + channels, but as an .ome.tiff it will
> be considered 5D by the imageio and the wrong dimensions could get truncated
> if you set -d less than 5.
> If necessary, set pixel type with "-t". The default is unsigned short (an
> ITK pixel type, which is equivalent to Bio-Formats's uint16). Your data is
> uint16 so there's no need to use this flag, just wanted you to be aware of
> it.
>
> So, the command that worked for me was:
>
> /SCIFIOTestDriver itkSCIFIOImageIOTest
> bfconvert_output/output_series_0.ome.tiff test.ome.tiff -w -d 5
>
> Let me know if you have any more problems,
>
> Mark
>
>
>
> On Sun, Jun 1, 2014 at 1:19 AM, Michael Meuli <michael.meuli at gmail.com>
> wrote:
>>
>> Hi Mark,
>>
>> I just tried "File > Import > Bio-Formats" and it works as well.
>> If I select one of the files produced with ...%s.ome.tiff in the first
>> open dialog box and then Hyperstack in the "Bio-Formats Import
>> Options", I can open any of the series files.
>> If I separate on of these files into another directory and try to open
>> it the same way, Fiji crashes.
>> I can also open the file containing all the series, but made some
>> coffee in between.
>> So far I've used "File > Open". Sorry. "File > Open" also works for
>> the separated file, in case this matters, but shows them in gray and
>> plane by plane (no channell selection).
>>
>> Thanks,
>> Michael
>>
>>
>> On 1 June 2014 03:08, Mark Hiner <hiner at wisc.edu> wrote:
>> > Hi Michael,
>> >
>> >
>> >> I can import the bfconvert output into Fiji with no problem.
>> >
>> >
>> > I just wanted to confirm, you're using "File > Import > Bio-Formats" or
>> > the
>> > Bio-Formats importer in Fiji, correct? It definitely didn't work for me
>> > when
>> > I tried yesterday. I can check next week to see if I was using an
>> > outdated
>> > Bio-Formats or something..
>> >
>> > Thanks,
>> > Mark
>> >
>> >
>> > On Sat, May 31, 2014 at 7:52 AM, Michael Meuli <michael.meuli at gmail.com>
>> > wrote:
>> >>
>> >> Hi Mark
>> >>
>> >> I can import the bfconvert output into Fiji with no problem.
>> >> (If I do "./bfconvert /path/to/wt-dead-A.lif /path/to/output.ome.tiff"
>> >> or
>> >> "./bfconvert /path/to/wt-dead-A.lif /path/to/output_series_%s.ome.tiff"
>> >> I can open the file(s) with Fiji.)
>> >> If I open the .lif file with Fiji and export one series to ome.tiff I
>> >> can read the output with ./SCIFIOTestDriver itkSCIFIOImageIOTest ...
>> >> whereas I'm not able to read the output of bfconvert.
>> >>
>> >> Thanks a lot and best regards
>> >> Michael
>> >>
>> >>
>> >>
>> >> On 30 May 2014 16:18, Mark Hiner <hinerm at gmail.com> wrote:
>> >> > Hi Michael,
>> >> >
>> >> >> "./SCIFIOTestDriver
>> >> >> itkSCIFIOImageIOTest/path/to/output_series_0.ome.tiff
>> >> >> /path/to/scifio-test.ome.tiff"
>> >> >
>> >> > Just a reminder, since you're writing ome.tiff you should have a "-w"
>> >> > flag
>> >> > at the end of this command. It sounds like you were using -w, just
>> >> > wanted to
>> >> > clarify for anyone else reading this thread.
>> >> >
>> >> >> Exception in thread "main" java.lang.OutOfMemoryError: Java heap
>> >> >> space
>> >> >
>> >> >  There are two problems here. First, it seems that in SCIFIOImageIO
>> >> > the
>> >> > use
>> >> > of Bio-Formats is overly aggressive in pulling in all the series,
>> >> > thus
>> >> > the
>> >> > OutOfMemoryError. If you physically isolate a single series (e.g.
>> >> > delete
>> >> > or
>> >> > move to another directory series 1-19) the application will run, but
>> >> > we
>> >> > get
>> >> > another exception (at least on my end):
>> >> >
>> >> > ./SCIFIOTestDriver itkSCIFIOImageIOTest output_series_0.ome.tiff
>> >> > scifio-test.ome.tiff -w
>> >> >
>> >> > reader->GetUseStreaming(): 1
>> >> > done checking streaming usage
>> >> > ITK test driver caught an ITK exception:
>> >> >
>> >> > itk::ExceptionObject (0x7ff8838b6ae8)
>> >> > Location: "unknown"
>> >> > File:
>> >> >
>> >> >
>> >> > /Users/mhiner/loci/ITK/build/Modules/Remote/SCIFIO/src/itkSCIFIOImageIO.cxx
>> >> > Line: 180
>> >> > Description: itk::ERROR: SCIFIOImageIO(0x7ff8838b4f00): SCIFIOImageIO
>> >> > exited
>> >> > abnormally. Creating new reader for output_series_0.ome.tiff
>> >> >
>> >> > log4j:WARN No appenders could be found for logger
>> >> > (loci.common.services.ServiceFactory).
>> >> > log4j:WARN Please initialize the log4j system properly.
>> >> > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
>> >> > for
>> >> > more info.
>> >> > Exception in thread "main" loci.formats.FormatException: Unmatched
>> >> > UUID:
>> >> > urn:uuid:b72fe97b-02c9-4029-a876-ca2bae46cbac
>> >> >     at loci.formats.in.OMETiffReader.initFile(OMETiffReader.java:456)
>> >> >
>> >> >     at loci.formats.FormatReader.setId(FormatReader.java:1244)
>> >> >     at loci.formats.ImageReader.setId(ImageReader.java:727)
>> >> >     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
>> >> >     at loci.formats.ChannelFiller.setId(ChannelFiller.java:263)
>> >> >     at
>> >> >
>> >> > loci.scifio.itk.SCIFIOITKBridge.createReader(SCIFIOITKBridge.java:617)
>> >> >     at
>> >> >
>> >> > loci.scifio.itk.SCIFIOITKBridge.readImageInfo(SCIFIOITKBridge.java:268)
>> >> >     at
>> >> >
>> >> > loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:133)
>> >> >
>> >> >     at
>> >> >
>> >> > loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:108)
>> >> >     at
>> >> > loci.scifio.itk.SCIFIOITKBridge.waitForInput(SCIFIOITKBridge.java:83)
>> >> >     at
>> >> >
>> >> > loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:163)
>> >> >     at loci.scifio.itk.SCIFIOITKBridge.main(SCIFIOITKBridge.java:688)
>> >> >
>> >> > I also get this exception when importing with Bio-Formats into Fiji.
>> >> >
>> >> > As a side note, there are flags for itkSCIFIOImageIOTest that, in
>> >> > theory,
>> >> > should help the out of memory side:
>> >> >
>> >> > "-s <n1, n2>" should limit the series read.
>> >> > "-v <n>" should divide the data into divisions, which might mitigate
>> >> > the
>> >> > risk of OutOfMemoryErrors.
>> >> >
>> >> > But neither of these will help the UUID exception. Also, I still got
>> >> > OutOfMemoryErrors with these, so it was still trying to initialize
>> >> > using
>> >> > all
>> >> > the series. So I created a new issue in the SCIFIO-ImageIO tracker.
>> >> >
>> >> > Anyway, try importing the bfconvert output into Fiji with Bio-Formats
>> >> > and
>> >> > let me know if you don't get the UUID exception. If you do, then I
>> >> > think
>> >> > the
>> >> > bfconvert output is just invalid..
>> >> >
>> >> > Regards,
>> >> > Mark
>> >> >
>> >> >
>> >> > On Thu, May 29, 2014 at 12:17 PM, Michael Meuli
>> >> > <michael.meuli at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi Mark,
>> >> >>
>> >> >> Thanks for the quick response. I can see I was using
>> >> >> itkSCIFIOImageIOTest the wrong way and it doesn't make sense to use
>> >> >> it
>> >> >> like this.
>> >> >> If I use the -w flag I get an error in all cases but at the moment I
>> >> >> don't care as my aim now is to read ome.tiff files with itk.
>> >> >> I've written a small program (mainly by copy paste from examples /
>> >> >> https://github.com/michaelmeuli/bactelize) which so far reads the
>> >> >> multi-channel-z-series.ome.tif file, does a maximum intensity
>> >> >> projection of one channel and writes a 2D image and it seems to work
>> >> >> with the multi-channel-z-series.ome.tif but not with the files
>> >> >> converted with bfconvert.
>> >> >>
>> >> >> Expressed with itkSCIFIOImageIOTest:
>> >> >> "./SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> /path/to/multi-channel-z-series.ome.tif
>> >> >> /path/to/scifio-test.ome.tiff"
>> >> >> doesn't give an error and seems to produce a tiff file of the first
>> >> >> z-plane, but
>> >> >> "./SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> /path/to/output_series_0.ome.tiff /path/to/scifio-test.ome.tiff"
>> >> >> gives
>> >> >> an error:
>> >> >>
>> >> >> reader->GetUseStreaming(): 1
>> >> >> done checking streaming usage
>> >> >> ITK test driver caught an ITK exception:
>> >> >>
>> >> >> itk::ExceptionObject (0x2f87680)
>> >> >> Location: "unknown"
>> >> >> File:
>> >> >>
>> >> >>
>> >> >> /home/michael/Colocalization/ITK-Release-build/Modules/Remote/SCIFIO/src/itkSCIFIOImageIO.cxx
>> >> >> Line: 180
>> >> >> Description: itk::ERROR: SCIFIOImageIO(0x2f78970): SCIFIOImageIO
>> >> >> exited abnormally. Creating new reader for
>> >> >>
>> >> >>
>> >> >>
>> >> >> /home/michael/bioimage/ome/data/wt-dead-A-ome-tiff/output_series_0.ome.tiff
>> >> >> log4j:WARN No appenders could be found for logger
>> >> >> (loci.common.services.ServiceFactory).
>> >> >> log4j:WARN Please initialize the log4j system properly.
>> >> >> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
>> >> >> for more info.
>> >> >> Exception in thread "main" java.lang.OutOfMemoryError: Java heap
>> >> >> space
>> >> >> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
>> >> >> at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
>> >> >> at
>> >> >>
>> >> >>
>> >> >> loci.common.NIOByteBufferProvider.allocateDirect(NIOByteBufferProvider.java:132)
>> >> >> at
>> >> >>
>> >> >>
>> >> >> loci.common.NIOByteBufferProvider.allocate(NIOByteBufferProvider.java:118)
>> >> >> at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:532)
>> >> >> at loci.common.NIOFileHandle.seek(NIOFileHandle.java:254)
>> >> >> at
>> >> >>
>> >> >>
>> >> >> loci.common.RandomAccessInputStream.seek(RandomAccessInputStream.java:140)
>> >> >> at loci.formats.tiff.TiffParser.getIFDOffsets(TiffParser.java:308)
>> >> >> at loci.formats.tiff.TiffParser.getIFDs(TiffParser.java:226)
>> >> >> at
>> >> >>
>> >> >> loci.formats.in.MinimalTiffReader.initFile(MinimalTiffReader.java:426)
>> >> >> at loci.formats.FormatReader.setId(FormatReader.java:1244)
>> >> >> at loci.formats.in.OMETiffReader.initFile(OMETiffReader.java:704)
>> >> >> at loci.formats.FormatReader.setId(FormatReader.java:1244)
>> >> >> at loci.formats.ImageReader.setId(ImageReader.java:727)
>> >> >> at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
>> >> >> at loci.formats.ChannelFiller.setId(ChannelFiller.java:263)
>> >> >> at
>> >> >>
>> >> >> loci.scifio.itk.SCIFIOITKBridge.createReader(SCIFIOITKBridge.java:622)
>> >> >> at
>> >> >>
>> >> >> loci.scifio.itk.SCIFIOITKBridge.readImageInfo(SCIFIOITKBridge.java:266)
>> >> >> at
>> >> >>
>> >> >>
>> >> >> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:131)
>> >> >> at
>> >> >>
>> >> >>
>> >> >> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:108)
>> >> >> at
>> >> >>
>> >> >> loci.scifio.itk.SCIFIOITKBridge.waitForInput(SCIFIOITKBridge.java:83)
>> >> >> at
>> >> >>
>> >> >>
>> >> >> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:161)
>> >> >> at loci.scifio.itk.SCIFIOITKBridge.main(SCIFIOITKBridge.java:693)
>> >> >>
>> >> >> Best regards
>> >> >> Michael Meuli
>> >> >>
>> >> >>
>> >> >>
>> >> >> On 29 May 2014 17:41, Mark Hiner <hiner at wisc.edu> wrote:
>> >> >> > Hi Michael,
>> >> >> >
>> >> >> >
>> >> >> >> The problem is that ./SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> >> /path/to/output_series_0.ome.tiff /path/to/scifiotest.ome.tiff
>> >> >> >> (and my little program) throws an exception.
>> >> >> > ...
>> >> >> >
>> >> >> >> How do I use bfconvert the correct way for my case?
>> >> >> >
>> >> >> > You used bfconvert correctly to convert your dataset. Just so it's
>> >> >> > clear, if
>> >> >> > your goal was to produce an OME-TIFF:
>> >> >> >
>> >> >> >> /SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> >> /path/to/output_series_0.ome.tiff
>> >> >> >> /path/to/scifiotest.ome.tiff
>> >> >> >
>> >> >> > would be unnecessary, as you already have the OME-TIFF output from
>> >> >> > bfconvert. The itkSCIFIOImageIOTest is an analogous tool to
>> >> >> > bfconvert
>> >> >> > written in the Bio-Formats C++ framework. (although comparing the
>> >> >> > outputs of
>> >> >> > the itkSCIFIOImageIOTest conversion would be a good way to check
>> >> >> > for
>> >> >> > errors
>> >> >> > in the conversion process)
>> >> >> >
>> >> >> > That said, I believe your conversion failed because the output was
>> >> >> > picked up
>> >> >> > by the ITK TIFF ImageIO, which I believe only supports 3D data.
>> >> >> > The
>> >> >> > itkSCIFIOImageIOTest utility does basically no automation - you
>> >> >> > have
>> >> >> > to
>> >> >> > tell
>> >> >> > it exactly how to run, and because the SCIFIOImageIO is an
>> >> >> > external
>> >> >> > module
>> >> >> > it is not prioritized by the ITK format detection framework over
>> >> >> > internal
>> >> >> > modules (I'm not sure it's part of the automatic detection
>> >> >> > framework
>> >> >> > at
>> >> >> > all,
>> >> >> > actually). So you need to force the use of the SCIFIOImageIO for
>> >> >> > writing.
>> >> >> >
>> >> >> > If you run:
>> >> >> >
>> >> >> >  /SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> >
>> >> >> > with no arguments from the command line, it will print the usage +
>> >> >> > available
>> >> >> > flags. The flag you wanted here is "-w"  to write with the
>> >> >> > SCIFIOImageIO
>> >> >> > (which will use Bio-Formats, and thus the OME-TIFF writer).
>> >> >> >
>> >> >> > So the final syntax is:
>> >> >> >
>> >> >> > /SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> > /path/to/output_series_0.ome.tiff
>> >> >> > /path/to/scifiotest.ome.tiff -w
>> >> >> >
>> >> >> > which worked for me locally.
>> >> >> >
>> >> >> > Let me know if you have any other questions or issues!
>> >> >> >
>> >> >> > Regards,
>> >> >> > Mark
>> >> >> >
>> >> >> > On Thu, May 29, 2014 at 4:40 AM, Michael Meuli
>> >> >> > <michael.meuli at gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> ./SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> >> /path/to/output_series_0.ome.tiff /path/to/scifiotest.ome.tiff
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >
>> >> >
>> >
>> >
>
>