[ITK] [ITK-users] [ITK-dev] TIFFImageIO refactoring, please test

Bradley Lowekamp blowekamp at mail.nih.gov
Fri Nov 14 11:06:57 EST 2014 

Actually further reading the TIFFReadScanline documentation says that should not work for tiled images. So I guess it must be using the RGBA read method, but I am not sure how that is happening either. 

Are you used the tiled tiff images from the VTK tests?

Brad


On Nov 14, 2014, at 10:46 AM, Bill Lorensen <bill.lorensen at gmail.com> wrote:

> Meant to say:
> Wow. The new itk tiff reader read my tiled images.
> 
> 
> On Fri, Nov 14, 2014 at 10:45 AM, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>> Wow. The new itk tiff reader sad my tiled images.
>> 
>> On Fri, Nov 14, 2014 at 10:25 AM, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>>> There is a TIFFReadTile call in lib tiff. I some tiled images I will
>>> try with the updated itk reader. I'll be really syprised if it works
>>> since reading tiled images is tricky and took quite a bit of code.
>>> 
>>> I'll keep you posted.
>>> 
>>> Bill
>>> 
>>> On Fri, Nov 14, 2014 at 10:19 AM, Bradley Lowekamp
>>> <blowekamp at mail.nih.gov> wrote:
>>>> Bill,
>>>> 
>>>> There was a tile block of code before, but that was not covered, looked unreachable and was removed.
>>>> 
>>>> Currently there are two ways that a tiff page can be read with the TIFFReadRGBAImageOriented, and with the TIFFReadScanline. The RGBA method from libtiff is fairly robust but reads the image into an RGBA buffer so is not memory efficient. Our custom method uses the scanline method [1], which should work with both stripped and tiled pages.
>>>> 
>>>> But please test, and add a test? I'm not 100% if there is a test image for tiled images.
>>>> 
>>>> Where the difference between stripped and tiled images would really matter is streaming, but that is not there...yet.
>>>> 
>>>> Brad
>>>> 
>>>> [1] http://www.remotesensing.org/libtiff/libtiff.html#scanlines
>>>> 
>>>> 
>>>> On Nov 14, 2014, at 10:04 AM, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>>>> 
>>>>> Brad,
>>>>> 
>>>>> I recently added tiled image reading to VTK's tiff reader. Do you
>>>>> think that would be useful in IUTK. I don't know how often tiled tiff
>>>>> images are encountered.
>>>>> 
>>>>> BTW I do not mean tiled pyramid tiff images.
>>>>> 
>>>>> Bill
>>>>> 
>>>>> On Fri, Nov 14, 2014 at 9:53 AM, Bradley Lowekamp
>>>>> <blowekamp at mail.nih.gov> wrote:
>>>>>> Greetings,
>>>>>> 
>>>>>> I have recently refactored ITK's TIFFImageIO to remove a significant amount
>>>>>> of dead code, and improve performance up to 3-5X. A number of bug related to
>>>>>> reading BigTIFF files have been addressed. Additionally, Tiff tags are now
>>>>>> place into the Meta-DataDictionary to allow access to additional TIFF fields
>>>>>> such as OME-XML tags ( Originally contributed by Richard Beare
>>>>>> http://www.insight-journal.org/browse/publication/728 ).
>>>>>> 
>>>>>> Significant performance improvements were made to both scalar images and
>>>>>> palette images. Handling of certain orientations are now "better" handled,
>>>>>> while not changing how the default TOPLEFT orientation is loaded or written.
>>>>>> There were a number of cases which appear unreachable and illogical such as
>>>>>> support from Zeiss two-componet types, and support for some type of SGI
>>>>>> tiled tiff as a 3D image. These were removed.
>>>>>> 
>>>>>> If you are an active user of TIFF images with ITK, please checkout the
>>>>>> latest ITK master and test the refactored TIFFImageIO.
>>>>>> 
>>>>>> Thanks,
>>>>>> Brad
>>>>>> 
>>>>>> p.s. Here is the git log for the recent changes.
>>>>>> 
>>>>>> $git log --no-merges 6791b4a3613aade07e3f11f765f1316faf03b113..
>>>>>> 
>>>>>> commit 46f4125df47aeff9c42c9f2b3699c06e67ffaca9
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Fri Oct 31 14:25:13 2014 -0400
>>>>>> 
>>>>>>   ENH: adding TIFFImageIO test for RGB palette images
>>>>>> 
>>>>>>   Change-Id: I320acc8beac8225616451d11b291b1d7e12a17d0
>>>>>> 
>>>>>> commit 9adc54c7f53b4da84a41f500d561ae494cb85158
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Fri Oct 24 13:42:02 2014 -0400
>>>>>> 
>>>>>>   BUG: Adding missing parentheses around boolean expression
>>>>>> 
>>>>>>   Change-Id: I669d160ff3ba830f3c33ac87de434db41472eb79
>>>>>> 
>>>>>> commit 40afd95161727be3274f8094ce19fe45e2c3aa50
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Wed Oct 22 15:05:39 2014 -0400
>>>>>> 
>>>>>>   PERF: Refactor color table lookup
>>>>>> 
>>>>>>   Change to use a index lookup for the color and gray scale
>>>>>>   palettes. The remove excessive exception checking done on a per pixel
>>>>>>   basis with the GetColor method. Additionally the InitializeColor
>>>>>>   method was change do initialize the color palette variables.
>>>>>> 
>>>>>>   An exception no longer occurs when an index exceeds the number of
>>>>>>   entries in a palette. Instead a module operator is used to prevent
>>>>>>   out of bounds memory access.
>>>>>> 
>>>>>>   Change-Id: Ia62c65cd8b1be0d66c488540709a8fb961190cc9
>>>>>> 
>>>>>> commit d98cbeb7206f6007f7d0c25fd4a4becad8973053
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Fri Oct 3 09:01:28 2014 -0400
>>>>>> 
>>>>>>   ENH: Refactor per pixel conversion function to per scan-line method
>>>>>> 
>>>>>>   This patch changes the implementation of conversion from the TIFF
>>>>>>   scanline buffer from a function call on each pixel to a function call
>>>>>>   per scanline. The enable efficient copying of similar scanline and
>>>>>>   better reuse of variables for palette look ups.
>>>>>> 
>>>>>>   This can increase the performance of reading grayscale images by up to
>>>>>>   3X for cached files.
>>>>>> 
>>>>>>   Change-Id: I24bb744fda35629f1c1ed7154e26e5708e912059
>>>>>> 
>>>>>> commit 8e488f40e8cb7defa046700798b77e3944ac33c9
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Tue Sep 30 08:56:58 2014 -0400
>>>>>> 
>>>>>>   ENH: RGBA read images should stay unchanged.
>>>>>> 
>>>>>>   The image that are read with the TIFFReadRGBAImage should not have
>>>>>>   their pixel types converted. This libtiff method always read images
>>>>>>   into an ABGR uint8 format. When the tiff file is read by this method
>>>>>>   it should report that uint8 and RGBA is the component an pixel type
>>>>>>   for the image. This allows the standard ITK buffer conversion to be
>>>>>>   performed.
>>>>>> 
>>>>>>   This change remove extra cases and template instantiations.
>>>>>> 
>>>>>>   Additionally the RGBAImageToBuffer method was flipping the image along
>>>>>>   the y-axis, this has been more explicitly done with the call to
>>>>>>   TIFFReadRGBAImageOriented.
>>>>>> 
>>>>>>   Change-Id: I022b12921d406edb09730f905213b0db811f1bd4
>>>>>> 
>>>>>> commit e5834d5b00e4e3fab3255d88e7d8fd917ee722e9
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Fri Sep 26 14:04:09 2014 -0400
>>>>>> 
>>>>>>   ENH: Remove support for TIFF tile as 3D and dead code
>>>>>> 
>>>>>>   This TIFF file format is a file which contains a list of directories,
>>>>>>   if there are no directories in the file it's not a valid tiff. Code
>>>>>>   which supported 3D tiles was conditioned on a file with no directories
>>>>>>   and no data. Additional code was removed related to a private SGI tag
>>>>>>   TIFFTAG_TILEDEPTH, ImageJ specific tag also pre-conditioned on this
>>>>>>   invalid case.
>>>>>> 
>>>>>>   Change-Id: Id4ef171c5dbe0c797be206e0059c7488719ef6e6
>>>>>> 
>>>>>> commit 3ac19a9ffd94e02254900dd0e5e2661c329e3183
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Mon Oct 6 10:01:50 2014 -0400
>>>>>> 
>>>>>>   BUG: Address Coverity warning about null pointer dereferences
>>>>>> 
>>>>>>   The patch addressed the following Coverity warnings:
>>>>>> 
>>>>>>   ** CID 1243373:  Explicit null dereferenced  (FORWARD_NULL)
>>>>>>   itkTIFFImageIO.cxx:
>>>>>>   1192 in itk::TIFFImageIO::ReadTIFFTags()()
>>>>>> 
>>>>>>   ** CID 1243372:  Dereference null return value  (NULL_RETURNS)
>>>>>>   itkTIFFImageIO.cxx:
>>>>>>   1096 in itk::TIFFImageIO::ReadTIFFTags()()
>>>>>> 
>>>>>>   ** CID 1243371:  Dereference before null check  (REVERSE_INULL)
>>>>>>   itkTIFFImageIO.cxx:
>>>>>>   1100 in itk::TIFFImageIO::ReadTIFFTags()()
>>>>>>   Last login: Fri Oct  3 19:47:29 on ttys004
>>>>>> 
>>>>>>   Change-Id: I02c8435628c860ff03889b9b57e9dd1543bf82da
>>>>>> 
>>>>>> commit 52ce992f3a57ef9f187f9c4b4ebd7ce20db365bd
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Fri Sep 26 11:04:38 2014 -0400
>>>>>> 
>>>>>>   ENH: Reduce code duplicate in TIFFImageIO::ReadCurrentPage
>>>>>> 
>>>>>>   Refactoring of the pixelOffset to include the number of components
>>>>>>   remove the need for a separate block for RGB multi-component files.
>>>>>> 
>>>>>> commit 97b33787ec17b76a61fdb43b5f1e1ea708bbac2c
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Wed Sep 24 15:00:54 2014 -0400
>>>>>> 
>>>>>>   ENH: add arbitrary TIFF TAGs to meta-data dictionary
>>>>>> 
>>>>>>   A more general approach to adding custom tiff tags to the meta-data
>>>>>>   dictionary has been used based on the implementation of libtiff's
>>>>>>   PrintDirectory method and the libtiff "Defining New TIFF Tags"
>>>>>>   documentation.
>>>>>> 
>>>>>>   Additionally a new test image was added generated in Photoshop, which
>>>>>>   has a novel pixel layout, and also contains additional tag types such
>>>>>>   as byte, and size specified ascii.
>>>>>> 
>>>>>>   Change-Id: Idb2e669e540cfbe27a932444011f4b9896ca403d
>>>>>> 
>>>>>> commit 795c71abb8b5c3a09d461ec7b34eefbfdcb460e9
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Mon Sep 22 15:08:36 2014 -0400
>>>>>> 
>>>>>>   BUG: Use array delete operator for array new allocations
>>>>>> 
>>>>>>   Change-Id: Ic9bc3c3792083a1a1b3fd842486a7436a4086150
>>>>>> 
>>>>>> commit 83e8b6e699b3d9e2da8533dd1ea36dc0692bc9eb
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Tue Sep 23 11:14:15 2014 -0400
>>>>>> 
>>>>>>   STYLE: Fix minor kwstyle defects in test and test results
>>>>>> 
>>>>>>   Change-Id: I4b2641d31acb5c764de4041bb7d21c08b3c99846
>>>>>> 
>>>>>> commit 869f3275120646829b27d5cd75c23800d9df01cd
>>>>>> Author: Richard Beare <Richard.Beare at ieee.org>
>>>>>> Date:   Tue Sep 23 11:11:34 2014 -0400
>>>>>> 
>>>>>>   ENH: Include TIFF tags in the MetaDataDictionary
>>>>>> 
>>>>>>   Add contribution from the Insight Journal:
>>>>>>   http://hdl.handle.net/10380/3170
>>>>>> 
>>>>>>   Change-Id: I20b02394d92c421bad75c5d0af496a6e80089ac0
>>>>>> 
>>>>>> commit c7a76c9acb319862647a651a2060dd3fb16560d7
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Mon Sep 22 14:46:44 2014 -0400
>>>>>> 
>>>>>>   BUG: Remove dead separated plannar code, add test
>>>>>> 
>>>>>>   The PLANARCONFIG_SEPARATE block in TIFFImageIO::ReadGenericImage, was
>>>>>>   preceded by a unreachable due to a conditional and exception
>>>>>>   check. Support for this configuration remains in the TIFFReadRGBAImage
>>>>>>   method. A test for the configuration has been added.
>>>>>> 
>>>>>>   A new input image is added from the from the libTiff test images:
>>>>>> 
>>>>>>   PlanarConfiguration = 2 (separated samples)
>>>>>>   -------------------------------------------
>>>>>>   oxford.tif      601x81 8-bit RGB (lzw) screendump off oxford
>>>>>> 
>>>>>>   Change-Id: Ibb874f80809b70b1aee7a2c9ff1fc8544ed56c04
>>>>>> 
>>>>>> commit 1b57d75964496dd2cc9e050a4a4425f1163d4b22
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Mon Sep 22 13:32:23 2014 -0400
>>>>>> 
>>>>>>   BUG: Fix right oriented tiff images
>>>>>> 
>>>>>>   The TIFFImageIO::ReadGenericImage method correctly handled top-left
>>>>>>   oriented images, and bottom-right image, however images oriented with
>>>>>>   right were non-correctly handled. These cases are now handled
>>>>>>   correctly by the TIFFRGBAImage generic path.
>>>>>> 
>>>>>>   New test images of the cthead1 image were added which have been
>>>>>>   flipped the to the different orientation, and the matching orientation
>>>>>>   tag specified so that they should all be read as equivalent
>>>>>>   images. These have been added as a test case.
>>>>>> 
>>>>>>   Change-Id: I058382d6d2e18341ab87edd165490be2187c1d15
>>>>>> 
>>>>>> commit 6263f4389026f1bd770054e86c428b56629d705b
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Thu Sep 18 16:20:41 2014 -0400
>>>>>> 
>>>>>>   ENH: Refactor duplicated code to read a page
>>>>>> 
>>>>>>   Created a method to read a single page or sub-image in a tiff
>>>>>>   file. Simular code was both in the ReadVolume method for 3-d and the
>>>>>>   Read method for 2-d. The code block from ReadVolume was used because
>>>>>>   it contained more typed cases. There for this patch does improve the
>>>>>>   2d reading capabilities for tiffio, so that they are consistent with
>>>>>>   the 3d.
>>>>>> 
>>>>>>   Change-Id: I0fbfb19f04435a703bb3fa09ea4e79179592e443
>>>>>> 
>>>>>> commit 233571f1e626917bce872e450907bc8e8521e371
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Thu Sep 18 14:59:58 2014 -0400
>>>>>> 
>>>>>>   ENH: Refactor method to convert RGBA image to output buffer
>>>>>> 
>>>>>>   BUG: Address overflow issue when computing page offset with
>>>>>>   TIFFReadRGBAImage method.
>>>>>> 
>>>>>>   Extract repeated code to convert from ReadRGBAImage to output image
>>>>>>   type.
>>>>>> 
>>>>>>   Change-Id: Ia90b5e104f22c77493b8a4cd5bf640fd793b8b4b
>>>>>> 
>>>>>> commit 1cd59825fa65f1123726a32f67c1a443015b40ec
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Thu Sep 18 11:30:41 2014 -0400
>>>>>> 
>>>>>>   BUG: Remove Zeiss 2-channel support code in TIFFImageIO ( and LSMImageIO
>>>>>> )
>>>>>> 
>>>>>>   There are no tests for this branch of code. It only works in 3D under
>>>>>>   certain cases. There are apparent bugs in the code such as not setting
>>>>>>   all of the input image, and logically dead branches. Some downloaded
>>>>>>   test images for similar described format do not load. This may effect
>>>>>>   files with the extensions tif, tiff, or lsm.
>>>>>> 
>>>>>>   Removal of this code will enable expansion of the current code to more
>>>>>>   generically support multi-sample per pixel images.
>>>>>> 
>>>>>>   For more robust reading of this type of image the SCIFIO remote
>>>>>>   module should be used.
>>>>>> 
>>>>>>   Change-Id: Idb98d69538db3bc8520881ea4a5e22476238c41e
>>>>>> 
>>>>>> commit 690cfdc89343fcc8c7d3cc266d652cfb14de9d41
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Tue Sep 16 15:31:52 2014 -0400
>>>>>> 
>>>>>>   ENH: Refactor GenericReadImage into template function
>>>>>> 
>>>>>>   Reduce code duplication by using template method.
>>>>>> 
>>>>>>   Change-Id: I79411ad6595258d54cbfa5a4152cc55994acd8e0
>>>>>> 
>>>>>> commit 5f73f24bfb668fa9f33804218c0db05cf5cadb3f
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Tue Sep 16 12:09:59 2014 -0400
>>>>>> 
>>>>>>   ENH: Refactor ReadTwoSamplePerPixelImage into template function
>>>>>> 
>>>>>>   Reduce code duplication by using template method.
>>>>>> 
>>>>>>   Change-Id: I45f417486e60a4d3e67ff31fa967dac069b3ce3e
>>>>>> 
>>>>>> commit 0b573136d636613217df9683ea846db8039f9312
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Tue Sep 16 11:12:45 2014 -0400
>>>>>> 
>>>>>>   ENH: Extract TiffReaderInternal to separate file
>>>>>> 
>>>>>>   Refactored private class to be in separate private header.
>>>>>> 
>>>>>>   Change-Id: Icb4008c1605a3a0b62c563350cf2e3bbe948ec6d
>>>>>> 
>>>>>> commit 6791b4a3613aade07e3f11f765f1316faf03b113
>>>>>> Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
>>>>>> Date:   Mon Sep 15 16:46:26 2014 -0400
>>>>>> 
>>>>>>   BUG: Fix overflows computing size of read tiff image
>>>>>> 
>>>>>>   When multiplying the height and width of an tiff image (32-bits),
>>>>>>   at least one element should be converted to size_t to help prevent
>>>>>>   numeric overflow.
>>>>>> 
>>>>>>   This change enables reading of larger RGB and large 2D image, that
>>>>>>   were previously truncated when reading. This does not address
>>>>>>   potential buffer overflow or resource limits which could occur.
>>>>>> 
>>>>>>   Change-Id: I192370606677c8dc1a903c67c2a0701026256d5a
>>>>>> 
>>>>>> commit b6fabfeb2a5b8d339b2d459edd3e3c50232b55ff
>>>>>> Author: Matt McCormick <matt.mccormick at kitware.com>
>>>>>> Date:   Mon Aug 25 00:35:48 2014 -0400
>>>>>> 
>>>>>>   COMP: Do not use _stat64 with MinGW-32.
>>>>>> 
>>>>>>   Unlike MinGW-w64, MinGW-32 aka MinGW, does not define as much of the API
>>>>>>   including _stat64.  Do not use use _stat64 in TIFFImageIO when building
>>>>>> with
>>>>>>   MinGW-32.
>>>>>> 
>>>>>>   Change-Id: I8f13a4500cdadf4f4adae7b5cf8893fe8b8cad55
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> Powered by www.kitware.com
>>>>>> 
>>>>>> Visit other Kitware open-source projects at
>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>> 
>>>>>> Kitware offers ITK Training Courses, for more information visit:
>>>>>> http://kitware.com/products/protraining.php
>>>>>> 
>>>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>>> 
>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>> http://public.kitware.com/mailman/listinfo/insight-developers
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Unpaid intern in BillsBasement at noware dot com
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Unpaid intern in BillsBasement at noware dot com
>> 
>> 
>> 
>> --
>> Unpaid intern in BillsBasement at noware dot com
> 
> 
> 
> -- 
> Unpaid intern in BillsBasement at noware dot com

_____________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://www.kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-users

More information about the Community mailing list