[Insight-developers] Design question

[Daniel Blezek]

Hi All,

   I'm trying to design a class for ITK that would allow out-of-core  
processing of extremely large datasets.  Often fMRI datasets can be  
several hundred 3d volumes and it is not practical, nor necessary to  
load all the image data into memory at once in order to process.   
Instead, I'd like to write a class that would transform a series of  
images into a different format on disk to allow rapid retrieval of  
subsets of the data, i.e.,

1) volume by volume through time
2) one slice through time, in any orientation, e.g. x-y, y-z or x-z
3) one pixel through time

   The idea is to provide more of a database interface rather than a  
fancy 4D reader/write IO class.

   I hope to provide an iterator that would allow the application  
developer to iterate across the voxels, retrieving one voxel's  
timeseries data in a rapid manner.  This, I believe is the most common  
way of processing fMRI data.  The developer may also wish to align  
that data to a common space making volume by volume retrieval  
imperative.

I've done some preliminary work playing around with different methods  
of storing these data on disk.  I used raw C fread/fwrite, HDF5, and  
SQLite.  The results are summarized here: http://wiki.na-mic.org/Wiki/index.php/Slicer3:TimeSeries_and_Multi-Volume_Data#Proposed_Data_Model

I would appreciate any and all comments.
-dan