[vtkusers] saving memory for XML writers?

[tom fogal]

 <a530bfb505062408253691c8f6 at mail.gmail.com>Peter Schmitt writes:
>It sounds like you open the file and load each datapoint into memory. 
>Once the file has been completely read, you then create the grid and
>store the data.  If that's the case, you may be able to improve your
>performance by trying something along the lines of what I haev done:
>(note: this is for a structuredgrid, for rectilinear grid you will
>need to set the spacing arrays, obviously)
>1.  Scan file to detemine dimensions of my grid
>2.  Create grid, set dimensions
>3.  Create point array, set dimensions
>4.  Scan file, insert each point using its proper integer offset
>	vtkPoints::InsertPoint (vtkIdType id, const float x[3])
>
>This way vtk manages all of the memory for the grid, while you only
>manage memory for one datapoint at a time.

>Does that help?

Hrm, thats certainly more difficult than what I'm doing now =). I
follow your logic though, and yes I imagine that will save a lot of
memory.

I'll let you know after I've implemented it =)

Thanks,

-tom

>On 6/24/05, tom fogal <tfogal at apollo.sr.unh.edu> wrote:
>> Hi all, I've got a data file convertor that takes a proprietary file
>> format and outputs VTK XML formats. For a typical 1.6gig input file, it
>> unfortunately takes almost 6gigs of memory at its peak. I'd like to
>> lower this significantly so we can use it on workstations.
>> 
>> Just reading in the data file takes about 3gigs (as I would expect).
>> Then, I build a giant RectilinearGrid, and pass that to an XML writer.
>> I would like to know if I could build the RectilinearGrid piecemeal,
>> that is, define a grid, write out the grid, delete grid data, write out
>> one of the three scalar fields, delete it, write out another field,
>> delete it, etc.. Basically I would like to keep the program overhead to
>> sizeof(data) + sizeof(one_field_in(data)). Eventually I hope to fix the
>> proprietary reader so it can read in a single field at a time...
>> 
>> Does anyone know of a way I could do this, or have any other ideas to
>> save memory?
>> 
>> Thanks,
>> 
>> -tom
>> _______________________________________________
>> This is the private VTK discussion list.
>> Please keep messages on-topic. Check the FAQ at: http://www.vtk.org/Wiki/VTK_FAQ
>> Follow this link to subscribe/unsubscribe:
>> http://www.vtk.org/mailman/listinfo/vtkusers
>>
>_______________________________________________
>This is the private VTK discussion list. 
>Please keep messages on-topic. Check the FAQ at: http://www.vtk.org/Wiki/VTK_FAQ
>Follow this link to subscribe/unsubscribe:
>http://www.vtk.org/mailman/listinfo/vtkusers
>