[Paraview] Error when reading large XMLUnstructuredData file

[Berk Geveci]

This won't work. To write a dataset in parallel, you need to use a
partitioning filter such as D3 (vtkDistributedDataFilter). Try opening the
example dataset in ParaView, applying D3 and then saving as pvtu. Ideally,
you would directly write the data partitioned from the simulation. One
partition per rank maybe or you could sub-group using MPI to have fever
files. This is where you would get most scalability out of ParaView if you
are running it in parallel.

-berk

On Wed, Jan 5, 2011 at 2:38 AM, Guenter Bartsch <
guenter.bartsch at googlemail.com> wrote:

> robert,
>
> thanks for your quick response
>
> On Tue, Jan 4, 2011 at 6:16 PM, Robert Maynard
> <robert.maynard at kitware.com> wrote:
> > Hi,
> >
> > This is a known issue when a VTK XML file has an appended data section
> > larger than 2GB(http://public.kitware.com/Bug/view.php?id=6938).
>
> ah, that explains it
>
> > To work around the problem you can look at writing out the file as any of
> > the following:
> >
> > Partitioned VTK XML
> > Ensight Gold Binary
> > Exodus II ( Will require you too link to the exodus library found
> > at http://sourceforge.net/projects/exodusii/ )
>
> tried the first option for now - i assume partitioned vtk == parallel vtk?
>
>  vtkSmartPointer<vtkUnstructuredGridReader> reader =
> vtkSmartPointer<vtkUnstructuredGridReader>::New();
>  reader->SetFileName(filename);
>
>   char *ofilename = "foo.pvtu";
>   cout << "Output filename: " << ofilename << "\n";
>   vtkSmartPointer<vtkXMLPUnstructuredGridWriter> writer =
> vtkSmartPointer<vtkXMLPUnstructuredGridWriter>::New();
>  writer->SetFileName(ofilename);
>  writer->SetNumberOfPieces(8);
>   writer->SetInputConnection(reader->GetOutputPort());
>  writer->Write();
>
> and it works nicely for a smaller dataset (although it writes only one
> piece in that case). but, when i run it on the large data set, i get a
> segfault:
>
> (gdb) run
> /home/alex/projekte/1231/WP1/windfolding/windfolding1/windfield.vtk
> Starting program: /home/guenter/projects/vtk/vtk2xml
> /home/alex/projekte/1231/WP1/windfolding/windfolding1/windfield.vtk
> [Thread debugging using libthread_db enabled]
> Input filename:
> /home/alex/projekte/1231/WP1/windfolding/windfolding1/windfield.vtk
> Output filename: foo.pvtu
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x00000000006225a5 in vtkUnstructuredGrid::SetCells(int*, vtkCellArray*) ()
> (gdb) where
> #0  0x00000000006225a5 in vtkUnstructuredGrid::SetCells(int*,
> vtkCellArray*) ()
> #1  0x00000000005b33fd in
> vtkUnstructuredGridReader::RequestData(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) ()
> #2  0x00000000005d4252 in
> vtkDataReader::ProcessRequest(vtkInformation*, vtkInformationVector**,
> vtkInformationVector*) ()
> #3  0x00000000006077ca in vtkExecutive::CallAlgorithm(vtkInformation*,
> int, vtkInformationVector**, vtkInformationVector*) ()
> #4  0x000000000060279d in
> vtkDemandDrivenPipeline::ExecuteData(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) ()
> #5  0x00000000006066fa in
> vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) ()
> #6  0x000000000061f9fd in
> vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) ()
> #7  0x0000000000609a72 in vtkExecutive::ForwardUpstream(vtkInformation*) ()
> #8  0x0000000000606675 in
> vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) ()
> #9  0x000000000061f9fd in
> vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) ()
> #10 0x0000000000606249 in vtkDemandDrivenPipeline::UpdateData(int) ()
> #11 0x000000000061e4ad in vtkStreamingDemandDrivenPipeline::Update(int) ()
> #12 0x00000000006081ab in vtkExecutive::Update() ()
> #13 0x0000000000604815 in vtkDemandDrivenPipeline::Update() ()
> #14 0x00000000006169c1 in vtkStreamingDemandDrivenPipeline::Update() ()
> #15 0x00000000005e9d68 in vtkAlgorithm::Update() ()
> #16 0x00000000005c50fd in vtkXMLWriter::Write() ()
> #17 0x00000000005b18a5 in main (argc=2, argp=0x7fffffffcd58) at
> /home/guenter/projects/vtk/vtk2xml.cxx:41
>
> is this a bug or am i using the parallel writer incorrectly?
>
> thanks again and best regards,
>
>   guenter
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://www.paraview.org/mailman/listinfo/paraview
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20110105/a55dc3d9/attachment-0001.htm>