[Paraview] Large vtr files not read - The data array in theelement may be too short

Mon Jul 30 06:15:53 EDT 2012

A follow up...

To build I changed the linking options to:
-lvtkCommonCore-6.0 -lvtkCommonDataModel-6.0 -lvtkIOCore-6.0 -lvtkIOXML-6.0
-lvtkRenderingCore-6.0

And the code to:
  //writer->SetInputConnection(grid->GetProducerPort());
  writer->SetHeaderTypeToUInt64();
  writer->SetInputData(grid);

I can now write a large vtr file using VTK6 and have ParaView
3.14.1-1300-g0596919 read the file with no errors. However as the data gets
larger ParaView is more unstable. I found that I had to start using ParaView
in parallel to prevent seg faults. 

So it is working thanks :)

Paul

  _____  

From: paraview-bounces at paraview.org [mailto:paraview-bounces at paraview.org]
On Behalf Of Paul McIntosh
Sent: Sunday, 29 July 2012 10:12 AM
To: 'Andy Bauer'
Cc: paraview at paraview.org
Subject: Re: [Paraview] Large vtr files not read - The data array in
theelement may be too short

Hi Andy,

Thanks - I am just trying to get my build working with the new VTK 6 library
system and if it works I'll post the updated code.

Cheers,

Paul

  _____  

From: Andy Bauer [mailto:andy.bauer at kitware.com] 
Sent: Thursday, 26 July 2012 12:35 AM
To: Paul McIntosh
Cc: paraview at paraview.org
Subject: Re: [Paraview] Large vtr files not read - The data array in the
element may be too short

Hi,

If you're using VTK master to create your VTR files you can use the
SetHeaderTypeToUInt64() method to write out these large data sets. ParaView
master will be able to read them in properly. Unfortunately though this has
been fixed since the 3.14.1 release so it won't work there. We're working on
fixing this limitation properly for the next ParaView release.

Andy

On Wed, Jul 25, 2012 at 3:47 AM, Paul McIntosh
<paul.mcintosh at internetscooter.com> wrote:

Hi All,

How does one go about writing large rectilinear data files (vtr) that are
suitable for reading by ParaView?

I have been trying using VTK and found that there is a size limit(?).
Googling showed that there was a bug in 2005 that should have been fixed but
I am still getting an error when I go above a certain size (even with 3.14
64bit precompiled). I have adapted a VTK example and reproduced the problem
- below is the error message followed by the code. "size = 1000" works,
"size = 1200" produced the error message, "size = 2000" ParaView just
segfaults.

Do I have to change it to use parallel writers?

Cheers,

Paul
---
www.internetscooter.com

ERROR: In
/usr/local/src/PARAVIEW/3.14.0/ParaView-3.14.0-Source/VTK/IO/vtkXMLStructure
dDataReader.cxx, line 325
vtkXMLRectilinearGridReader (0x1b2b54b0): Error reading extent 0 1199 0 1199
0 1199 from piece 0

ERROR: In
/usr/local/src/PARAVIEW/3.14.0/ParaView-3.14.0-Source/VTK/IO/vtkXMLDataReade
r.cxx, line 510
vtkXMLRectilinearGridReader (0x1b2b54b0): Cannot read point data array "Da"
from PointData in piece 0.  The data array in the element may be too short.

ERROR: In
/usr/local/src/PARAVIEW/3.14.0/ParaView-3.14.0-Source/VTK/IO/vtkXMLStructure
dDataReader.cxx, line 325
vtkXMLRectilinearGridReader (0x1b2b54b0): Error reading extent 0 1199 0 1199
0 1199 from piece 0

ERROR: In
/usr/local/src/PARAVIEW/3.14.0/ParaView-3.14.0-Source/VTK/IO/vtkXMLDataReade
r.cxx, line 510
vtkXMLRectilinearGridReader (0x1b2b54b0): Cannot read point data array "Da"
from PointData in piece 0.  The data array in the element may be too short.

ERROR: In
/usr/local/src/PARAVIEW/3.14.0/ParaView-3.14.0-Source/VTK/IO/vtkXMLStructure
dDataReader.cxx, line 325
vtkXMLRectilinearGridReader (0x1b2b54b0): Error reading extent 0 1199 0 1199
0 1199 from piece 0

ERROR: In
/usr/local/src/PARAVIEW/3.14.0/ParaView-3.14.0-Source/VTK/IO/vtkXMLDataReade
r.cxx, line 510
vtkXMLRectilinearGridReader (0x1b2b54b0): Cannot read point data array "Da"
from PointData in piece 0.  The data array in the element may be too short.

ERROR: In
/usr/local/src/PARAVIEW/3.14.0/ParaView-3.14.0-Source/VTK/IO/vtkXMLStructure
dDataReader.cxx, line 325
vtkXMLRectilinearGridReader (0x1b2b54b0): Error reading extent 0 1199 0 1199
0 1199 from piece 0

ERROR: In
/usr/local/src/PARAVIEW/3.14.0/ParaView-3.14.0-Source/VTK/IO/vtkXMLDataReade
r.cxx, line 510
vtkXMLRectilinearGridReader (0x1b2b54b0): Cannot read point data array "Da"
from PointData in piece 0.  The data array in the element may be too short.

// Qt (used for other code - not relevant here)
#include <QCoreApplication>
#include <QtEndian>
#include <QDebug>
#include <QStringList>

// C++
#include <list>
#include <iostream>
#include <fstream>
using namespace std;

// VTK
#include <vtkVersion.h>
#include <vtkSmartPointer.h>
#include <vtkDoubleArray.h>
#include <vtkFloatArray.h>
#include <vtkPointData.h>
#include <vtkCellData.h>
#include <vtkStructuredGrid.h>
#include <vtkRectilinearGrid.h>
#include <vtkXMLRectilinearGridWriter.h>
#include <vtkDataSetMapper.h>
#include <vtkXMLStructuredGridWriter.h>
#include <vtkXMLRectilinearGridReader.h>
#include <vtkSmartPointer.h>
#include <vtkProperty.h>
#include <vtkDataSetMapper.h>
#include <vtkXMLRectilinearGridReader.h>
#include <vtkRectilinearGridGeometryFilter.h>
#include <vtkRenderWindow.h>
#include <vtkRenderWindowInteractor.h>
#include <vtkRenderer.h>

#include <vtkDataSetMapper.h>
#include <vtkActor.h>
#include <vtkRenderWindow.h>
#include <vtkRenderer.h>
#include <vtkRenderWindowInteractor.h>

int main(int, char *[])
{
  long int size = 1200;
  // Create a grid
  vtkSmartPointer<vtkRectilinearGrid> grid =
vtkSmartPointer<vtkRectilinearGrid>::New();

  grid->SetDimensions(size,size,size);

  vtkSmartPointer<vtkDoubleArray> xArray =
vtkSmartPointer<vtkDoubleArray>::New();
  xArray->SetNumberOfComponents(1);
  vtkSmartPointer<vtkDoubleArray> yArray =
vtkSmartPointer<vtkDoubleArray>::New();
  yArray->SetNumberOfComponents(1);
  vtkSmartPointer<vtkDoubleArray> zArray =
vtkSmartPointer<vtkDoubleArray>::New();
  zArray->SetNumberOfComponents(1);

  // Container for data
  vtkSmartPointer<vtkFloatArray> DaArray =
vtkSmartPointer<vtkFloatArray>::New();
  DaArray->SetNumberOfComponents(1);
  DaArray->SetName("Da");

  for (long int i = 0; i < size; i++)
  {
      xArray->InsertNextValue(i);
      yArray->InsertNextValue(i);
      zArray->InsertNextValue(i);
  }

  grid->SetXCoordinates(xArray);
  grid->SetYCoordinates(yArray);
  grid->SetZCoordinates(zArray);

  for (long int i = 0; i < size*size*size; i++)
  {
        DaArray->InsertNextValue(i);
  }
  grid->GetPointData()->SetScalars(DaArray);

  std::cout << "There are " << grid->GetNumberOfPoints()
            << " points." << std::endl;
  std::cout << "There are " << grid->GetNumberOfCells()
            << " cells." << std::endl;

  vtkSmartPointer<vtkXMLRectilinearGridWriter> writer =
vtkSmartPointer<vtkXMLRectilinearGridWriter>::New();
  //writer->SetFileName("rectilineargrid.vtr");
  writer->SetFileName("test.vtr");
  writer->SetInputConnection(grid->GetProducerPort());
  cout << "Writing " << "test.vtr" << "..." << endl;
  writer->Write();
  cout << "Wrote!" << endl;

  // Create a mapper and actor
  vtkSmartPointer<vtkDataSetMapper> mapper =
vtkSmartPointer<vtkDataSetMapper>::New();
  mapper->SetInputConnection(grid->GetProducerPort());

  vtkSmartPointer<vtkActor> actor = vtkSmartPointer<vtkActor>::New();
  actor->SetMapper(mapper);

  // Visualize
  vtkSmartPointer<vtkRenderer> renderer =
vtkSmartPointer<vtkRenderer>::New();
  vtkSmartPointer<vtkRenderWindow> renderWindow =
vtkSmartPointer<vtkRenderWindow>::New();
  renderWindow->AddRenderer(renderer);
  vtkSmartPointer<vtkRenderWindowInteractor> renderWindowInteractor =
vtkSmartPointer<vtkRenderWindowInteractor>::New();
  renderWindowInteractor->SetRenderWindow(renderWindow);

  renderer->AddActor(actor);
  renderer->SetBackground(.3, .6, .3); // Background color green

  renderWindow->Render();
  renderWindowInteractor->Start();

  return EXIT_SUCCESS;
}

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120730/2704cb97/attachment-0001.htm>