[Paraview] Convert Large CGNS Dataset to numpy

[Thomas PEDOT]

Thanks.

So it might come from something else. Python crashed with a 1.4gb file and
not with a smaller one.

I'm on windows with a 32bits version of python and launching it with
pycharm. I will test on a Linux cluster and/or with a 64bits version.

Thanks for your fast and clear answer !
Thomas

Le lun. 5 déc. 2016 17:49, Cory Quammen <cory.quammen at kitware.com> a écrit :

> On Mon, Dec 5, 2016 at 11:04 AM, Thomas PEDOT <thomas.pedot at gmail.com>
> wrote:
> > Hi,
> > I have some CGNS files I want to "convert" to numpy array. To write my
> > script I first use a more direct method :
> >
> > Converting CGNS to VTK
> > Use vtk_to_numpy method
> > process numpy array
> >
> > This task is time consuming and I need a more direct option (if
> possible).
> >
> > For this, I tried to use paraview CGNSReader and simple module.
> >
> > I'm able to convert CGNS data to numpy with small data set (Some MB) with
> > this approach :
> >
> > Case = CGNSSeriesReader(filename)
> > Case.UpdatePipeline()
> > rawData = servermanager.Fetch(Case)
> > import vtk.numpy_interface.dataset_adapter as dsa
> > # Wrap the raw data object to access NumPy friendly API
> > data = dsa.WrapDataObject(rawData)
> >
> > But as it said here : Do not do this if data is large otherwise the
> client
> > will run out of memory.
> >
> > So, I can I do this without running out of memory ? I don't need to load
> all
> > Point Arrays at once.
>
> That warning is more for if you've got some very large data set loaded
> into a ParaView server running on a large computing resource. If you
> call this, it will load all that data onto the client, which may not
> have enough RAM to hold the very large data set.
>
> If you are working locally with a data set that the ParaView client
> can load, then you shouldn't have any problems with the script above.
>
> > Is it possible to select just 'Temperature' for example and only one
> block ?
> > I guess it might be less memory consuming.
>
> To select just 'Temperature', you should be able to do something like:
>
> Case.PointVariables = ['Temperature']
>
> If you need to, you can add an ExtractBlock to your pipeline:
>
> extractBlock1 = ExtractBlock(Input=Case)
> extractBlock1.BlockIndices = [1]
>
> I probably wouldn't do the above unless you try your original script
> and run into out-of-memory errors.
>
> HTH,
> Cory
>
>
> --
> Cory Quammen
> Staff R&D Engineer
> Kitware, Inc.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20161205/f3381d5c/attachment.html>