[Paraview] Data updates in Paraview+MPI

[Moreland, Kenneth]

Jing,

I'm still unclear about where this numpy array is being generated. Is it using the Python Shell in the GUI, running a Python script, or using a batch interpreter like pvpython or pvbatch? In that case, the numpy array is in a separate memory space as the data being loaded from the netCDF file (and has to be due to the distributed nature of parallel computers). There is a mechanism for pulling data to the Python script, but I doubt there is a way to push it back.

The calculator filter (as all other filters) has access to arrays that are either read from the netCDF file or generated in upstream filters. For example, the calculator filter will itself create an array that can be accessed by a second calculator if you add one to the first. So it is possible to add arrays after they are loaded.

One option you might try is to look into filters that internally use Python scripting. These are the Python Calculator and Programable Filter. Both of these will be able to access arrays from the netCDF file and then add arrays. If you can compute the values in the numpy array, then one or both of these should be able to add the data you want.

-Ken

From: Jing Li <jlih at masonlive.gmu.edu<mailto:jlih at masonlive.gmu.edu>>
Date: Tuesday, December 2, 2014 at 2:00 AM
To: Kenneth Moreland <kmorel at sandia.gov<mailto:kmorel at sandia.gov>>
Cc: "paraview at paraview.org<mailto:paraview at paraview.org>" <paraview at paraview.org<mailto:paraview at paraview.org>>
Subject: [EXTERNAL] Re: [Paraview] Data updates in Paraview+MPI

Hi Ken,
Thanks for your prompt response. Sorry about the confusion here.
I have a large Netcdf file which is visualized with Parallel Paraview. Now I have no problems with visualizing the values of an attribute( e.g., temperature). The next step is to modify the attribute values based on the another numpy array storing the weighting values. The weighting values are changing during the interactive manipulation.

For  the first question, I think I can resample the numpy data to make the dimensions the same.  I noticed there is a calculator for array data but it seems only to work with data read through Netcdf reader from Paraview. I guess the less efficient way is to store the scaling array to the original Netcdf file and use the calculator to generate a new array based on both original and weighting arrays.   Do you think this will work? I don't want to "modify" the core API of Paraview.

Thanks again.
Jing

On Dec 2, 2014, at 8:52 AM, Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>> wrote:

Jing,

I'm having a little trouble conceptualizing exactly how you are planning to use it, but I suspect you are going to run into several problems because it sounds like you are breaking some of the assumptions of VTK and ParaView.

First, I'm not sure how you could attach an array describing 2x2x2 values to 4x4x4 blocks. There is no convention in VTK to specify a field arrangement other than 1 to 1. All filters will assume the array is 4x4x4, which will cause bad data access and could potentially crash ParaView.

Second, trying to modify an array interactively in the view and then access it through filters in the pipeline is not going to work very well. The pipeline will not catch the changes and the changed values will not appear in the visualization. ParaView additionally has update suppressors to prevent updates that were not modified by the ParaView system.

I suspect that for whatever it is you are trying to do to work right you will need to create a custom view or representation that internally handles this numpy array.

-Ken

From: jlih <jlih at masonlive.gmu.edu<mailto:jlih at masonlive.gmu.edu>>
Date: Monday, December 1, 2014 at 9:03 AM
To: "paraview at paraview.org<mailto:paraview at paraview.org>" <paraview at paraview.org<mailto:paraview at paraview.org>>
Subject: [EXTERNAL] [Paraview] Data updates in Paraview+MPI


Dear all,

I am using the Paraview with MPI enabled to visualize a NetCDF dataset and need some suggestions on developing the following application.


If I want to modify the NetCDF array based on the data values from another numpy array (not in NetCDF format), do I need to store the numpy array in a NetCDF in every computing node with MPI installed?   The dimensions of the numpy array are not the same as the NetCDF array (e.g., numpy 2*2*2; NetCDF 4*4*4, one element from the numpy array corresponds to 8 elements in the NetCDF array ).


Also, the numpy array is updated during the interactive manipulation of the view.  Whenever the view changes (e.g., zoom), the values are updated.  If I store the numpy array as files, I am not sure how much communication overhead will be introduced. Is it possible to store the numpy array in memory and update the array on the fly? I'm not sure how the shared memory works in this case..


Any suggestions are highly appreciated. Thanks in advance.


Jing
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20141203/e1ae8088/attachment.html>