[Paraview] loading nc formatted data

Fri Mar 18 21:25:21 EDT 2011

Oh, yea.  If you feel that vtkNetCDFCFReader should be closing your seems and you want to add it, OK.  But please be mindful not to break the CF convention.

-Ken

   ****      Kenneth Moreland
    ***      Sandia National Laboratories
***********
*** *** ***  email: kmorel at sandia.gov
**  ***  **  phone: (505) 844-8919
    ***      web:   http://www.cs.unm.edu/~kmorel

From: Kenneth Moreland <kmorel at sandia.gov<mailto:kmorel at sandia.gov>>
Date: Sat, 19 Mar 2011 01:18:05 +0000
To: Simon Su <newsgroup4ssu at gmail.com<mailto:newsgroup4ssu at gmail.com>>, Andy Bauer <andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>>
Cc: "paraview at paraview.org<mailto:paraview at paraview.org>" <paraview at paraview.org<mailto:paraview at paraview.org>>
Subject: Re: [Paraview] loading nc formatted data

The reader that is being used is vtkNetCDFCFReader.  It is defined in ParaView/Servers/ServerManager/Resources/readers.xml, although it is not obvious.

As the name implies, this reader reads netCDF files using the CF convention.  (As the default netCDF reader, it also gracefully handles files that do not follow this convention.) With this assumption, I will try to explain what it does.  It reads arrays as regular 1, 2, or 3D arrays, possibly with time.  The CF convention also provides a means to assign coordinates to each grid point and to identify the coordinates as longitude or latitude.

Even though coordinates are defined as longitude and latitude, the topology of the grid itself is still a grid.  Thus, the grid gets wrapped around, but still has these seems that you see because topologically the one end of the regular grid is not attached to the other.  I can't think of any filter that will identify and close these seems.  In fact, it's not straightforward to do at all.  If you look at your topology, it is not lain out on a simple spherical grid.

The "right" way to solve your problem, which may or may not be in your control, is to create netCDF files that specify cell boundaries for a closed topology.  Your netCDF file must be following at least some parts of the CF convention; your data would not show up as a sphere if it were not.  The CF convention provides a way of defining cells that are not constrained by a regular grid topology.  It is done through a "bounds" attribute on the dimension descriptor variables.  You need either 1D or 2D bounds.  They are described in this section of the CF convention documentation: http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#cell-boundaries.

The VTK test data has examples of both 1D bounds (tos_O1_2001-2002.nc) and 2D bounds (sampleCurveGrid4.nc).  You can get the VTK test data from git (http://vtk.org/VTKData.git), or download them directly from the gitourious server:

http://vtk.org/gitweb?p=VTKData.git;a=blob;f=Data/tos_O1_2001-2002.nc;h=30aa4a9b5e08b9bdf64540f2b144d83b279cca6c;hb=HEAD

http://vtk.org/gitweb?p=VTKData.git;a=blob;f=Data/sampleCurveGrid4.nc;h=0ab89c27a25f92c047b58dca8b3057ca8d4df017;hb=HEAD

-Ken

   ****      Kenneth Moreland
    ***      Sandia National Laboratories
***********
*** *** ***  email: kmorel at sandia.gov
**  ***  **  phone: (505) 844-8919
    ***      web:   http://www.cs.unm.edu/~kmorel

From: Simon Su <newsgroup4ssu at gmail.com<mailto:newsgroup4ssu at gmail.com>>
Date: Fri, 18 Mar 2011 16:43:42 -0400
To: Andy Bauer <andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>>
Cc: <paraview at paraview.org<mailto:paraview at paraview.org>>
Subject: Re: [Paraview] loading nc formatted data

Hi Andy,

python trace gave me

zos_Omon_GFDLESM2M_historical_r1i1p1_186101188012_nc = NetCDFReader( FileName=['/work/sms/data/cmor-20110128/mon/ocean/zos/r1i1p1/zos_Omon_GFDL-ESM2M_historical_r1i1p1_186101-188012.nc'] )

and there are tons of netcdf reader in ParaView and it is not in the plugin directory

sms:/local/home/build/paraview/ParaView-3.10.0/Plugins> pwd
/local/home/build/paraview/ParaView-3.10.0/Plugins
sms:/local/home/build/paraview/ParaView-3.10.0/Plugins> ll
total 84
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 AdiosReader/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 AnalyzeNIfTIReaderWriter/
-rw-r--r-- 1 sms t 3261 Mar  9 13:31 CMakeLists.txt
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 CoProcessingScriptGenerator/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 CosmoFilters/
drwxr-xr-x 4 sms t 4096 Mar 10 12:37 EyeDomeLighting/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 ForceTime/
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 H5PartReader/
drwxr-xr-x 4 sms t 4096 Mar 10 12:37 Manta/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 Moments/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 NetDMFReader/
drwxr-xr-x 6 sms t 4096 Mar 10 12:37 PointSprite/
drwxr-xr-x 4 sms t 4096 Mar 10 12:37 PrismPlugins/
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 pvblot/
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 SierraPlotTools/
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 SLACTools/
drwxr-xr-x 4 sms t 4096 Mar 10 12:37 StreamingView/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 SurfaceLIC/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 Vapor/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 VRPN/
drwxr-xr-x 2 sms t 4096 Mar  9 13:31 VRUI/
sms:/local/home/build/paraview/ParaView-3.10.0/Plugins>

Can you help describe how Netcdf files plugins are done in ParaView. Do they have a super class of Netcdf that they all derive from to write the different flavors of netcdf readers? If so, where can the code be found?

thanks
-simon

On Fri, Mar 18, 2011 at 10:30 AM, Andy Bauer <andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>> wrote:
Hi Simon,

Replies below...

On Thu, Mar 17, 2011 at 12:41 PM, Simon Su <newsgroup4ssu at gmail.com<mailto:newsgroup4ssu at gmail.com>> wrote:
Hi All,

ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/1.png
ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/2.png
ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/3.png

the above are screen shot from older version of *.nc files that I have that I loaded up with Paraview 3.10.0 64-bit which I compiled myself. As you can see, the grid is correctly loaded. But in the data, there is a crack. Is there a filter that can fix this in ParaView? :)

I'm not aware of any filter that will fix this automatically.   Can you describe the grid a bit more?  It kind of looks like a multiblock of structured grids.

ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/4.png is the latest nc file that I have of similar simulation preprocessed output. When I loaded it up, it is clearly doing making assumption on the grid that is not correct and hence, the double looking grid.

The questions now are:

1. ParaView has lots of *.nc file loader. How do I know which loader is ParaView using to load the data? If I pick a type for the Files of type option in the open File window, will that gurantees that ParaView will be using that particular file loader?

If there is an ambiguity for which file loader to use (i.e. multiple readers assume the same extension), then the GUI should pop up a dialog for you to specify which one to use.  You can use the python trace to figure out exactly what reader is being used to load the file.

2. If I know which loader is used, I would like to see if I can modify the existing loader to create a new type of *.nc file loader to fix the grid of my data. Where is the code in the source tree of ParaView is the loader plugin placed?
Based on the name of the name of the reader from the python script, you can look up the actual class name in the ParaView/Servers/ServerManager/Resources/readers.xml file.

or if there is an easier solution to fix my grid problem that doesn't involve developing a new data loader plugin that would be better.... :)  please let me know...

thanks
-simon

_______________________________________________
Powered by www.kitware.com<http://www.kitware.com>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview

_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20110319/7d2a1752/attachment.htm>