[Paraview] using ParaView for in-situ visualization
Andy Bauer
andy.bauer at kitware.com
Tue Feb 7 10:12:37 EST 2012
Hi Matthieu,
Please respond to the whole list so that everyone can share in the
conversation.
When you start to get into parallel with the structured grids, I suggest
you play around a bit with running pvserver in parallel, connect to it with
the GUI, and play with some of the sources like Wavelet and Mandelbrot.
As for comparisons with VisIt's in-situ visualization. We are working on
getting interaction through the GUI to process the data directly coming out
of the simulation without hitting the disk but it's not ready yet. There
would be 2 use cases for that. The first is where it stops the simulation
and waits for the user to finish their viz/analysis, i.e. like VisIt. The
second would be where the user changes the outputting pipeline while the
simulation code is running. The simulation code keeps running while the
user is setting up new pipelines and the changes don't take effect until
the next opportunity after the user adds their new pipeline or changes
parameters to their existing pipeline. Our design started working with the
use case that the user would submit their simulation+coprocessing as a
batch job. This makes sense if the simulation code is "fast". For
example, one of the simulation codes that we've been working with, PHASTA
led by Ken Jansen at UC Boulder, is able to do an implicit time step solve
(I believe it's implicit but don't quote me on that) in less than a second
for a problem with hundreds of millions of unknowns. We've done this with
up to 160,000 cores. While ParaView can still process that efficiently for
many filters, it may take the user minutes to actually examine the data.
That's obviously going to be a major waste of computational power if the
supercomputer is waiting on the user before computing the next time step.
In this case, the predefined pipeline works very well in that it completes
the requested coprocessing and immediately gives control back to the
simulation code to continue its processing. Hopefully this makes sense.
I've inlined some comments below to make sure that I address your questions.
Andy
On Tue, Feb 7, 2012 at 8:52 AM, Matthieu Dorier
<matthieu.dorier at gmail.com>wrote:
> Hi Andy,
>
> Thank you a lot.
>
> I'm not quiet sure to fully understand the SetWholeExtent stuff for
> parallel runs but I will certainly come back to you in the future when I
> effectively run in parallel.
>
> For now I have another question: I always assumed that in-situ
> visualization with ParaView was done in a way similar than VisIt, with a
> client that attempts to connect to perform interactive visualization. But
> reading again all the Paraview tutorial, it seems I was wrong.
>
> Can you please tell me if my understanding is right: ParaView only feeds a
> predefined pipeline (written in python or C++) with VTK objects, the
> "normal" end of a pipeline consists in writing output (images, for
> instance).
>
Yes, this is the current situation where you should consider the pipelines
that are created by the script generator gui plugin to be static and to
only output data files and screenshot images. You can modify the scripts
to add in better control though. We also are working on allowing the user
to change the pipeline but I don't know when this will be ready.
> Using data staging, the pipeline's output is redirected to the
> visualization cluster through sockets and the pvservers are used to
> interact with the user. So there is a "static" part from the simulation to
> staging where always the same coprocessing is performed, and an interactive
> part from staging to user where the user can request more specific
> visualization. Am I right?
>
This is also in the works where the data set or some subset of it will be
staged from the compute cluster to a viz cluster for the user to interact
with the "current" result. This will get done over the network and once
the data has been passed to the viz cluster, the compute cluster can
continue on with simulation computations, if that's what the user desires.
>
> Thanks,
>
> Matthieu
>
>
> 2012/2/6 Andy Bauer <andy.bauer at kitware.com>
>
>> Hi Matthieu,
>>
>> Your temperature array needs to be associated with your grid as point
>> data. The temperature array will be stored as a derived class of
>> vtkDataArray (I'll use a vtkDoubleArray). If you're storing your
>> temperature data as a contiguous array with the same size as the number of
>> points in your grid and you iterate through them such that you go through
>> the x-direction fastest, then the y-direction next fastest, and finally
>> z-direction, you'll probably want to use the SetArray() method to reuse the
>> memory you're already using for your native storage. Then your
>> wrapTemperature method would look like:
>>
>> vtkDataArray* wrapTemperature(double* temperature, vtkIdType
>> temperatureSize)
>> {
>> vtkDoubleArray* temperatureArray = vtkDoubleArray::New();
>> temperatureArray->SetName("Temperature");
>> temperatureArray->SetArray(temperature, temperatureSize, 1); // use 1
>> to tell VTK not to delete temperature when it doesn't need the array anymore
>> return temperatureArray;
>> }
>>
>> Alternatively, creating a new array would look like:
>> vtkDataArray* wrapTemperature(double* temperature, <more data>)
>> {
>> vtkDoubleArray* temperatureArray = vtkDoubleArray::New();
>> temperatureArray->SetNumberOfComponents(1); // assume temperature is a
>> scalar tensor
>> temperatureArray->SetNumberOfTuples(grid->GetNumberOfPoints());
>> temperatureArray->SetName("Temperature");
>> for(int zid=0;zid<PTZ;zid++)
>> for(int yid=0;yid<PTY;yid++)
>> for(int xid=0;xid<PTX;xid++)
>> temperatureArray->SetTupleValue(xid+yid*PTX+zid*PTX*PTY,
>> temperature+<corresponding index in array>);
>>
>> return temperatureArray;
>> }
>>
>> Then your wrapMeshData() would look like:
>> vtkDataObject* wrapMeshData()
>> {
>> vtkFloatArray* xCoords, yCoords, zCoords;
>> xCoords = vtkFloatArray::New();
>> xCoords->setArray(mesh_x,PTX,1);
>> yCoords = vtkFloatArray::New();
>> yCoords->setArray(mesh_y,PTY,1);
>> zCoords = vtkFloatArray::New();
>> zCoords->setArray(mesh_z,PTZ,1);
>> vtkRectilinearGrid *grid = vtkRectilinearGrid::New();
>> grid->setDimensions(PTX,PTY,PTZ);
>> grid->setXCoordinates(xCoords);
>> grid->setYCoordinates(yCoords);
>> grid->setZCoordinates(zCoords);
>> vtkDataArray* array = wrapTemperature();
>> grid->GetPointData()->AddArray(array);
>> array->Delete(); // decrement array reference counter
>> return (vtkDataObject*)grid;
>> }
>>
>> Finally, don't forget to call "vtkCPDataDescription::SetWholeExtent(0,
>> PTX-1, 0, PTY-1, 0, PTZ-1);". When running in parallel the input grid
>> should be partitioned and wrapMeshData() would also call SetExtent() with
>> that process's sub extent of the whole extent.
>>
>> Andy
>>
>>
>> On Mon, Feb 6, 2012 at 5:49 AM, Matthieu Dorier <
>> matthieu.dorier at gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I'm following the SC'10 tutorial with the help of the VTK doxygen, but I
>>> don't understand how to do the following:
>>>
>>> I have a 3D rectilinear grid which coordinates are defined by mesh_x,
>>> mesh_y and mesh_z (float arrays of dimensions PX, PY and PZ). I have a
>>> variable "temperature" defined as a 3D array representing the temperature
>>> at each point of the grid. Currently I have followed the slides 24 and 25
>>> from the SC'10 tutorial, at some point vtkDataObjects have to be created,
>>> here is how I created the grid object :
>>>
>>> // This function is called to retrieve the mesh
>>> vtkObject* wrapMeshData()
>>> {
>>> vtkFloatArray* xCoords, yCoords, zCoords;
>>> xCoords = vtkFloatArray::New();
>>> xCoords->setArray(mesh_x,PTX,1);
>>> yCoords = vtkFloatArray::New();
>>> yCoords->setArray(mesh_y,PTY,1);
>>> zCoords = vtkFloatArray::New();
>>> zCoords->setArray(mesh_z,PTZ,1);
>>> vtkRectilinearGrid *grid = vtkRectilinearGrid::New();
>>> grid->setDimensions(PTX,PTY,PTZ);
>>> grid->setXCoordinates(xCoords);
>>> grid->setYCoordinates(yCoords);
>>> grid->setZCoordinates(zCoords);
>>> return (vtkObject*)grid;
>>> }
>>>
>>> Now I want to make a function that retrieves the temperature so that I
>>> can map each data value to each point of the mesh, something like :
>>>
>>> vtkDataObject* wrapTemperature() {
>>> // ???
>>> }
>>>
>>> How can I do that without copying the original array that I want to wrap?
>>> Also what functions do I then call from the vtkCPDataDescription object?
>>> (on the tutorial they call SetGrid, I guess there is something else to call
>>> for the temperature field)
>>>
>>> Thank you for your help,
>>>
>>> Matthieu
>>>
>>> 2012/1/19 Berk Geveci <berk.geveci at kitware.com>
>>>
>>>> Please feel free to ask questions if anything is not clear or if you
>>>> need some help. We will be working on better documentation in the coming
>>>> few months.
>>>>
>>>> Best,
>>>> -berk
>>>>
>>>>
>>>> On Wed, Jan 18, 2012 at 12:08 PM, Matthieu Dorier <
>>>> matthieu.dorier at gmail.com> wrote:
>>>>
>>>>> Thank you, this tutorial will help a lot.
>>>>>
>>>>> Matthieu
>>>>>
>>>>>
>>>>> 2012/1/18 Andy Bauer <andy.bauer at kitware.com>
>>>>>
>>>>>> The current main wiki page for coprocessing with ParaView is at
>>>>>> http://paraview.org/Wiki/CoProcessing. In there it has links to 2
>>>>>> examples, a C++ driven example and a python driven example. You were
>>>>>> looking at the C++ example. There's also a powerpoint presentation and
>>>>>> some more code examples at
>>>>>> http://www.paraview.org/Wiki/SC10_Coprocessing_Tutorial. We will
>>>>>> but putting together a more extensive book but that probably won't be ready
>>>>>> for a month or two.
>>>>>>
>>>>>> My suggestion would be to first work on creating a vtkDataObject to
>>>>>> represent your data. Look at the powerpoint presentation for information
>>>>>> on doing that and then maybe the VTK doxygen (
>>>>>> http://www.vtk.org/doc/nightly/html/classes.html) for more specific
>>>>>> API questions.
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>> On Wed, Jan 18, 2012 at 5:12 AM, Matthieu Dorier <
>>>>>> matthieu.dorier at gmail.com> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> I have difficulties to understand how to instrument a simulation
>>>>>>> with ParaView in order to perform in-situ visualization.
>>>>>>> Is there any documentation other than the simple example provided at
>>>>>>> http://paraview.org/Wiki/Coprocessing_example ?
>>>>>>> Thank you,
>>>>>>>
>>>>>>> --
>>>>>>> Matthieu Dorier
>>>>>>> ENS Cachan, Brittany (Computer Science dpt.)
>>>>>>> IRISA Rennes, Office C113
>>>>>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Powered by www.kitware.com
>>>>>>>
>>>>>>> Visit other Kitware open-source projects at
>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>
>>>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>>>> http://paraview.org/Wiki/ParaView
>>>>>>>
>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>> http://www.paraview.org/mailman/listinfo/paraview
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Matthieu Dorier
>>>>> ENS Cachan, Brittany (Computer Science dpt.)
>>>>> IRISA Rennes, Office C113
>>>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
>>>>>
>>>>> _______________________________________________
>>>>> Powered by www.kitware.com
>>>>>
>>>>> Visit other Kitware open-source projects at
>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>
>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>> http://paraview.org/Wiki/ParaView
>>>>>
>>>>> Follow this link to subscribe/unsubscribe:
>>>>> http://www.paraview.org/mailman/listinfo/paraview
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Matthieu Dorier
>>> ENS Cachan, Brittany (Computer Science dpt.)
>>> IRISA Rennes, Office C113
>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
>>>
>>
>>
>
>
> --
> Matthieu Dorier
> ENS Cachan, Brittany (Computer Science dpt.)
> IRISA Rennes, Office C113
> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120207/d297d565/attachment-0001.htm>
More information about the ParaView
mailing list