[Paraview] using ParaView for in-situ visualization
Matthieu Dorier
matthieu.dorier at gmail.com
Tue Feb 7 10:21:13 EST 2012
Hi Andy,
Thank you for your answer (and sorry, I didn't notice I did a "reply"
instead of a "reply to all").
Matthieu
2012/2/7 Andy Bauer <andy.bauer at kitware.com>
> Hi Matthieu,
>
> Please respond to the whole list so that everyone can share in the
> conversation.
>
> When you start to get into parallel with the structured grids, I suggest
> you play around a bit with running pvserver in parallel, connect to it with
> the GUI, and play with some of the sources like Wavelet and Mandelbrot.
>
> As for comparisons with VisIt's in-situ visualization. We are working on
> getting interaction through the GUI to process the data directly coming out
> of the simulation without hitting the disk but it's not ready yet. There
> would be 2 use cases for that. The first is where it stops the simulation
> and waits for the user to finish their viz/analysis, i.e. like VisIt. The
> second would be where the user changes the outputting pipeline while the
> simulation code is running. The simulation code keeps running while the
> user is setting up new pipelines and the changes don't take effect until
> the next opportunity after the user adds their new pipeline or changes
> parameters to their existing pipeline. Our design started working with the
> use case that the user would submit their simulation+coprocessing as a
> batch job. This makes sense if the simulation code is "fast". For
> example, one of the simulation codes that we've been working with, PHASTA
> led by Ken Jansen at UC Boulder, is able to do an implicit time step solve
> (I believe it's implicit but don't quote me on that) in less than a second
> for a problem with hundreds of millions of unknowns. We've done this with
> up to 160,000 cores. While ParaView can still process that efficiently for
> many filters, it may take the user minutes to actually examine the data.
> That's obviously going to be a major waste of computational power if the
> supercomputer is waiting on the user before computing the next time step.
> In this case, the predefined pipeline works very well in that it completes
> the requested coprocessing and immediately gives control back to the
> simulation code to continue its processing. Hopefully this makes sense.
>
> I've inlined some comments below to make sure that I address your
> questions.
>
> Andy
>
> On Tue, Feb 7, 2012 at 8:52 AM, Matthieu Dorier <matthieu.dorier at gmail.com
> > wrote:
>
>> Hi Andy,
>>
>> Thank you a lot.
>>
>> I'm not quiet sure to fully understand the SetWholeExtent stuff for
>> parallel runs but I will certainly come back to you in the future when I
>> effectively run in parallel.
>>
>> For now I have another question: I always assumed that in-situ
>> visualization with ParaView was done in a way similar than VisIt, with a
>> client that attempts to connect to perform interactive visualization. But
>> reading again all the Paraview tutorial, it seems I was wrong.
>>
>> Can you please tell me if my understanding is right: ParaView only feeds
>> a predefined pipeline (written in python or C++) with VTK objects, the
>> "normal" end of a pipeline consists in writing output (images, for
>> instance).
>>
> Yes, this is the current situation where you should consider the pipelines
> that are created by the script generator gui plugin to be static and to
> only output data files and screenshot images. You can modify the scripts
> to add in better control though. We also are working on allowing the user
> to change the pipeline but I don't know when this will be ready.
>
>
>> Using data staging, the pipeline's output is redirected to the
>> visualization cluster through sockets and the pvservers are used to
>> interact with the user. So there is a "static" part from the simulation to
>> staging where always the same coprocessing is performed, and an interactive
>> part from staging to user where the user can request more specific
>> visualization. Am I right?
>>
> This is also in the works where the data set or some subset of it will be
> staged from the compute cluster to a viz cluster for the user to interact
> with the "current" result. This will get done over the network and once
> the data has been passed to the viz cluster, the compute cluster can
> continue on with simulation computations, if that's what the user desires.
>
>>
>> Thanks,
>>
>> Matthieu
>>
>>
>> 2012/2/6 Andy Bauer <andy.bauer at kitware.com>
>>
>>> Hi Matthieu,
>>>
>>> Your temperature array needs to be associated with your grid as point
>>> data. The temperature array will be stored as a derived class of
>>> vtkDataArray (I'll use a vtkDoubleArray). If you're storing your
>>> temperature data as a contiguous array with the same size as the number of
>>> points in your grid and you iterate through them such that you go through
>>> the x-direction fastest, then the y-direction next fastest, and finally
>>> z-direction, you'll probably want to use the SetArray() method to reuse the
>>> memory you're already using for your native storage. Then your
>>> wrapTemperature method would look like:
>>>
>>> vtkDataArray* wrapTemperature(double* temperature, vtkIdType
>>> temperatureSize)
>>> {
>>> vtkDoubleArray* temperatureArray = vtkDoubleArray::New();
>>> temperatureArray->SetName("Temperature");
>>> temperatureArray->SetArray(temperature, temperatureSize, 1); // use 1
>>> to tell VTK not to delete temperature when it doesn't need the array anymore
>>> return temperatureArray;
>>> }
>>>
>>> Alternatively, creating a new array would look like:
>>> vtkDataArray* wrapTemperature(double* temperature, <more data>)
>>> {
>>> vtkDoubleArray* temperatureArray = vtkDoubleArray::New();
>>> temperatureArray->SetNumberOfComponents(1); // assume temperature is
>>> a scalar tensor
>>> temperatureArray->SetNumberOfTuples(grid->GetNumberOfPoints());
>>> temperatureArray->SetName("Temperature");
>>> for(int zid=0;zid<PTZ;zid++)
>>> for(int yid=0;yid<PTY;yid++)
>>> for(int xid=0;xid<PTX;xid++)
>>> temperatureArray->SetTupleValue(xid+yid*PTX+zid*PTX*PTY,
>>> temperature+<corresponding index in array>);
>>>
>>> return temperatureArray;
>>> }
>>>
>>> Then your wrapMeshData() would look like:
>>> vtkDataObject* wrapMeshData()
>>> {
>>> vtkFloatArray* xCoords, yCoords, zCoords;
>>> xCoords = vtkFloatArray::New();
>>> xCoords->setArray(mesh_x,PTX,1);
>>> yCoords = vtkFloatArray::New();
>>> yCoords->setArray(mesh_y,PTY,1);
>>> zCoords = vtkFloatArray::New();
>>> zCoords->setArray(mesh_z,PTZ,1);
>>> vtkRectilinearGrid *grid = vtkRectilinearGrid::New();
>>> grid->setDimensions(PTX,PTY,PTZ);
>>> grid->setXCoordinates(xCoords);
>>> grid->setYCoordinates(yCoords);
>>> grid->setZCoordinates(zCoords);
>>> vtkDataArray* array = wrapTemperature();
>>> grid->GetPointData()->AddArray(array);
>>> array->Delete(); // decrement array reference counter
>>> return (vtkDataObject*)grid;
>>> }
>>>
>>> Finally, don't forget to call "vtkCPDataDescription::SetWholeExtent(0,
>>> PTX-1, 0, PTY-1, 0, PTZ-1);". When running in parallel the input grid
>>> should be partitioned and wrapMeshData() would also call SetExtent() with
>>> that process's sub extent of the whole extent.
>>>
>>> Andy
>>>
>>>
>>> On Mon, Feb 6, 2012 at 5:49 AM, Matthieu Dorier <
>>> matthieu.dorier at gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> I'm following the SC'10 tutorial with the help of the VTK doxygen, but
>>>> I don't understand how to do the following:
>>>>
>>>> I have a 3D rectilinear grid which coordinates are defined by mesh_x,
>>>> mesh_y and mesh_z (float arrays of dimensions PX, PY and PZ). I have a
>>>> variable "temperature" defined as a 3D array representing the temperature
>>>> at each point of the grid. Currently I have followed the slides 24 and 25
>>>> from the SC'10 tutorial, at some point vtkDataObjects have to be created,
>>>> here is how I created the grid object :
>>>>
>>>> // This function is called to retrieve the mesh
>>>> vtkObject* wrapMeshData()
>>>> {
>>>> vtkFloatArray* xCoords, yCoords, zCoords;
>>>> xCoords = vtkFloatArray::New();
>>>> xCoords->setArray(mesh_x,PTX,1);
>>>> yCoords = vtkFloatArray::New();
>>>> yCoords->setArray(mesh_y,PTY,1);
>>>> zCoords = vtkFloatArray::New();
>>>> zCoords->setArray(mesh_z,PTZ,1);
>>>> vtkRectilinearGrid *grid = vtkRectilinearGrid::New();
>>>> grid->setDimensions(PTX,PTY,PTZ);
>>>> grid->setXCoordinates(xCoords);
>>>> grid->setYCoordinates(yCoords);
>>>> grid->setZCoordinates(zCoords);
>>>> return (vtkObject*)grid;
>>>> }
>>>>
>>>> Now I want to make a function that retrieves the temperature so that I
>>>> can map each data value to each point of the mesh, something like :
>>>>
>>>> vtkDataObject* wrapTemperature() {
>>>> // ???
>>>> }
>>>>
>>>> How can I do that without copying the original array that I want to
>>>> wrap?
>>>> Also what functions do I then call from the vtkCPDataDescription
>>>> object? (on the tutorial they call SetGrid, I guess there is something else
>>>> to call for the temperature field)
>>>>
>>>> Thank you for your help,
>>>>
>>>> Matthieu
>>>>
>>>> 2012/1/19 Berk Geveci <berk.geveci at kitware.com>
>>>>
>>>>> Please feel free to ask questions if anything is not clear or if you
>>>>> need some help. We will be working on better documentation in the coming
>>>>> few months.
>>>>>
>>>>> Best,
>>>>> -berk
>>>>>
>>>>>
>>>>> On Wed, Jan 18, 2012 at 12:08 PM, Matthieu Dorier <
>>>>> matthieu.dorier at gmail.com> wrote:
>>>>>
>>>>>> Thank you, this tutorial will help a lot.
>>>>>>
>>>>>> Matthieu
>>>>>>
>>>>>>
>>>>>> 2012/1/18 Andy Bauer <andy.bauer at kitware.com>
>>>>>>
>>>>>>> The current main wiki page for coprocessing with ParaView is at
>>>>>>> http://paraview.org/Wiki/CoProcessing. In there it has links to 2
>>>>>>> examples, a C++ driven example and a python driven example. You were
>>>>>>> looking at the C++ example. There's also a powerpoint presentation and
>>>>>>> some more code examples at
>>>>>>> http://www.paraview.org/Wiki/SC10_Coprocessing_Tutorial. We will
>>>>>>> but putting together a more extensive book but that probably won't be ready
>>>>>>> for a month or two.
>>>>>>>
>>>>>>> My suggestion would be to first work on creating a vtkDataObject to
>>>>>>> represent your data. Look at the powerpoint presentation for information
>>>>>>> on doing that and then maybe the VTK doxygen (
>>>>>>> http://www.vtk.org/doc/nightly/html/classes.html) for more specific
>>>>>>> API questions.
>>>>>>>
>>>>>>> Andy
>>>>>>>
>>>>>>> On Wed, Jan 18, 2012 at 5:12 AM, Matthieu Dorier <
>>>>>>> matthieu.dorier at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I have difficulties to understand how to instrument a simulation
>>>>>>>> with ParaView in order to perform in-situ visualization.
>>>>>>>> Is there any documentation other than the simple example provided
>>>>>>>> at http://paraview.org/Wiki/Coprocessing_example ?
>>>>>>>> Thank you,
>>>>>>>>
>>>>>>>> --
>>>>>>>> Matthieu Dorier
>>>>>>>> ENS Cachan, Brittany (Computer Science dpt.)
>>>>>>>> IRISA Rennes, Office C113
>>>>>>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Powered by www.kitware.com
>>>>>>>>
>>>>>>>> Visit other Kitware open-source projects at
>>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>>
>>>>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>>>>> http://paraview.org/Wiki/ParaView
>>>>>>>>
>>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>>> http://www.paraview.org/mailman/listinfo/paraview
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Matthieu Dorier
>>>>>> ENS Cachan, Brittany (Computer Science dpt.)
>>>>>> IRISA Rennes, Office C113
>>>>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
>>>>>>
>>>>>> _______________________________________________
>>>>>> Powered by www.kitware.com
>>>>>>
>>>>>> Visit other Kitware open-source projects at
>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>
>>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>>> http://paraview.org/Wiki/ParaView
>>>>>>
>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>> http://www.paraview.org/mailman/listinfo/paraview
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Matthieu Dorier
>>>> ENS Cachan, Brittany (Computer Science dpt.)
>>>> IRISA Rennes, Office C113
>>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
>>>>
>>>
>>>
>>
>>
>> --
>> Matthieu Dorier
>> ENS Cachan, Brittany (Computer Science dpt.)
>> IRISA Rennes, Office C113
>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
>>
>
>
--
Matthieu Dorier
ENS Cachan, Brittany (Computer Science dpt.)
IRISA Rennes, Office C113
http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120207/9e957f5a/attachment-0001.htm>
More information about the ParaView
mailing list