[Paraview] CoProcessing

Thu May 5 09:23:38 EDT 2011

Hi Andrey,

For 1), you need to pass pvserver a .pvx file on the command line.  You can
put the pvx filename at the very end of the pvserver command line.  The pvx
file should contain the hostname or IP address of each pvserver process.  So
if there will be two pvserver processess running on the machines
myserverhost0 and myserverhost1, then the pvx file would look like:

<?xml version="1.0" ?>
<pvx>
  <Process Type="server">
    <Machine Name="myserverhost0" />
    <Machine Name="myserverhost1" />
  </Process>
</pvx>

When you have this pvx file, the pvserver to send these hostnames to the
coprocessor, this allows the coprocessor nodes to know which hosts they
should connect to.  You also need to make sure that your cpscript.py
contains the right hostname for the pvserver root process.  So the cpscript
would contain:

host = "myserverhost0"
port = 22222

Also, make sure you have the latest commits in the live-data branch on
github, because I added a fix related to this pvx file.

For 2), the coprocessor code can send any type of dataset to the pvserver,
so unstructured grid should be fine.  But, if you are doing a data
aggregation step on the coprocessor, then that code has only been
implemented for vtkPolyData.  For example, if you have 1000 simulation procs
and 10 pvserver procs, then the coprocessor will need to aggregate all the
data from 1000 procs to 10 simulation procs, then those 10 simulation procs
will send their data to the 10 pvserver procs.

I have new code that allows you to write your own custom aggregation
routine.  What does your unstructured grid look like?  If it is something
with homogeneous cell type, like all tetrahedron, then it will be really
easy to modify some example code I have to work for your data.  Also, I
added the ability to open multiple sockets on the pvserver process.  So if
each of the 10 pvserver procs open 20 sockets, then there will be 200
sockets available to the coprocessor, so then the data only needs to be
aggregated from 1000 to 200 procs before sending to pvserver.  This code has
not made it's way onto github, but I could add it today.  Let me know if you
would benefit from this.

Pat

On Thu, May 5, 2011 at 3:28 AM, Andrey Lomtev <andreylomtev at gmail.com>wrote:

>  On 26.01.2011 20:50, pat marion wrote:
>
> Hi,
>
> I could help you better if you could get a backtrace of the crash.  It's
> hard to guess where it might be segfaulting.  It's possibly an issue with
> MPI though.  The coprocessor tries to initialize mpi, so if you are running
> the regular python program, mpi might not be available.
>
> Pat
>
> On Wed, Jan 26, 2011 at 12:45 PM, Andrey Lomtev <andreylomtev at gmail.com>wrote:
>
>>   On 23.01.2011 23:07, pat marion wrote:
>>
>> Hi Andrey,
>>
>> Once you have run through the steps to generate a coprocessing python
>> script, described here:
>>
>>
>> http://www.paraview.org/Wiki/CoProcessing#Running_the_CoProcessing_Script_Generator
>>
>> If you look at the top of the generated script, it will have these
>> options:
>>
>> host = "localhost"
>> port = 22222
>>
>> set_use_network(True)
>> set_do_reduce_data(True)
>> set_do_writing(False)
>> set_use_psets(False)
>> set_log_messages(False)
>> set_procs_per_partition(32)
>> set_final_partition_size(1)
>>
>> So you just need to edit the two lines with host and port.  You can ignore
>> the other options, they are set by default to do the standard live-data use
>> case.  When you generate the coprocessing script, you will add writers to
>> the end of your pipeline(s).  A writer represents the end of the pipeline,
>> the sink.  Instead of data flowing into the writer, it will be re-directed
>> over the network to the Live Data Source on the pvserver.
>>
>> If the number of coprocessor nodes is greater than the number of pvserver
>> nodes, then the coprocessor will have to aggregate data to a subset its
>> nodes that perform the network sends.  The aggregation code currently
>> expects vtkPolyData and won't work with anything else.
>>
>> Pat
>>
>>
>> On Sat, Jan 22, 2011 at 5:44 AM, Andrey Lomtev <andreylomtev at gmail.com>wrote:
>>
>>>   On 12.01.2011 22:24, pat marion wrote:
>>>
>>> Hi Andrey,
>>>
>>> When I wrote those slides, the code for the Live Data Source had not yet
>>> received approval for public release.  The code is now available but has yet
>>> to be merged with the paraview main release.  You'll have to compile
>>> paraview yourself in order to use the Live Data Source.  The code can be
>>> found in the live-data branch of the git repository git://
>>> github.com/patmarion/ParaView.git
>>>
>>> Pat
>>>
>>> On Wed, Jan 12, 2011 at 1:07 PM, Andrey Lomtev <andreylomtev at gmail.com>wrote:
>>>
>>>>  Hello everyone,
>>>>
>>>> I'm going to use CoProcessing library. I'm moving step by step as
>>>> described in SC10_tutorial. I don't understand slide 99. I can't  create
>>>> Live Data Source. It isn't presented in my PV version (3.8.1). How can I add
>>>> it?
>>>>
>>>> Thanks,
>>>> Andrey.
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>> http://paraview.org/Wiki/ParaView
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://www.paraview.org/mailman/listinfo/paraview
>>>>
>>>
>>>   Hi Pat,
>>> Thank you for previous answer. I have managed to add Live Data Source in
>>> PV. I have a question: how can I assign host:port to make Coprocessor
>>> connect with pvserver?
>>> Andrey.
>>>
>>
>>   Hi Pat,
>> I have got a problem. While carring out "from paraview.simple import *"
>> from external python     interpreter  I am receiving "Segmentation fault".
>> In pv in python shell everything is doing well. What kind of problem I have?
>> Could you specify - with what and how I have to merge live-data branch?
>> Thank you.
>> Andrey.
>>
>
>  Hi Pat,
> I have returned to studing CoProcessing library. I have managed to solve my
> previous problem and launch C++ coprocessing example from Paraview.org. Now
> two questions have arised.
> 1) If I launch simulation code and pvserver on one node - everything is
> o'key, but if on different nodes - connect isn't established. Is it possible
> to do this way?
> 2) In my task I am going to use only UnstructuredGrid dataset without any
> filters. Can I do in such a way? Or for displaying it is possible to use
> only vtkPolyData objects?
> Andrey.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20110505/f433393e/attachment-0001.htm>