[Paraview] Parallel Load/Rendering of Polygon Data

[Wylie, Brian]

Phillip,

I don't know the exact right file format for loading polygonal data in
parallel, but I think Sam is on the right track.

If you want to load balance the data once it is in you can use the D3
filter. :)

Also for the short term if you can 'squeeze' the dataset on one node you
can use D3 to take data from 1 processor and send data to all the
others. Then save the dataset out as a pvtp file. You will now have
parallel files !

You can then load these up in paraview and they will be nicely load
balanced.

You could even do something like

Load up on a 2 proc machine with 4 Gig memory save out 2 files then
start up a cluster job on those two files and split again and save 16
files or whatever.... How's that for goofy!

Brian Wylie - Org 9227
Sandia National Laboratories
MS 0822 - Building 880/A1-J
(505)844-2238 FAX(505)845-0833                       
     ____                  _    __
    / __ \____  _________ | |  / (_)__ _      __ 
   / /_/ / __ `/ ___/ __ `/ | / / / _ \ | /| / /
  / ____/ /_/ / /  / /_/ /| |/ / /  __/ |/ |/ /
 /_/    \__,_/_/   \__,_/ |___/_/\___/|__/|__/

                                    Unleash the Beast
 

> -----Original Message-----
> From: paraview-bounces+bnwylie=sandia.gov at paraview.org 
> [mailto:paraview-bounces+bnwylie=sandia.gov at paraview.org] On 
> Behalf Of SamuelKey
> Sent: Tuesday, May 31, 2005 6:02 PM
> To: Phillip Ezolt; Paraview Mailing List
> Subject: Re: [Paraview] Parallel Load/Rendering of Polygon Data
> 
> Phillip,
> 
> I have had some success with *.pvtu file sets.
> The aaa.pvtu (or in your case it would be aaa.pvtp) is a 
> "mother" file that references all of the polygon subsets into 
> which the parent aaa.vrml has been "decomposed," that is, 
> aaa.001.vtp,...,aaa.00n.vtp is a sequence of smaller, 
> component files that represent the entire polygon set. Each 
> of the subdomain *.vtp files must be displayable in its own 
> right by ParaView.
> 
> I have attached a typical (for me) aaa.vtpu file as an example.
> In this example, the "parallel subdomains" are three separate 
> material regions (m001,m002,m003). ParaView doesn't care 
> whether you created them from a serial run or a parallel run; 
> all PV needs to know is that you want to display all of *.vtu 
> files and you want to think of the set as the entire model. 
> (If your vrml file has a bunch of individual items, this is 
> also a handy way to display selectively each item, provided 
> you can conviently split your vrml data set.)
> 
> I hope this helps to parallel process your 2.5 gig data set.
> 
> Regards,
> 
> Sam Key
> 
> 
> ----- Original Message -----
> From: "Phillip Ezolt" <Phillip.Ezolt at hp.com>
> To: "Paraview Mailing List" <paraview at paraview.org>
> Sent: Tuesday, May 31, 2005 12:51 PM
> Subject: [Paraview] Parallel Load/Rendering of Polygon Data
> 
> 
> > Hi All,
> > 
> > I have a large polygon model (2.5 gig) that I want to view within
> > paraview.  Individually, each of the nodes do not have 
> enough memory (2
> > gig) to load the model, however, in parallel, they should 
> have enough.
> > I've tried several different ways to load and render my model in
> > parallel without success....
> > 
> > Questions:
> > 1) Does paraview provide way to load and render this polygon
> >    data in parallel, with each node loading and rendering a
> >    subset of the data? 
> > 
> > If I:
> > A) load another relatively large model, 
> > B) split it across many nodes, 
> > C) use "map process id to color"
> > 
> > The image appears on the screen with a series of horizontal
> > bands. This is as if each of the nodes had all of the polygons,
> > but only drew a band of it. (Sort-last)
> > 
> > I would prefer that each node loaded and drew a different subset
> >         of the geometry and then depth-composited it together.
> > 
> > (At first, round robin selection of polygons would be just
> >          fine..)
> > 
> > Does paraview support this?  If not, where would I look to add
> > the round-round loading/rendering of polygon data?
> > 
> > 2) What file format should I save the polygon data in? 
> > 
> > I've used a single 4-gig x86_64 node to convert it from VRML 2.0
> > to a 'pvd' file.  However, the 'pvd' file only has one
> >         piece, "lucy_0.vtp".  When I try to load it on many 
> (8) 2-gig
> > nodes in parallel, they each run out of memory.  
> > 
> > When I load the model on the single 4-gig node, the RSS of 
> > paraview is ~2.5 gig.  As a result, I would expect an 8 way run
> >         of paraview to each be 300M... That isn't the case.  (Yes, I
> >         know that things don't scale like that, but each node should
> >         not require more than 1 gig...)
> > 
> > What is the right file format to use for parallel loading?  How
> > should I generate this file from VRML data? 
> > 
> > NOTE: I'd had success loading/parallizing raw data across 
> many nodes,
> > but polygon data doesn't seem to work.  However, I may not 
> have reached
> > the single-node memory threshold....
> > 
> > 
> > Thanks in advance,
> > -- 
> > Phillip Ezolt <Phillip.Ezolt at hp.com>
> > 
> > _______________________________________________
> > ParaView mailing list
> > ParaView at paraview.org
> > http://www.paraview.org/mailman/listinfo/paraview
> > 
>