[Paraview] distributed stream tracer scalability issue

Tue Aug 25 15:13:40 EDT 2009

Burlen,

OK, if you are using rectilinear grids and image data and it is too 
slow, then findCell is probably not the cause of your trouble, and using 
locators won't help much. A modified version of the particle tracer may 
well buy you something.
(Lets assume that your data is well balanced when loaded from disk, such 
that pieces are approximately equal on each core and the more cores, the 
better the loading).
The PT first classifies all seed points according to the bounds of the 
pieces of the input data and distributes the seeds according to the 
process they start in, then all processes integrate until all are 
terminated, or have left the domain.
When each process has reached stagnation/termination, it goes through a 
send/receive from it's neighbour processes (there is a BSP tree of 
bounding boxes generated on initialization) so that each process 
sends/receives only to/from where particles travel.
Each process then iterates again through it's new particle list and so on.

In your case, if the number of processes increases in such a way that 
the number of particles per block is scaling nicely (at least at start 
of integration), then you should see an improvement, because you don't 
have to wait for one process to complete before starting. The total time 
will still be dominated by the longest streamline + communication and 
waits, but there is some scope for optimization.

I haven't committed the latest PT code yet, so maybe if you want to try 
a customized version for your data you should contact me off list - 
particularly if you can adapt the loading of the data / pieces to suit 
the geometry of the seed points you are starting with (eg plane slices 
through the data would benefit from pieces which are orthogonal to the 
slices so that seeds become more distributed as core count increases - 
etc etc)...

JB
> Actually I wasn't clear in my original post, I didn't really mean so 
> much that the DST was slow itself, but rather that it simply is taking 
> too long for my application, which requires many many streamlines to 
> be generated. My first tendency is to throw more processes at it in 
> hopes of a speed up. but in reality that really hurts the DST 
> performance.
>
>
> Adriano Gagliardi wrote:
>> When you say slow, can you quantify that? Also, what is the size of 
>> the mesh
>> in the regions where you are generating the streamtraces? I ask purely
>> because I've always found streamtraces to take a long time regardless 
>> of the
>> application used.
>>
>> ===================================
>>
>> Adriano Gagliardi MEng PhD
>> Project Scientist
>> Computational Aerodynamics
>> Aircraft Research Association Ltd.
>> Manton Lane
>> Bedford
>>
>> Tel: 01234 32 4644
>> E-mail: agagliardi at ara.co.uk
>> Url: www.ara.co.uk -----Original Message-----
>> From: paraview-bounces at paraview.org 
>> [mailto:paraview-bounces at paraview.org]
>> On Behalf Of burlen
>> Sent: 25 August 2009 00:59
>> To: paraview
>> Subject: [Paraview] distributed stream tracer scalability issue
>>
>> We've been using the distributed stream tracer to generate 100s-1000s of
>> stream lines per time step. It's very slow, and it doesn't scale at all.
>> The class comments say as much. I'm sure there is a reason why this
>> implementation was chosen. Is there something that generally prevents 
>> real
>> parallel implementation? Is there a better implementation available out
>> there?
>>
>> There is this post a while back
>> http://www.paraview.org/pipermail/paraview/2009-July/012959.html
>>
>> What's the status?
>>
>> Thanks
>> Burlen
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the ParaView Wiki at:
>> http://paraview.org/Wiki/ParaView
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.paraview.org/mailman/listinfo/paraview
>>
>>
>> ---------------------------
>> This email contains information that is private and confidential and 
>> is intended only for the addressee.  If you are not the intended 
>> recipient please delete it and notify us immediately by e-mailing the 
>> sender.
>> Note: All email sent to or from this address may be accessed by 
>> someone other than the recipient, for system management and security 
>> reasons.
>> Aircraft Research Association Ltd.  Registered in England, 
>> Registration No 503668 Registered Office: Manton Lane, Bedford MK41 
>> 7PF England VAT No GB 196351245
>>
>>
>>   
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at 
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at: 
> http://paraview.org/Wiki/ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://www.paraview.org/mailman/listinfo/paraview

-- 
John Biddiscombe,                            email:biddisco @ cscs.ch
http://www.cscs.ch/
CSCS, Swiss National Supercomputing Centre  | Tel:  +41 (91) 610.82.07
Via Cantonale, 6928 Manno, Switzerland      | Fax:  +41 (91) 610.82.82