[Paraview] Recommendations for Benchmarking Paraview

[Wylie, Brian]

Jeremy,

We have done quite a bit of benchmarking here at Sandia. The
recommendation I would give is to base the benchmark on something
everyone can easily replicate.

For instance:

If you are benchmarking parallel rendering performance use something
like Sphere source.

You can easily make just about any number of total triangles you wish. 

Example: Sphere source Theta resolution=800 Phi resolution=800 Num
tris=1.27 Million

Also things like sphere source will give nice even 'pieces' to each
processor, so that you don't have to worry about things like load
balancing.

Run ProcessIDScalars and make sure that you have a reasonable load
balance. (If you don't have a reasonable load balance... Then run
'Balance' or 'AllToN' for Poly data or 'D3' for any other kind of data.)

Things that I think are important to record...
1) You should record times for both the 'interactive render' and 'still
render'
2) You should record whether 'Triangle strips' is ON or OFF
3) You should record whether 'Immediate Mode' is ON or OFF
4) You should record the version of ParaView
5) You should record what you used for 'Load balancing'
6) You should record the average memory usage on the processors (I've
had folks complain that ParaView was running very slow on their 4M cell
datasets... The problem was obviously too much mem usage...1% cpu...99%
disk access)

7) You should look carefully at the timer log for any thing folks might
find interesting... :)
8) You should record some of the info under the 'Help...About...'
dialog, which render module is being used, whether software rendering is
ON or OFF, etc, etc... (Note older versions of ParaView do not have
this.... :)

9) You should also record the normal things like...
I'm using 3.4Ghz P4's with 1Gig mem and using MPI over Myrinet... Or
whatever...

If you want to benchmark things like 'computing isosurfaces on
structured data', then I suggest things like Mandelbrot source and
wavelet source for the same reasons as above.


!!! Five cautionary notes !!!

If you are doing a more general benchmark... Like how fast can I
interact with a 13M Tet10 dataset. Be careful of the following...

1) Run ProcessIDScalars and make sure that you have a reasonable load
balance. 

2) If you're looking at unstructured data, run 'Extract Surface' and
look at the information tab to get the number of triangles you are
actually looking at. Or if you've computed an isosurface, same thing.
(We had a dataset that we couldn't interact with, 13M unstructured
dataset. We found out we had generated 80 Million triangles from the
data when computing 'External Faces' within the vtkDataSetMapper, not
having ghost cells computed in the load balancing is a killer... We ran
D3 and then we only had 2 Million triangles...)

3) Mem usage again... We've found that about 1M Unstructured cells per
processor, gives fairly good performance.

4) LOD is mostly your friend but every once and a while it can shot you
in the foot. If you have a very large dataset and the size of the LOD
happens to be just under your composite threshold you may spend a bunch
of time computing the LOD (server side) and collecting the LOD (client
side).

5) Related to 4.. Compositing Threshold default of 10MB might be too
high in my opinion. (I would vote for 5MB perhaps)

Example: Dataset with 400 time steps. Animate through time. Very common
operation obviously.

Lets say that the geometry generated at each time step is 9.99MB, so
that it is beneath the threshold and will be delivered to the client.
Lets do the math... (using 10MB just for easy math).


400 time steps x 10MB = 4GB

In this example the 4GB is delivered over a 100T network connection to
the client.

So 100T gives you about 10MB/sec at best. So 4GB / 10MB/sec = 400 sec or
6.66 minutes! Obviously an 'evil' amount of time that's basically
'wasted' on delivering geometry to the client, since the client will
only render it once and then throw away.


Good luck, if you want me to post Sandia's benchmarks just give a
holler.

Best,


	Brian Wylie
	Sandia National Laboratories
	MS 0822 - Org 9227 - Building 880/A1-J
      (505) 844-2238   FAX (505) 845-0833
 

> -----Original Message-----
> From: paraview-bounces at paraview.org 
> [mailto:paraview-bounces at paraview.org] On Behalf Of Jeremy Stout
> Sent: Tuesday, December 28, 2004 7:44 AM
> To: paraview at paraview.org
> Subject: [Paraview] Recommendations for Benchmarking Paraview
> 
> Hello, I am looking for recommendations for benchmarking 
> Paraview. I would like to run some tests to obtain some 
> performance measures for a few of my Paraview installations.  
> Has anyone done anything similar to this? Any input would be 
> appreciated.
> 
> Jeremy Stout
> _______________________________________________
> ParaView mailing list
> ParaView at paraview.org
> http://www.paraview.org/mailman/listinfo/paraview
> 
>