[Paraview] Help with optimizing Paraview Python script

[Neal,Christopher R]

Hi Cory,


I'm using a Windows 64bit version that was built from the binary installer option on the Paraview downloads page.


The grids can be upwards of 20 million cells.  One timestep worth of data is about 100MB.


I'm running on a system with 16GB of RAM.


In terms of optimization I'm inclined to believe that my script isn't written very well and that it is the source of the slowdown that I'm seeing when running it. I CC'd you on my reply to Andy's email where we identified some areas of the script that were particularly slow during execution.


Best regards,


Christopher R. Neal
Graduate Student
Center for Compressible Multiphase Turbulence
Mechanical and Aerospace Engineering Department
University of Florida
Cell: (863)-697-1958
E-mail:  chrisneal at ufl.edu


________________________________
From: Cory Quammen <cory.quammen at kitware.com>
Sent: Wednesday, October 21, 2015 11:18 PM
To: Andy Bauer
Cc: Neal,Christopher R; paraview at paraview.org
Subject: Re: [Paraview] Help with optimizing Paraview Python script

A few questions:

* How did you obtain the ParaView executable? (download, build)? If you built ParaView in Debug mode, that could slow things down.

* How big are your data files, both in terms of disk size and grid elements?

* How much RAM is on your system?

- Cory

On Wed, Oct 21, 2015 at 8:50 PM, Andy Bauer <andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>> wrote:
Did you profile this at all to see where most of the time being spent?

Looking at this, I don't see anything really bad. I think you can take out the time.sleep(1.0) part though. Also, replacing the Calculator filter with the PythonCalculator filter should help some. If you're running this in parallel, the fetch is probably hurting performance as well.

On Wed, Oct 21, 2015 at 4:02 PM, Neal,Christopher R <chrisneal at ufl.edu<mailto:chrisneal at ufl.edu>> wrote:

Hello,


I have written a python script that loads in an Ensight data file which has multiple solution times associated with it and performs a set of fairly simple operations on it. It is running very slow, and I'm wondering if there are any experts who use Python to control Paraview that could help me to optimize the script.  In a nutshell the script does the following.


1.) Go to a timestep

2.) Generate a series of 2D slices in the 3D domain

3.) Compute some values of the solution variables on those slices

4.) Write data to a file

5.) Delete the entire pipeline except the original data

6.) Go back to 1.)


This process is painfully slow(2+days just to run the script on a modest size data set).  I believe that the script having to delete the pipeline and then re-create it for every timestep is contributing to the slowdown. My  trouble is that I do not know how to selectively activate certain elements in the pipeline if I want to extract information from them, so I have to add them sequentially so that the newest one is the active one.  The script may also be rendering each step, which is unnecessary for me, but I do not know how to disable that as well.


I have attached a copy of the script. I have a minimal working example that contains the data set with only 3 timesteps worth of data.


I am even willing to pay  $100 for an expert to help me out with this.


Thank you,


Christopher R. Neal
Graduate Student
Center for Compressible Multiphase Turbulence
Mechanical and Aerospace Engineering Department
University of Florida
Cell: (863)-697-1958<tel:%28863%29-697-1958>
E-mail:  chrisneal at ufl.edu<mailto:chrisneal at ufl.edu>

_______________________________________________
Powered by www.kitware.com<http://www.kitware.com>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView

Search the list archives at: http://markmail.org/search/?q=ParaView

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview



_______________________________________________
Powered by www.kitware.com<http://www.kitware.com>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView

Search the list archives at: http://markmail.org/search/?q=ParaView

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview




--
Cory Quammen
R&D Engineer
Kitware, Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20151022/6619a08c/attachment.html>