[Paraview] peak memory use on remote nodes

[Biddiscombe, John A.]

Dave

So it turns out that aprun/alps won’t let me run a script with dstat to monitor the memory use, and using craypat makes the execution take too long (weeks it seems). So I tried importing part of the dstat python into my paraview python

Using this I am able to query memory use on the node and dump it out.

Question :  If I start a background python thread that polls memory like this

#####################
# start a thread that prints mem use
#####################
def dump_memory():
    # do something here ...
    global stop_thread
    line = ''
    for o in stats:
        o.extract()
        line = line + '  ' + o.show()
    print line + dstat.ansi['reset']

    # call f() again in n seconds
    if stop_thread==0:
      threading.Timer(0.5, dump_memory).start()


and set it going at the start of my paraview python pipeline : Will it actually run and collect info whilst the paraview script is executing the rest of the pipeline.

intuition and brief googling tells me that this won’t work.

Can it be done?

JB

From: ParaView [mailto:paraview-bounces at paraview.org] On Behalf Of Biddiscombe, John A.
Sent: 18 April 2016 14:53
To: David E DeMarle
Cc: paraview at paraview.org
Subject: Re: [Paraview] peak memory use on remote nodes

Thanks Dave

A colleague here suggested I put my
aprun pvbatch <blah> into a script and add a ‘dtrace” command at the start of the script. This will then dump out a txt file on each node and after the job completes I can grep the info I need.

I’m going to give that a try first as it seems straightforward, but I will revert to your suggestion if I fail and nothing else comes up.

cheers

JB

From: David E DeMarle [mailto:dave.demarle at kitware.com]
Sent: 18 April 2016 14:48
To: Biddiscombe, John A.
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] peak memory use on remote nodes

Perhaps extend the benchmark to glean /proc/<PID>/status's VmHWM entry on Linux at least. Get the number for t1 and at t2 and subtract to see what the maximum was in between those two times.

Seems the right place to do that is kwsys/SystemInformation.cxx, but you could prototype in the script.
Hope that helps and please post some code. Seems like a very useful thing to have.


David E DeMarle
Kitware, Inc.
R&D Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909

On Mon, Apr 18, 2016 at 2:42 AM, Biddiscombe, John A. <biddisco at cscs.ch<mailto:biddisco at cscs.ch>> wrote:
The python scripting module “paraview.benchmark” allows one to get the memory use using “paraview.benchmark.get_memuse” - but I presume this is the memory reported as used by the system at the moment when the function is called.

Does anyone know a way of recording the peak memory use on remote nodes between t1 and t2 - where t1 and t2 are the start and stop of either a function, job or even paraview python script on the remote node?

thanks

JB

--
John Biddiscombe,                        email:biddisco @.at.@ cscs.ch<http://cscs.ch>
http://www.cscs.ch/
CSCS, Swiss National Supercomputing Centre  | Tel:  +41 (91) 610.82.07<tel:%2B41%20%2891%29%20610.82.07>
Via Trevano 131, 6900 Lugano, Switzerland   | Fax:  +41 (91) 610.82.82<tel:%2B41%20%2891%29%20610.82.82>


_______________________________________________
Powered by www.kitware.com<http://www.kitware.com>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView

Search the list archives at: http://markmail.org/search/?q=ParaView

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20160421/6922d1cb/attachment.html>