<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">G’Day all,<o:p></o:p></p>
<p class="MsoNormal">I am looking at the performance of a very large volume rendered fire. My user has been using lots of nodes, and we wanted to understand if these nodes are needed. Here is my result:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Dataset: 1296 files, 1.3 GByte each, 18 TBytes total, 67 million cells, 468 timesteps. These viz runs are taking around 12 hours with a reasonable number of nodes. When hitting play, TimerLog says each frame uses the following time:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">2 nodes – died<o:p></o:p></p>
<p class="MsoNormal">4 nodes/ 64 cores and ranks - 79 seconds/frame, node memory 65% full<o:p></o:p></p>
<p class="MsoNormal">8 nodes/ 128 cores and ranks - 64 seconds/frame, node memory 41% full<o:p></o:p></p>
<p class="MsoNormal">16 nodes/ 128 cores and ranks - 59 seconds/frame, node memory 23% full<o:p></o:p></p>
<p class="MsoNormal">24 nodes/ 192 cores and ranks – 53 seconds/frame, node memory unknown.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">For 16 nodes, timer log says we spent:<o:p></o:p></p>
<p class="MsoNormal">37 seconds – Regenerate Kd-Tree<o:p></o:p></p>
<p class="MsoNormal">15 seconds – Redistributing data<o:p></o:p></p>
<p class="MsoNormal">7 seconds – OpenGL Rendering<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">For 8 nodes, timer log says we spent:<o:p></o:p></p>
<p class="MsoNormal">39 seconds – Regenerate Kd-Tree<o:p></o:p></p>
<p class="MsoNormal">17 seconds – Redistributing data<o:p></o:p></p>
<p class="MsoNormal">13 seconds – OpenGL Rendering<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So, we are getting scaling with increased memory, but not increased cpu. More specifically, the Regenerate Kd-Tree is not speeding up as you halve each nodes work (which is a surprise), and ditto redistributing data. OpenGL Rendering
is linearly changing speeds. Last, he majority of our time is regenerating the Kd-Tree. What is this tree, why do we need it, what does it do? I have had this dataset in a debugger, and frankly, it was really hard to figure out where time was being spent.
Any ideas how to proceed? Is there something we could do to increase information from the timer log?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks team!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Alan<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">--------------------------------------------------------<o:p></o:p></p>
<p class="MsoNormal">W. Alan Scott<o:p></o:p></p>
<p class="MsoNormal">ParaView Support Manager<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">SAIC<o:p></o:p></p>
<p class="MsoNormal">Sandia National Laboratories, MS 0822<o:p></o:p></p>
<p class="MsoNormal">Org 9326 - Building 880 A1-K<o:p></o:p></p>
<p class="MsoNormal">(505) 284-0932 FAX (505) 284-5619<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The most exciting phrase to hear in science <o:p></o:p></p>
<p class="MsoNormal">is not "Eureka!" but "That's funny..." -- Isaac Asimov<o:p></o:p></p>
<p class="MsoNormal">---------------------------------------------------------<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>