<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 12px; font-family: Courier, sans-serif;">
<div>
<div>
<div>Hey Alan: We originally encountered the Generic Warning in v4.3.1; Andy suggested trying 5.2 (see below). Subsequently (see below) we have tried v5.2, v5.3, and v5.4 - all of which behave much the same with large point numbers. I believe that Andy’s
assessment is correct: this is an MPI issue. However, why ParaView should find it necessary to pass all of the point data, all at once, is kinda baffling. So maybe you have pointed to something to study: has the 4.* bug really been fixed? Especially in
the case of point data (rather than grid data, which, in our case, is always considerably smaller). Thanks for writing, and thinking about the issue. Cheers, b.</div>
<div><br>
</div>
<div><span class="Apple-style-span" style="font-family: Courier;">=======================================================================73</span><span class="Apple-style-span" style="font-family: Courier;"><br>
</span><span class="Apple-style-span" style="font-family: Courier;"><>Bucky Kashiwa PhD, PE <> Post: MS B216, Los Alamos, NM 87545 <></span><span class="Apple-style-span" style="font-family: Courier;"><br>
</span><span class="Apple-style-span" style="font-family: Courier;"><> Ofc: TA3-SM123-RM276 <>Email: <a href="mailto:bak@lanl.gov">bak@lanl.gov</a>, <a href="mailto:kashiwa@qwest.net">kashiwa@qwest.net</a> <></span><span class="Apple-style-span" style="font-family: Courier;"><br>
</span><span class="Apple-style-span" style="font-family: Courier;"><>Voice: 505-667-8812 <> Fax: 505-665-5926 <> </span><span class="Apple-style-span" style="font-family: Courier;"><br>
</span><span class="Apple-style-span" style="font-family: Courier;"> <> Home: 505-988-7332 <> Cell: 505-795-5581 <></span><span class="Apple-style-span" style="font-family: Courier;"><br>
</span><span class="Apple-style-span" style="font-family: Courier;">=======================================================================73</span></div>
</div>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>"Scott, W Alan" <<a href="mailto:wascott@sandia.gov">wascott@sandia.gov</a>><br>
<span style="font-weight:bold">Date: </span>Thursday, June 29, 2017 at 11:31 AM<br>
<span style="font-weight:bold">To: </span>Bucky Kashiwa <<a href="mailto:bak@lanl.gov">bak@lanl.gov</a>>, "Bauer, Andy (External Contacts)" <<a href="mailto:andy.bauer@kitware.com">andy.bauer@kitware.com</a>>, andrealphus <<a href="mailto:andrealphus@gmail.com">andrealphus@gmail.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:ParaView@ParaView.org">ParaView@ParaView.org</a>" <<a href="mailto:ParaView@paraview.org">ParaView@paraview.org</a>><br>
<span style="font-weight:bold">Subject: </span>RE: [EXTERNAL] Re: [Paraview] ... not yet supported for more than 2147483647 bytes.<br>
</div>
<div><br>
</div>
<div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Courier;
panose-1:2 7 4 9 2 2 5 2 4 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.apple-style-span
{mso-style-name:apple-style-span;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">I notice you are running ParaView 4.3.1? We had a bug in the earlier 4.*.* versions where we were passing massive amounts of information from all processes to all processes
on the state of the status bar, bottom of the screen. We stopped doing that. I wonder if this could be an issue with huge numbers of points?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Try PV 5.4.0? (Or possibly wait for 5.4.1, out Real Soon Now?) <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Alan<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> ParaView [<a href="mailto:paraview-bounces@paraview.org">mailto:paraview-bounces@paraview.org</a>]
<b>On Behalf Of </b>Kashiwa, Bucky<br>
<b>Sent:</b> Thursday, June 29, 2017 10:53 AM<br>
<b>To:</b> Bauer, Andy (External Contacts) <<a href="mailto:andy.bauer@kitware.com">andy.bauer@kitware.com</a>>; andrealphus <<a href="mailto:andrealphus@gmail.com">andrealphus@gmail.com</a>><br>
<b>Cc:</b> <a href="mailto:ParaView@ParaView.org">ParaView@ParaView.org</a><br>
<b>Subject:</b> [EXTERNAL] Re: [Paraview] ... not yet supported for more than 2147483647 bytes.<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black">Andy, Ashton: We have now tried versions 5.2, 5.3, and 5.4. With large point data sets we still have the same Generic Warning cited below (followed by ERRORS that cause PV
to hang). I reckon that we are hitting the MPI wall associated with 2^31 items communicated at once - as per Andy’s note below.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black">All of our recent work has been using the downloadable binaries from paraview.org, for both the client and the server. We see the same behavior using the MacOS client to Linux
server, and Linux client to Linux server.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black">This MPI shortcoming is a serious showstopper for us so we are going to have to find a remedy.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black">The first question is: why should either the client or the server think that more than 2^31 items need to be communicated in the first place? On the surface, this seems to be
unreasonable.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black">To be clear, there is no problem opening and reading the datafiles, and displaying the outline view. The warning appears when we try to display the point view, when the number
of points is too large. If we take a slice that reduces the number of points to below some threshold, then the point view display is okay. As an example, consider a .cosmo64 formatted point data file. There are 36 Bytes of data per point. We can display
~28,000,000 points and ~35,500,000 points just fine. At 45,500,000 points (=1.638 GBytes) the Generic Warning gets thrown. This seems to be independent of the number of nodes and PEs used by the server, and whether the view RenderView or EyeDomeLighting.
We also write/read .vtm format that behaves in a similar fashion: slices will display okay until the slice is thick enough that there are too many points in the image.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black">I have tried doing MPI on the client, which seems to have no effect on the foregoing limitation on the number of displayable points. Please let me know if you can think of other
switches that can be thrown, that may shed some more light on the issue, or if you are aware of a forthcoming repair. Thanks very much, b.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span class="apple-style-span"><span style="font-size:9.0pt;font-family:Courier;color:black">=======================================================================73</span></span><span style="font-size:9.0pt;font-family:Courier;color:black"><br>
<span class="apple-style-span"><>Bucky Kashiwa PhD, PE <> Post: MS B216, Los Alamos, NM 87545 <></span><br>
<span class="apple-style-span"><> Ofc: TA3-SM123-RM276 <>Email: <a href="mailto:bak@lanl.gov">bak@lanl.gov</a>, <a href="mailto:kashiwa@qwest.net">kashiwa@qwest.net</a> <></span><br>
<span class="apple-style-span"><>Voice: 505-667-8812 <> Fax: 505-665-5926 <> </span><br>
<span class="apple-style-span"> <> Home: 505-988-7332 <> Cell: 505-795-5581 <></span><br>
<span class="apple-style-span">=======================================================================73</span><o:p></o:p></span></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><o:p> </o:p></span></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">From:
</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">Andy Bauer <<a href="mailto:andy.bauer@kitware.com">andy.bauer@kitware.com</a>><br>
<b>Date: </b>Monday, December 19, 2016 at 4:17 PM<br>
<b>To: </b>andrealphus <<a href="mailto:andrealphus@gmail.com">andrealphus@gmail.com</a>><br>
<b>Cc: </b>Bucky Kashiwa <<a href="mailto:bak@lanl.gov">bak@lanl.gov</a>>, "<a href="mailto:ParaView@ParaView.org">ParaView@ParaView.org</a>" <<a href="mailto:ParaView@paraview.org">ParaView@paraview.org</a>><br>
<b>Subject: </b>Re: [Paraview] ... not yet supported for more than 2147483647 bytes.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><o:p> </o:p></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black">There are two parts to this issue. The first is that that vtkMPICommunicator for PV 4.3.1 won't communicate data that is over 2^31 bytes of data. This is fixed in PV 5.2. The
other issue is due to MPI having a limit of 2^31 objects to be communicated in a single shot. This is MPI's API in that the count for objects that are typically sent/received is an int. See
<a href="http://www.mpich.org/static/docs/v3.1/www3/MPI_Send.html">http://www.mpich.org/static/docs/v3.1/www3/MPI_Send.html</a> for example.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black">On Mon, Dec 19, 2016 at 4:59 PM, andrealphus <<a href="mailto:andrealphus@gmail.com" target="_blank">andrealphus@gmail.com</a>> wrote:<o:p></o:p></span></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black">That is a 32 bit error, from trying to index something with more than<br>
(2^32)/2 elements or indices. Are you using any custom<br>
libraries/packages/modules which might not be 64 bit compliant? Are<br>
you sure you built a 64 bit version (check your gcc -v).<br>
<br>
-ashton<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><br>
On Mon, Dec 19, 2016 at 1:32 PM, Kashiwa, Bucky <<a href="mailto:bak@lanl.gov">bak@lanl.gov</a>> wrote:<br>
> On Linux using ParaView version 4.3.1 built with OSMesa-9.0.1, OpenMPI,<br>
> etc. Running pvserver with 12 PEs, client-server mode, Standard Release<br>
> ParaView-4.3.1-Linux-64bit client. With large point data (>2Gb) we get<br>
> this error, upon trying to display a large number of points:<br>
><br>
> Generic Warning: In<br>
> ../ParaView-v4.3.1-source/VTK/Parallel/MPI/vtkMPICommunicator.cxx,<br>
> line 194<br>
><br>
> This operation not yet supported for more than 2147483647 bytes.<br>
><br>
> Our CMakeCache.txt is attached, in case it may provide helpful clues.<br>
><br>
> Thanks much. B. Kashiwa<br>
><br>
><br>
><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black">> _______________________________________________<br>
> Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
><br>
> Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">
http://www.kitware.com/opensource/opensource.html</a><br>
><br>
> Please keep messages on-topic and check the ParaView Wiki at: <a href="http://paraview.org/Wiki/ParaView" target="_blank">
http://paraview.org/Wiki/ParaView</a><br>
><br>
> Search the list archives at: <a href="http://markmail.org/search/?q=ParaView" target="_blank">
http://markmail.org/search/?q=ParaView</a><br>
><br>
> Follow this link to subscribe/unsubscribe:<br>
> <a href="http://public.kitware.com/mailman/listinfo/paraview" target="_blank">http://public.kitware.com/mailman/listinfo/paraview</a><br>
><br>
_______________________________________________<br>
Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">
http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the ParaView Wiki at: <a href="http://paraview.org/Wiki/ParaView" target="_blank">
http://paraview.org/Wiki/ParaView</a><br>
<br>
Search the list archives at: <a href="http://markmail.org/search/?q=ParaView" target="_blank">
http://markmail.org/search/?q=ParaView</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://public.kitware.com/mailman/listinfo/paraview" target="_blank">http://public.kitware.com/mailman/listinfo/paraview</a><o:p></o:p></span></p>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Courier;color:black"><o:p> </o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</span>
</body>
</html>