<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style id="owaParaStyle" type="text/css">P {margin-top:0;margin-bottom:0;}</style>
</head>
<body ocsi="0" fpstyle="1">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">Hi David,<br>
<br>
Here's the output from the code changes (parsed output of one scalar field), focusing on XdmfValuesBinary.cxx.<br>
<br>
<font face="Courier New">XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 231 (Accessing Binary CDATA)<br>
Seek as string is 3221225664<br>
XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 267 (Data Size : 134217728)<br>
XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 268 (Size[Byte]: 1073741824)<br>
XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 269 ( Byte 8)<br>
XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 287 (Opening Binary Data for Reading : F:/rstrt//r<br>
SEEK as size_t is 2147483647<br>
XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 322 (Seek: 2147483647)<br>
XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 331 (Hyperslab data)<br>
XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 161 (Reduce Rank: 3 to 1)<br>
XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 178 (Contiguous byte: 1073741824)</font><br>
<br>
<br>
As a string, it looks like the seek location is being read right. Somewhere when it's converted to size_t, it defaults to
<font face="Courier New">2147483647.<br>
<font face="Helvetica"><br>
Jesus<br>
<br>
</font></font>
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<hr tabindex="-1">
<div style="direction: ltr;" id="divRpF216581"><font face="Tahoma" color="#000000" size="2"><b>From:</b> David E DeMarle [dave.demarle@kitware.com]<br>
<b>Sent:</b> Friday, August 14, 2015 6:33 AM<br>
<b>To:</b> Pulido, Jesus<br>
<b>Cc:</b> paraview@paraview.org<br>
<b>Subject:</b> Re: [Paraview] Memory seeking problems in using the xdmf2 reader<br>
</font><br>
</div>
<div></div>
<div>
<div dir="ltr">Mind trying this?
<div><br>
<div>
<div>diff --git a/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx b/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx</div>
<div>index 04ea0a7..d4cb2ed 100644</div>
<div>--- a/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx</div>
<div>+++ b/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx</div>
<div>@@ -187,7 +187,7 @@ public:</div>
<div> </div>
<div> size_t XdmfValuesBinary::getSeek(){</div>
<div> if(this->Seek==NULL)return 0;</div>
<div>- return static_cast<size_t>(atoi(this->Seek));</div>
<div>+ return static_cast<size_t>(atol(this->Seek));</div>
<div> }</div>
<div> </div>
<div> </div>
<div>@@ -244,6 +244,7 @@ XdmfValuesBinary::Read(XdmfArray *anArray){</div>
<div> }else{</div>
<div> this->Seek = NULL;</div>
<div> }</div>
<div>+ cerr << "Seek as string is " << Value << endl;</div>
<div> }</div>
<div> {</div>
<div> XdmfConstString Value = this->Get("Compression");</div>
<div>@@ -291,6 +292,7 @@ XdmfValuesBinary::Read(XdmfArray *anArray){</div>
<div> //strcpy(path+strlen(this->DOM->GetWorkingDirectory()), DataSetName);</div>
<div> try{</div>
<div> size_t seek = this->getSeek();</div>
<div>+ cerr << "SEEK as size_t is " << seek << endl;</div>
<div> switch(getCompressionType()){</div>
<div> case Zlib:</div>
<div> XdmfDebug("Compression: Zlib");</div>
</div>
<div><br>
</div>
</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div class="gmail_signature">David E DeMarle<br>
Kitware, Inc.<br>
R&D Engineer<br>
21 Corporate Drive<br>
Clifton Park, NY 12065-8662<br>
Phone: 518-881-4909</div>
</div>
<br>
<div class="gmail_quote">On Thu, Jul 30, 2015 at 5:55 PM, Pulido, Jesus <span dir="ltr">
<<a href="mailto:pulido@lanl.gov" target="_blank">pulido@lanl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div>
<div style="direction:ltr; font-family:Tahoma; color:#000000; font-size:10pt">Hi David,<br>
<br>
The patch did not work, the file is still not read correctly after the second scalar component.
<br>
<br>
After applying your patch, I did a bit of debugging myself. I enabled XDMF Debug in the code and found that the seek values are incorrect for scalar fields 3, 4, and 6.
<br>
They seem to be stuck at the value <a href="tel:2147483647" value="+12147483647" target="_blank">
2147483647</a>. On the plus side, it looks like the array size (array size in bytes) is being computed correctly.
<br>
<br>
Please take a look at the parsed debug log file that I've attached. Maybe the seek value is being malformed somewhere before it reaches XdmfValuesBinary.cxx?<br>
<br>
Thanks,<br>
Jesus<br>
<br>
<br>
<div style="font-family:Times New Roman; color:#000000; font-size:16px">
<hr>
<div style="direction:ltr"><font face="Tahoma" color="#000000" size="2"><b>From:</b> David E DeMarle [<a href="mailto:dave.demarle@kitware.com" target="_blank">dave.demarle@kitware.com</a>]<br>
<b>Sent:</b> Wednesday, July 29, 2015 10:36 PM<br>
<b>To:</b> Pulido, Jesus<br>
<b>Cc:</b> <a href="mailto:paraview@paraview.org" target="_blank">paraview@paraview.org</a><br>
<b>Subject:</b> Re: [Paraview] Memory seeking problems in using the xdmf2 reader<br>
</font><br>
</div>
<div>
<div class="h5">
<div></div>
<div>
<div dir="ltr">Would you mind trying this Jesus?
<div><br>
</div>
<div>
<div>diff --git a/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx b/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx</div>
<div>index 04ea0a7..6fe37d0 100644</div>
<div>--- a/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx</div>
<div>+++ b/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx</div>
<div>@@ -187,7 +187,7 @@ public:</div>
<div> </div>
<div> size_t XdmfValuesBinary::getSeek(){</div>
<div> if(this->Seek==NULL)return 0;</div>
<div>- return static_cast<size_t>(atoi(this->Seek));</div>
<div>+ return static_cast<size_t>(atol(this->Seek));</div>
<div> }</div>
</div>
<div><br>
</div>
<div>Once you confirm it fixes it for you too I'll check it in.</div>
<div><br>
</div>
<div>Thanks,</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div>David E DeMarle<br>
Kitware, Inc.<br>
R&D Engineer<br>
21 Corporate Drive<br>
Clifton Park, NY 12065-8662<br>
Phone: <a href="tel:518-881-4909" value="+15188814909" target="_blank">518-881-4909</a></div>
</div>
<br>
<div class="gmail_quote">On Fri, Jul 24, 2015 at 12:43 PM, Pulido, Jesus <span dir="ltr">
<<a href="mailto:pulido@lanl.gov" target="_blank">pulido@lanl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div>
<div style="direction:ltr; font-family:Tahoma; color:#000000; font-size:10pt">Hello,<br>
<br>
I have written a custom xmf file used by the xdmf2 reader to properly read in (raw) binary data into Paraview. This normally works but only for small datasets. The file requires the input of a memory "seek" location to start reading the different scalar components
of the data. A problem arises when there is a 512^3 dataset, with 5 scalar components that are required to be read in. The first two scalar components are read in correctly, then the next two components receive the error:<br>
<br>
<p style="margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; text-indent:0px">
<span style="font-family:'Courier'; font-size:13pt; color:#800000">ERROR: In /Users/kitware/Dashboards/MyTests/NightlyMaster/ParaViewSuperbuild-Release-Python27/paraview/src/paraview/VTK/IO/Xdmf2/vtkXdmfHeavyData.cxx, line 1128</span></p>
<p style="margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; text-indent:0px">
<span style="font-family:'Courier'; font-size:13pt; color:#800000">vtkXdmfReader (0x7f8614672af0): Failed to read attribute data</span></p>
<br>
The 5th component then becomes a duplicate of the 1st component.<br>
What I believe that's happening is that the "seek" value is only being represented as a signed integer, this would explain the memory location rolling from MAX_INT to -MAX_INT, and failing to read in scalar components 3 and 4. Their seek locations fall within
this case. By the time the 5th component is read, the seek value may have rolled over back to positive values (seek > 0) and then duplicates the same data as the 1st component.<br>
<br>
Is this assumption correct and intended? Is there fix for this or an alternative way around reading large, raw datasets using this format? I'm attaching a sample xmf file<br>
<br>
Thank you,<br>
Jesus<br>
</div>
</div>
<br>
_______________________________________________<br>
Powered by <a href="http://www.kitware.com" rel="noreferrer" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" rel="noreferrer" target="_blank">
http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the ParaView Wiki at: <a href="http://paraview.org/Wiki/ParaView" rel="noreferrer" target="_blank">
http://paraview.org/Wiki/ParaView</a><br>
<br>
Search the list archives at: <a href="http://markmail.org/search/?q=ParaView" rel="noreferrer" target="_blank">
http://markmail.org/search/?q=ParaView</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://public.kitware.com/mailman/listinfo/paraview" rel="noreferrer" target="_blank">http://public.kitware.com/mailman/listinfo/paraview</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</body>
</html>