[Paraview] ParaView 5.0.0 for Windows x64 fails to open EnSight Gold files

Tue Mar 1 11:30:40 EST 2016

Samuel, thanks for your experiment. File with size less than 2 GiB, of
course, can't expose this issue.
I read a bit more about stat() and it turned out that it should work as
64bit if FILE_OFFSET_BITS is set to 64. And it is set in other ParaView
sources but, IMHO, not in EnSight reader. Dear ParaView team, please, take
a look at this, I need to process my big case with Python script in
ParaView...
On Mar 1, 2016 18:52, "Samuel Key" <samuelkey at bresnan.net> wrote:

> Andrew,
>
> Regarding your question, "Do you have any EnSight Gold-Format files
> greater than 4GB?" No, the largest EnSight *.geom file I have is 1 GB.
>
> I just tried it in PV v5.0.0.  Reading the datum set and running through
> 67 time steps did not cause any problems. I don't think this *.geom file is
> big enough to expose any of the PV reader reader problems you are
> experiencing.
>
> Sam Key
>
> On 2/29/2016 10:53 PM, Andrew wrote:
>
> Hello Samuel.
>
> Sorry, but I can't undersatand how can the file fragmentation affect it's
> readability and parsing. Even if the file has thousands of fragments and
> "brain-damaging" :) runlist in many extended FILE records on NTFS
> partition, your program, if it uses Win API, will see it like a solid file
> (althougt there will be very low performance of sequental access). So there
> should be no any faults on reading highly fragmented files.
> I have checked out the "direct" reading of EnSight case from the cluster
> via samba with ParaView 5.0.0 on Windows machine. The same error appears as
> in my first message, although there are *different files and filesystems*.
> I think the problem is not a fragmentation but a *file size*. Some
> 32-bit-like relicts or so in EnSight reader (it's just a guess, sorry if
> I'm wrong). But my hypothesis does not asks the question why does the bug
> affect only Windows versions... Maybe there are differences between
> versions concerning data types. Do you have any >4 GB binary EnSight 6 or
> Gold case to test? I can't (not allowed) to publish my case and it's also
> big enough for upload (although theoretically I can do it from my home PC).
>
>
> 2016-02-29 20:27 GMT+03:00 Samuel Key <samuelkey at bresnan.net>:
>
>> Andrew,
>>
>> Let us focus on MS Windows for a moment. The NTS file system and the use
>> of the file system by the O/S is not quite as it should be.
>>
>> In my experience, the above combination routinely creates an unbelievable
>> number of file fragments. Think in terms of confetti. As time goes by, the
>> fragmentation builds. Even though you have a respectable amount of RAM, the
>> free space itself on the disk driver can get rather disorganized.
>>
>> You can start by using the MS-supplied defrager. Run this utility five to
>> seven times in a row and things might start looking better. However the
>> free-space can still remain disjointed. Personally, I prefer a 3rd party
>> utility for organizing and defraging the NTS file system (for example,
>> PerfectDisk from Raxco). Even this utility has to run a couple of times
>> plus once during the O/S re-boot phase to get a decent amount of
>> consolidated free space and a somewhat consolidate O/S paging file. All of
>> this has to be done on the O/S partition (usually C:) and the partition
>> where you read in the 5+ GB Ensight Geometry file.
>>
>> On my machine, the MS Win7 O/S has its own partition (100GB \C:) plus
>> another smaller dedicated partition for a second paging file (28GB \F:).
>> All of my simulation work including the EnSight results file-sets reside on
>> an entirely separate disk drive (280GB \G: that now needs to be even
>> larger). The 3rd party file consolidation software runs "day and night" on
>> the \G: partition.
>>
>> Just to emphasize the point, copying the 5+ GB EnSight geometry file on
>> to workstation can result in many scattered fragments even when the
>> free-space starts out consolidated. If the free space is not consolidated,
>> the file fragmentation will be even more.
>>
>> The attached screen capture, shows the O/S managed paging-file (brown) in
>> the \C: partition is beginning to fragment.
>>
>> If you go down this path and it helps, please let me know the outcome.
>>
>> Regards,
>>
>> Sam Key
>>
>> On 2/29/2016 1:04 AM, Andrew wrote:
>>
>> Another "experimental data". I was able to load this big case in ParaView
>> viat the CGNS format. It was neccesary to select CGNS 3.0 and HDF5 options
>> in CFX Converter. But when I apply the Extract Block filter I see no any
>> boundaries, only fluid domain (that is displayed OK). So CGNS format is not
>> a wayout...
>>
>> 2016-02-29 9:00 GMT+03:00 Andrew <antech777 at gmail.com>:
>>
>>> I performed another test with *EnSi.ght 6* (not Gold) format.
>>> Conversion was made on Win-7 x64 machine. The same results: PV 4.3 crashed,
>>> PV 5.0 "cannot stat" (full error message is in my first post on this
>>> topic). Sorry, now I need to work on with my report...
>>>
>>>
>>>
>>>
>>>
>>> 2016-02-29 8:47 GMT+03:00 Andrew <antech777 at gmail.com>:
>>>
>>>> Hello. Sorry for silence, I'm usually offline on holydays, although I
>>>> have internet connection and read your mail.
>>>>
>>>> Samuel Key
>>>>
>>>> Thanks for your attention.
>>>>
>>>> About transferring files from Linux to Windows. I use binary EnSight
>>>> files because CFX converter fails to write so large files in ASCII format.
>>>> But I tried to convert from CFX to EnSight (binary) on the same Win-7 x64
>>>> machine with ParaView. Both installed versions (4.3 and 5.0) failed to open
>>>> the geometry (I unchecked all fields). ParaView 4.3 crashed, ParaView 5.0
>>>> just cannot open file...
>>>>
>>>> I think that the root cause may be a geometry file size. It is 4.9 GiB
>>>> (5.3 GB), mesh have about 150 millions of cells, not 600 thusands like in
>>>> your case. Its possible that ParaView for Windows cannot process so large
>>>> EnSight files, although the Linux version reads in this geometry and
>>>> displays outline (swapping is so intensive so I cannot displaye something
>>>> else).
>>>>
>>>> 2016-02-26 19:31 GMT+03:00 Samuel Key <samuelkey at bresnan.net>:
>>>>
>>>>> Andrew,
>>>>>
>>>>> For what it is worth, I use ParaView
>>>>> (ParaView-5.0.0-Qt4-OpenGL2-Windows-64bit) on Windows-7 64bit. I just now
>>>>> tested two medium-sized, EnSight Gold-formatted simulation results files
>>>>> (600,000 finite elements) and both worked.
>>>>>
>>>>> In the past, I have had difficulty with the EnSight ASCII-formatted
>>>>> *.case file when moving the simulation results (*.case file and friends)
>>>>> from MS Windows to EnSight running on a linux platform. It was necessary to
>>>>> pass the ASCII *.case file through a 'dos2unix' filter. Transferring files
>>>>> the other way might have the same problem???
>>>>>
>>>>> Hope this helps.
>>>>>
>>>>> Samuel Key
>>>>> FMA Development, LLC
>>>>> 1005 39th Ave NE
>>>>> Great Falls, Montana 59404
>>>>> USA
>>>>>
>>>>> On 2/25/2016 11:26 PM, Andrew wrote:
>>>>>
>>>>> Hello. We use Ansys CFX in our work but the standard post-processor
>>>>> CFD-Post consumes one of Ansys licenses while Paraview is free. So I often
>>>>> make pictures for reports with ParaView to preserve Ansys license for
>>>>> another users/needs (because other users are not familiar with ParaView and
>>>>> CFD-Post may consume general "acfd" license feature that may be used for
>>>>> calculations). I convert CFX results to EnSight Gold format in CFX Solver
>>>>> Manager (Export Results) and then open in ParaView. It was working well on
>>>>> Linux (CentOS 6.7 x64, Ubuntu 12.04 x64 with updates) but now I work with
>>>>> large files and I need to use another machine that has 128 GB of RAM and
>>>>> runs Windows-7 x64.
>>>>> I downloaded and installed the following version:
>>>>> ParaView-5.0.0-Qt4-OpenGL2-Windows-64bit.exe
>>>>> When I open my EnSight-format file I receive the error message that
>>>>> geometry file cannot be read. I tried to move EnSight files to a simple
>>>>> location (D:\Temp) and renamed them (test.case, test.geom + edited the case
>>>>> to change geometry file name to "simple" one). The same result, it cannot
>>>>> find geometry file... Full error message is as follows:
>>>>>
>>>>> ERROR: In
>>>>> C:\bbd\df0abce0\source-paraview\VTK\IO\EnSight\vtkEnSightGoldBinaryReader.cxx,
>>>>> line 114
>>>>> vtkEnSightGoldBinaryReader (0000000009451230): stat failed.
>>>>>
>>>>>
>>>>> ERROR: In
>>>>> C:\bbd\df0abce0\source-paraview\VTK\IO\EnSight\vtkEnSightGoldBinaryReader.cxx,
>>>>> line 220
>>>>> vtkEnSightGoldBinaryReader (0000000009451230): Unable to open file:
>>>>> D:\Storage\Media\Work\Gradirni\CFX\EnSight\C-G002L3S-R020-S000-14.5\/C-G002L3S-R020-S000.geom
>>>>>
>>>>>
>>>>> ERROR: In
>>>>> C:\bbd\df0abce0\source-paraview\VTK\IO\EnSight\vtkEnSightReader.cxx, line
>>>>> 306
>>>>> vtkEnSightGoldBinaryReader (0000000009451230): error reading geometry
>>>>> file
>>>>>
>>>>>
>>>>> ERROR: In
>>>>> C:\bbd\df0abce0\source-paraview\VTK\Common\ExecutionModel\vtkExecutive.cxx,
>>>>> line 784
>>>>> vtkCompositeDataPipeline (0000000008E1A640): Algorithm
>>>>> vtkEnSightGoldBinaryReader(0000000009451230) returned failure for request:
>>>>> vtkInformation (00000000093C3920)
>>>>>   Debug: Off
>>>>>   Modified Time: 214873
>>>>>   Reference Count: 1
>>>>>   Registered Events: (none)
>>>>>   Request: REQUEST_DATA
>>>>>   ALGORITHM_AFTER_FORWARD: 1
>>>>>   FORWARD_DIRECTION: 0
>>>>>   FROM_OUTPUT_PORT: 0
>>>>>
>>>>> OK, I tried to open it in ParaView 4.3. It says nothing (only border
>>>>> of the message window appears) and crashes. At that moment I see in the
>>>>> status bar that EnSight reader progress is 100%.
>>>>>
>>>>> Both ParaView 5.0.0 and 4.3 was tested with EnSight files exported on
>>>>> Linux and Windows machine. It didn't help.
>>>>>
>>>>> OK, I tried to export CFX results to CGNS format. ParaView 5.0.0
>>>>> (without MPI) cannot read it, so I fed it to ParaView 4.3. It says that its
>>>>> "unable to filed any meshes".
>>>>>
>>>>> I also performed an experiment on my Linux laptop. It has only 16 GB
>>>>> of RAM so its unreal to work with such a big file on this machine but it
>>>>> reads the same EnSight case and displays the domain! No arrays selected
>>>>> (mesh only), swap file grows up to 8 GB but the file opens without errors
>>>>> and crashes. It's only a test because I have not any Linux machine with
>>>>> memory large enough (our Linux cluster has 128 GB of RAM on every node
>>>>> including master but it has very poor video).
>>>>>
>>>>> I switched to CFD Post for this case but I want to know are there any
>>>>> workarounds to open my EnSight-format results in ParaView for Windows.
>>>>>
>>>>> Thanks for your attention.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Powered by www.kitware.com
>>>>>
>>>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>>>
>>>>> Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView
>>>>>
>>>>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>>>>
>>>>> Follow this link to subscribe/unsubscribe:http://public.kitware.com/mailman/listinfo/paraview
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Powered by www.kitware.com
>>>>>
>>>>> Visit other Kitware open-source projects at
>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>
>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>> http://paraview.org/Wiki/ParaView
>>>>>
>>>>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>>>>
>>>>> Follow this link to subscribe/unsubscribe:
>>>>> http://public.kitware.com/mailman/listinfo/paraview
>>>>>
>>>>>
>>>>
>>>
>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView
>>
>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>
>> Follow this link to subscribe/unsubscribe:http://public.kitware.com/mailman/listinfo/paraview
>>
>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the ParaView Wiki at:
>> http://paraview.org/Wiki/ParaView
>>
>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/paraview
>>
>>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView
>
> Search the list archives at: http://markmail.org/search/?q=ParaView
>
> Follow this link to subscribe/unsubscribe:http://public.kitware.com/mailman/listinfo/paraview
>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Search the list archives at: http://markmail.org/search/?q=ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/paraview
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20160301/552321d0/attachment.html>