It's got a bit of a learning curve, but have you thought about trying to integrate HDF5 into your code? It supports a whole heck of alot of datatypes, and has built-in compression. Couple it with Xdmf from ARL (Write an HDF5 file & a simple ascii Metadata file) or Xmdf from ERDC (Just write the HDF5 stuff into specially named fields) and it's pretty nice to work with.
<br><br><div><span class="gmail_quote">On 1/31/06, <b class="gmail_sendername">Dominik Szczerba</b> <<a href="mailto:domi@vision.ee.ethz.ch">domi@vision.ee.ethz.ch</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I see the point. And yes, I want to stay far away from ASCII. So what<br>format would you recommend to STORE my results in a COMPRESSED way<br>maintaining readability with Paraview? Any binary legacy file I produce<br>takes gigabytes - crunched with gzip a fraction of it, but paraview
<br>won't read it. ASCII is simply prohibitive, base64'ed with<br>zlibcompressor not much better. At the moment I store my results in my<br>own gzipped format, and convert them to VTK format on demand - but that<br>is pretty annoying.
<br>regards,<br>--dsz<br><br>Berk Geveci wrote:<br>> Base64 encoding is only necessary if you want your XML file to be<br>> ASCII. This is necessary if you want the file to conform to XML so<br>> that the file can be opened in XML viewers (like a browser). Also,
<br>> some people prefer transferring ASCII files instead of binary files.<br>> If you are not interested in either of these, there is no need to have<br>> base64 encoding. I would recommend not using it in fact. One of these
<br>> days I will change the default in paraview so that it does not write<br>> base64 encoded files.<br>><br>><br>> On 1/31/06, Dominik Szczerba <<a href="mailto:domi@vision.ee.ethz.ch">domi@vision.ee.ethz.ch
</a>> wrote:<br>><br>> Sorry to interupt your discussion but nobody ever explained me this: why<br>> use base64 encoding at all? Isint it a step back to huge ASCII files??<br>> E.g. I see no significant compression with the VTK's zlib compressor
<br>> when using XML data files because the compressed stream is converted to<br>> ASCII. Will someone kindly explain me this paradox?<br>> Thanks<br>> Dominik<br>><br>> Thierry Dubuis wrote:<br>><br>
>>There are lot of encoding Base 64 program on the web but only in C, Java<br>>>or Visual Basic.<br>><br>>>If you can use C/C++ compiler, I thing the best way for you is to use<br>>>the Brad King's fortran code example (see my last email).
<br>>>To convert a data array to a string in C we use the memory copy in<br>>>fortran I don't know.<br>>>Could you use pointer? 2 pointer on the same memory adress<br>>>1. the fist with the real*8 type
<br>>>2. the second with a character type<br>><br>>>Try in this way.<br>><br>>>Best regards,<br>>>Thierry Dubuis<br>><br>>>Stéphane Montésino wrote:<br>><br>><br>>>>Thanks for all tese answers, but I 'm still like an an homosexual with
<br>>>>a wonderfull girl. I don't know how to use it (sorry her).<br>><br>>>>In fact, I would like to directly write this type of file that I can<br>>>>obtain when I save my data with Paraview:
<br>><br>>>><?xml version="1.0"?><br>>>><VTKFile type="RectilinearGrid" version="0.1"<br>>>>byte_order="LittleEndian" compressor="vtkZLibDataCompressor">
<br>>>> <RectilinearGrid WholeExtent="1 8 1 4 1 16"><br>>>> <Piece Extent="1 8 1 4 1 16"><br>>>> <PointData><br>>>> <DataArray type="Float32" Name="pressure" format="appended"
<br>>>>offset="0" /><br>>>> </PointData><br>>>> <CellData><br>>>> </CellData><br>>>> <Coordinates><br>>>> <DataArray type="Float32" Name="X_COORDINATES"
<br>>>>format="appended" offset="956" /><br>>>> <DataArray type="Float32" Name="Y_COORDINATES"<br>>>>format="appended" offset="1036" />
<br>>>> <DataArray type="Float32" Name="Z_COORDINATES"<br>>>>format="appended" offset="1092" /><br>>>> </Coordinates><br>>>> </Piece>
<br>>>> </RectilinearGrid><br>>>> <AppendedData encoding="base64"><br>>>> _AQAAAACAAAAACAAAuQIAAA==eJwVy8fvoFMUx+HDdIkyWOiG/0Bvmzfm etc.....<br>>>> </AppendedData>
<br>>>></VTKFile><br>><br>>>>My knowledge are only reduces to Fluid mechanics.<br>><br>><br>>>>>>>>>>>>>>>>>Encode in Base 64 ? More details please...How to ?
<br>><br>>>><<<<<<<<<<<<<<<<<<<<<<<br>><br>>>>Link properly to vtkIO ? More details please.<br>><br>>>>>>>>>>>>>>>>>>Does it mean that VTK IO library should be installed on
<br>><br>>>>the Computer where my CFD code run ?.<br>>>>I don't run my code on my own computer. It runs in Computing center<br>>>>where I can't install what I want.<br>><br>>>>>>>>>>>>So I would like to obtain some fortran routine I could link
<br>><br>>>>myself to my CFD code <<<<<<<<<<<<<<<<<br>><br>>>>When I try to write binary data in a formatted file:<br>>>>run-time error F6204: WRITE(
pressure.vtr)<br>>>>- unformatted I/O not consistent with OPEN options<br>><br>>>>When I try to write formatted string in an unformatted file:<br>>>>run-time error F6200: WRITE(pressure.vtr)
<br>>>>- formatted I/O not consistent with OPEN options<br>><br>><br>>>>>>>>>>>>>>>>>>>>How to convert my data to a string ?<br>><br>>>><<<<<<<<<<<<<<<<<<<<<
<br>><br>><br>>>>The format for raw appended binary data should be<br>><br>>>><AppendedData encoding="raw"><br>>>> _NNNN...........................<br>>>></AppendedData>
<br>><br>>>>Where the underscore is literal (as you already have) and the NNNN is a<br>>>>four-byte block containing an unsigned 32-bit count of the number of<br>>>>"words" in the array. A word is one value of the type of the array, so
<br>>>>in your case the NNNN would have the number of float values. Then the<br>>>>number of bytes of binary data to follow should be NNNN*sizeof(float).<br>><br>>>>If you have another appended data array in the same file then start it
<br>>>>immediately after the end of the first array (with no newline or any<br>>>>other character. Then specify in the offset="..." attribute for the<br>>>>corresponding DataArray the number of bytes beyond the underscore where
<br>>>>the second array begins.<br>><br>>>>-Brad<br>><br>><br>><br>><br>>>>Thierry Dubuis wrote:<br>><br>><br>>>>>Hello Stéphane,<br>>>>><br>>>>>perhaps this email can help you ?
<br>>>>><br>>>>>Else if you want code directly your program in fortran :<br>>>>>1. You must encode all you data in Base64<br>>>>>2. And write them like a string.<br>>>>>3. don't forget to add the size of your data block (integer*4)
<br>>>>>encoded in base64 before you block.<br>>>>><br>>>>>Best regards,<br>>>>>Thierry Dubuis<br>>>>><br>>>>>-------- Original Message --------<br>>>>>Subject: Re: [Paraview] Writing binary files: C/Fortran interface
<br>>>>>to VTK XML files.<br>>>>>Date: Tue, 07 Jun 2005 10:06:51 -0400<br>>>>>From: Brad King <<a href="mailto:brad.king@kitware.com">brad.king@kitware.com</a>><br>>>>>To: SamuelKey <
<a href="mailto:samuelkey@comcast.net">samuelkey@comcast.net</a>><br>>>>>CC: paraview <<a href="mailto:paraview@paraview.org">paraview@paraview.org</a>>, "Wylie, Brian"<br>>>>><
<a href="mailto:bnwylie@sandia.gov">bnwylie@sandia.gov</a>><br>>>>>References:<br>>>>><<a href="mailto:200506030152.j531quWp005650@rrcs-mta-04.hrndva.rr.com">200506030152.j531quWp005650@rrcs-mta-04.hrndva.rr.com
</a>><br>>>>><<a href="mailto:42A05837.3010206@kitware.com">42A05837.3010206@kitware.com</a>> <000a01c5684c$1d1a6740$6e01a8c0@last><br>>>>><br>>>>><br>>>>><br>
>>>>SamuelKey wrote:<br>>>>><br>>>>><br>>>>>>Needing to have a C-compiler is not a problem for me; I have one.<br>>>>>><br>>>>>>Let me try to work with what you commit next week and see if I have
<br>>>>>>any ideas. I will be gone all next week, so you won't hear from me<br>>>>>>until week after next. With luck you may hear from someone else<br>>>>>>that will help keep you busy, though I suspect "busy" is not a
<br>>>>>>problem for you.<br>>>>><br>>>>><br>>>>><br>>>>><br>>>>>Okay, the C/Fortran interface is now available. A C program can do<br>>>>>
<br>>>>>#include "vtkXMLWriterC.h"<br>>>>><br>>>>>and link to vtkIO. A Fortran program can use the interface (see the<br>>>>>example code below) but will need to compile one extra .c file to
<br>>>>>link properly to vtkIO. See VTK/IO/vtkXMLWriterF.h for instructions.<br>>>>><br>>>>>-Brad<br>>>>><br>>>>> PROGRAM Hello<br>>>>> INTEGER writer
<br>>>>> INTEGER success<br>>>>> INTEGER*8 numpoints/8/<br>>>>> INTEGER*8 numcells/1/<br>>>>> INTEGER*8 cellsSize/9/<br>>>>> REAL*4 pdata(8)/0,0,0,0,1,1,1,1/
<br>>>>> REAL*4 points(24)/0,0,0, 1,0,0, 1,1,0, 0,1,0,<br>>>>> _ 0,0,1, 1,0,1, 1,1,1, 0,1,1/<br>>>>> INTEGER*8 cells(9)/8,0,1,2,3,4,5,6,7/<br>>>>> call vtkXMLWriterF_New(writer)
<br>>>>> call vtkXMLWriterF_SetDataObjectType(writer, 4)<br>>>>> call vtkXMLWriterF_SetFileName(writer, 'test1.vtu')<br>>>>> call vtkXMLWriterF_SetPoints(writer, 10, points, numpoints)
<br>>>>> call vtkXMLWriterF_SetCellsWithType(writer, 12, numcells, cells,<br>>>>> _ cellsSize)<br>>>>> call vtkXMLWriterF_SetPointData(writer, 'example data', 10, pdata,
<br>>>>> _ numpoints, 1, 'SCALARS')<br>>>>> call vtkXMLWriterF_Write(writer, success)<br>>>>> call vtkXMLWriterF_Delete(writer)<br>>>>> PRINT *, 'success =', success
<br>>>>> END PROGRAM Hello<br>>>>>_______________________________________________<br>>>>>ParaView mailing list<br>>>>><a href="mailto:ParaView@paraview.org">ParaView@paraview.org
</a><br>>>>><a href="http://www.paraview.org/mailman/listinfo/paraview">http://www.paraview.org/mailman/listinfo/paraview</a><br>>>>><br>>>>><br>>>>><br>>>>><br>
>>>>Stéphane Montésino wrote:<br>>>>><br>>>>><br>>>>>>With Asci, all is OK except the size of my file.<br>>>>>>So how to write my data array in binary with my fortran code.
<br>>>>>>Does anyone have some details ?<br>>>>>><br>>>>>>!<br>>>>>>******************************************************************************<br>>>>>>
<br>>>>>>! **** subrout<br>>>>>>paraview_scalar ****<br>>>>>>!<br>>>>>>******************************************************************************
<br>>>>>><br>>>>>>! Export 3D scalar data upon a Non uniform rectilinear Grid<br>>>>>>!<br>>>>>>! n1m,n2m,n3m: size of the data and the grid<br>>>>>>! y1,y2,y3: coordinates
<br>>>>>>! scal_data: data to write<br>>>>>>! name: name of the data<br>>>>>>!<br>>>>>> subroutine paraview_3d_scalar(n1m,n2m,n3m,y1,y2,y3,scal_data,name)
<br>>>>>>!<br>>>>>> implicit none<br>>>>>>!<br>>>>>> INTEGER ,INTENT(IN)::n1m,n2m,n3m<br>>>>>> REAL*8,DIMENSION(n1m) ,INTENT(IN)::y1
<br>>>>>> REAL*8,DIMENSION(n2m) ,INTENT(IN)::y2<br>>>>>> REAL*8,DIMENSION(n3m) ,INTENT(IN)::y3<br>>>>>> REAL*8,DIMENSION(n1m,n2m,n3m),INTENT(IN)::scal_data
<br>>>>>> CHARACTER(LEN=10) ,INTENT(IN)::name<br>>>>>>!<br>>>>>> integer::i,j,k,nfil<br>>>>>>!<br>>>>>> nfil=41<br>>>>>> open(nfil,file=trim(name)//'.vtr')
<br>>>>>> write(nfil,*)'<VTKFile type="RectilinearGrid" version="0.1"',<br>>>>>> & ' byte_order="LittleEndian">'<br>>>>>> write(nfil,*)' <RectilinearGrid WholeExtent=',
<br>>>>>> & '"1 ',n1m,' 1 ',n2m,' 1 ',n3m,'">'<br>>>>>> write(nfil,*)' <Piece Extent=',<br>>>>>> & '"1 ',n1m,' 1 ',n2m,' 1 ',n3m,'">'
<br>>>>>> write(nfil,*)' <Coordinates>'<br>>>>>> write(nfil,*)' <DataArray type="Float32"',<br>>>>>> & ' Name="X_COORDINATES"',
<br>>>>>> & ' NumberOfComponents="1">'<br>>>>>> write(nfil,*) (y1(i),i=1,n1m)<br>>>>>> write(nfil,*)' </DataArray>'
<br>>>>>> write(nfil,*)' <DataArray type="Float32"',<br>>>>>> & ' Name="Y_COORDINATES"',<br>>>>>> & ' NumberOfComponents="1">'
<br>>>>>> write(nfil,*) (y2(j),j=1,n2m)<br>>>>>> write(nfil,*)' </DataArray>'<br>>>>>> write(nfil,*)' <DataArray type="Float32"',<br>
>>>>> & ' Name="Z_COORDINATES"',<br>>>>>> & ' NumberOfComponents="1">'<br>>>>>> write(nfil,*) (y3(k),k=1,n3m)
<br>>>>>> write(nfil,*)' </DataArray>'<br>>>>>> write(nfil,*)' </Coordinates>'<br>>>>>> write(nfil,*)' <PointData Scalars="scalar">'
<br>>>>>> write(nfil,*)' <DataArray Name="'//trim(name)//'"',<br>>>>>> & ' type="Float32"',<br>>>>>> & ' NumberOfComponents="1"',
<br>>>>>> & ' format="ascii">'<br>>>>>> write(nfil,*) (((scal_data(i,j,k),i=1,n1m),j=1,n2m),k=1,n3m)<br>>>>>> write(nfil,*)' </DataArray>'
<br>>>>>> write(nfil,*)' </PointData>'<br>>>>>> write(nfil,*)' </Piece>'<br>>>>>> write(nfil,*)' </RectilinearGrid>'<br>>>>>> write(nfil,*)'</VTKFile>'
<br>>>>>> close(nfil)<br>>>>>>!<br>>>>>> return<br>>>>>> end<br>>>>>><br>>>>>>_______________________________________________
<br>>>>>>ParaView mailing list<br>>>>>><a href="mailto:ParaView@paraview.org">ParaView@paraview.org</a><br>>>>>><a href="http://www.paraview.org/mailman/listinfo/paraview">http://www.paraview.org/mailman/listinfo/paraview
</a><br>>>>>><br>>>>>><br>>>>><br>><br>><br>> --<br>> Dominik Szczerba, Dr.<br>> Computer Vision Lab CH-8092 Zurich<br>> <a href="http://www.vision.ee.ethz.ch/~domi">
http://www.vision.ee.ethz.ch/~domi</a><br>_______________________________________________<br>ParaView mailing list<br><a href="mailto:ParaView@paraview.org">ParaView@paraview.org</a><br><a href="http://www.paraview.org/mailman/listinfo/paraview">
http://www.paraview.org/mailman/listinfo/paraview</a><br><br>--<br>Dominik Szczerba, Dr.<br>Computer Vision Lab CH-8092 Zurich<br><a href="http://www.vision.ee.ethz.ch/~domi">http://www.vision.ee.ethz.ch/~domi</a><br>_______________________________________________
<br>ParaView mailing list<br><a href="mailto:ParaView@paraview.org">ParaView@paraview.org</a><br><a href="http://www.paraview.org/mailman/listinfo/paraview">http://www.paraview.org/mailman/listinfo/paraview</a><br></blockquote>
</div><br><br clear="all"><br>-- <br>Randall Hand<br>Visualization Scientist, <br>ERDC-MSRC Vicksburg, MS<br>Homepage: <a href="http://www.yeraze.com">http://www.yeraze.com</a>