[Paraview] CSV to PVTU

Mon Mar 6 13:18:40 EST 2017

Sorry, I might have mislead you a little bit before. What you actually need is the Glyph filter with Glyph Type set to 2D Glyph and then the secondary Glyph Type set to Vertex.

By default the glyph filter subsamples the data (to prevent occlusion when creating glyphs with size). Under Masking change the Glyph Mode to All Points, and you should get what you want.

Also, check the vtu files being generated by ParaView 4.0.1 carefully. I suspect one of them is very large and all the others are very small. I think the old version of ParaView is writing a bunch of vtu files with no points whereas new versions of ParaView are not writing out pointless empty vtu files.

-Ken

From: Faiz Abidi [mailto:fabidi89 at vt.edu]
Sent: Sunday, March 5, 2017 4:09 PM
To: Moreland, Kenneth <kmorel at sandia.gov>
Cc: ParaView <paraview at paraview.org>; Rajamohan, Srijith <srijithr at vt.edu>; Nicholas Polys <npolys at vt.edu>; Ayat Mohammed <maaayat at vt.edu>
Subject: Re: [EXTERNAL] Re: [Paraview] CSV to PVTU

Btw, I am a bit confused that why after applying a "2D Glyph" filter and "Glyph Type" as "Vertex", the number of points decreases. For example, if after applying "TableToPoints" filter I have 1 cell and 100k points, I get  502 cells and 502 points after applying a "2D Glyph" filter. I was expecting the number of cells to increase but total number of points should have remained 100k, no?

I am not really sure if applying a 2D Glyph is the best solution in my case. I may go back to using 4.0.1 to convert this csv data into pvtu and then use the latest version to load it up.

On Fri, Mar 3, 2017 at 2:14 PM, Faiz Abidi <fabidi89 at vt.edu<mailto:fabidi89 at vt.edu>> wrote:
I see.

I did that because when I choose "Assign Cells Uniquely" as the "Boundary Mode" in the D3 filter, I was getting a warning like below:

Warning: In /home/faiz89/git/ParaView/VTK/Common/DataModel/vtkKdTree.cxx, line 1887
vtkKdTree (0x2db5950): vtkKdTree::BuildMapForDuplicatePoints - invalid tolerance

But in spite of the warning, 8 vtu files get generated along with 1 pvtu file. So, maybe that warning can be ignored.

On Thu, Mar 2, 2017 at 6:42 PM, Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>> wrote:
I’m pretty sure if you select Duplicate Cells it will just copy all the points to all processes, so that is not a good option. The divide cells is also giving poor results on this front. I think you are better off using the 2D Glyph filter to break up the cell first.

-Ken

From: Faiz Abidi [mailto:fabidi89 at vt.edu<mailto:fabidi89 at vt.edu>]
Sent: Thursday, March 2, 2017 2:44 PM
To: Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>>
Cc: ParaView <paraview at paraview.org<mailto:paraview at paraview.org>>; Rajamohan, Srijith <srijithr at vt.edu<mailto:srijithr at vt.edu>>; Nicholas Polys <npolys at vt.edu<mailto:npolys at vt.edu>>; Ayat Mohammed <maaayat at vt.edu<mailto:maaayat at vt.edu>>
Subject: Re: [EXTERNAL] Re: [Paraview] CSV to PVTU

Ohk, that makes more sense. I could get it to work as well but also had to check "Boundary Mode" as "Duplicate Cells" while applying the D3 filter.

Thanks Moreland!

On Thu, Mar 2, 2017 at 3:59 PM, Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>> wrote:
Faiz,

I was able to replicate your problem. It looks like it is not an issue with D3 but a change with how Table to Points creates data (or perhaps how data are written). It looks like Table to Points is creating one “cell” with a lot of points. This saves a little memory but ends up preventing D3 from dividing the points up.

The best solution is to add a “2D Glyph” filter after the Table to Points but before the D3 filter. Set the “Glyph Type” to “Vertex”. Then apply the D3 filter to the output of the 2D Glyph filter.

-Ken

From: Faiz Abidi [mailto:fabidi89 at vt.edu<mailto:fabidi89 at vt.edu>]
Sent: Thursday, March 2, 2017 1:38 PM
To: Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>>
Cc: ParaView <paraview at paraview.org<mailto:paraview at paraview.org>>; Rajamohan, Srijith <srijithr at vt.edu<mailto:srijithr at vt.edu>>; Nicholas Polys <npolys at vt.edu<mailto:npolys at vt.edu>>; Ayat Mohammed <maaayat at vt.edu<mailto:maaayat at vt.edu>>
Subject: [EXTERNAL] Re: [Paraview] CSV to PVTU

Hi Moreland,

Thanks for your reply.

I thought same except maybe not use the GUI since my data is big and takes forever to load up in Paraview.

Quick question/observation - using the GUI and 8 processes, and applying the Tables to Points filter + D3 and then trying to save files doesn't create 8 vtu files and 1 pvtu file as expected. Instead, what happens is that I get only 1 pvtu file along with 1 vtu file instead. Clearly something wrong. This was done using v5.2.0.

I tried the same thing using v4.0.1 (yes, old but it works!) and I got 8 vtu files and 1 pvtu file as expected.

Do you think this is a bug with 5.2.0? Or do I need to do something different with 5.2.0? Any pointers appreciated!

On Wed, Mar 1, 2017 at 4:15 PM, Moreland, Kenneth <kmorel at sandia.gov<mailto:kmorel at sandia.gov>> wrote:
You can use ParaView itself to read in the csv file, convert it to an unstructured grid (Table to Points filter), redistributed on your 8 parallel processors (D3 filter) and then write out a pvtu file.

Offhand I cannot think of an easier way.

-Ken

From: ParaView [mailto:paraview-bounces at paraview.org<mailto:paraview-bounces at paraview.org>] On Behalf Of Faiz Abidi
Sent: Wednesday, March 1, 2017 2:03 PM
To: ParaView <paraview at paraview.org<mailto:paraview at paraview.org>>
Subject: [EXTERNAL] [Paraview] CSV to PVTU

Hi again community!

Looking for suggestions - what would be a decently fast way to convert a huge CSV file into PVTU files?

What I want to do - load this huge file in a CAVE like environment, and use 8 parallel processors.
--
Faiz Abidi | Master's Student at Virginia Tech | www.faizabidi.com<http://www.faizabidi.com> | +1-540-998-6636<tel:(540)%20998-6636>

--
Faiz Abidi | Master's Student at Virginia Tech | www.faizabidi.com<http://www.faizabidi.com> | +1-540-998-6636<tel:(540)%20998-6636>

--
Faiz Abidi | Master's Student at Virginia Tech | www.faizabidi.com<http://www.faizabidi.com> | +1-540-998-6636<tel:(540)%20998-6636>

--
Faiz Abidi | Master's Student at Virginia Tech | www.faizabidi.com<http://www.faizabidi.com> | +1-540-998-6636<tel:(540)%20998-6636>

--
Faiz Abidi | Master's Student at Virginia Tech | www.faizabidi.com<http://www.faizabidi.com> | +1-540-998-6636
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20170306/009bb982/attachment.html>