[Paraview] MongoDB Reader

[Eric E. Monson]

Hey Scott,

I haven't heard of anyone doing this sort of work. In general, most of the VTK-related work with database queries seems to come from the statistics and information visualization side, rather than the large data scientific visualization folks which ParaView is traditionally aimed towards. So, you get people doing SQL queries to produce a VTK table, and from there the various things you can do with that.

It's not clear from your email what type of data you would be wanting to visualize with ParaView, and how the data in MongoDB would be structured. One large issue (and a reason you don't see much work in this direction) is that even if you store something like point coordinates in your database, you also need to describe the topology of the structures you're going to build with those points, whether that means just point vertices, or polygons of some sort, or regular grids. There just aren't any standard data formats which include something like a document-based NoSQL database for storing the data. The analogy that comes to my mind for something similar that ParaView can handle is the XDMF file format (xdmf.org). In this format, an HDF5 file can be used to store the data, but since that file format is so general and flexible, they need an XML file to go with the data which describes the topology, time steps, etc., and where to look in the HDF5 file for that information. You would have a similar issue, that MongoDB is almost infinitely flexible, and so unless there is a large community using it for similar purposes, who is willing to come up with a standard data format stored in the DB, there probably won't be something pre-made that you can use.

Now, on a more optimistic note, :), I tried a few minutes ago and had some success with reading information from a MongoDB database into ParaView using the Python Programmable Source. Some examples for creating data are here:

http://www.paraview.org/Wiki/Python_Programmable_Filter#Generating_Data_.28Programmable_Source.29

I had pymongo installed for the version of Python that ParaView 3.14 (downloaded binary, Mac OS X 10.7) happens to use (which I didn't know, but found by printing out sys.path and saw that it included /Library/Python/2.5/site-packages so my Python 2.5 was being used). You can see from the attached PNG image that I didn't generate any real geometric data to display, but I accessed MongoDB and queried it and printed out some values from the returned documents in the Output Window, so I'm confident that I could put some data in Mongo and use the examples on the ParaView site to create something real to display.

My knowledge isn't too deep in any of these areas, but I'll do what I can to answer other questions if you get stuck.

Good luck,
-Eric

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·
Eric E Monson
Duke Visualization Technology Group




On Jun 23, 2012, at 2:10 PM, Scott Schenck wrote:

> Hi,
> I'm very new to the list and paraview.  I'm curious if anyone has done any work to read data from a MongoDB database.  I'm not sure of the best way to get large amounts of simulation results out of MongoDB to visualize in Paraview.
> 
> Obviously, I could write it out to a cvs file and use a cvs reader.  Is there a more elegant way to accomplish this task without taking an intermediate step?  MongoDB is a nosql database that stores data in BSON/JSON data structures.  
> 
> I guess I could write a reader or plugin tailored expressly towards MongodB.  I was hoping to not have to develop something from the ground up.
> 
> Any ideas?
> Thanks,
> Scott
> 
> Scott N. Schenck, Ed.S., CHt
> Conscious Choices Corporation
> 
> 131 4th Ave W, Suite 204
> Hendersonville, NC. 28792
> 
> (828) 329-7992 (HVL)
> 
> Scott at consciouschoices.com
> www.consciouschoices.com
> 
> -Transform your life through Conscious Choices!
> 
> _______________________________________________
> Powered by www.kitware.com
> 
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
> 
> Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView
> 
> Follow this link to subscribe/unsubscribe:
> http://www.paraview.org/mailman/listinfo/paraview

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120702/2c45e580/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MongoDB_prog_source.png
Type: image/png
Size: 56197 bytes
Desc: not available
URL: <http://www.paraview.org/pipermail/paraview/attachments/20120702/2c45e580/attachment-0001.png>