[Paraview] Linking to Catalyst

Tue May 10 10:18:20 EDT 2016

Hi Tim,

If I remember correctly, that was VTK's suggested way at that time with
dealing with arrays that had different memory layouts than VTK used (and
yes, I think I may have put that in the code). In PV master now there's a
much better way of doing it (it's not in PV 5.0.1 but will be in PV 5.1).
If you use the vtkSOADataArrayTemplate class it will save on memory and be
more efficient. An example looks like:
    vtkSOADataArrayTemplate<double>* coords =
vtkSOADataArrayTemplate<double>::New();
    coords->SetNumberOfComponents(3);
    coords->SetNumberOfTuples(this->NumPoints);
    coords->SetArray(0, this->CoordsArray, this->NumPoints, false, true);
    coords->SetArray(1, this->CoordsArray+this->NumPoints, this->NumPoints,
false, true);
    coords->SetArray(2, this->CoordsArray+this->NumPoints*2,
this->NumPoints, false, true);

Here, CoordsArray is a double* that stores the coordinates in SOA form (x0,
x1, x2, ..., y0, y1, y2, ..., z0, z1, z2, ...) and NumPoints is the number
of points in the array (i.e. the length of the array divided by 3). Some
very fine details are at http://www.vtk.org/Wiki/VTK/Tutorials/DataArrays
if you want to read more about it but that's really for people writing VTK
filters and stuff like that.

As for the warning, you can comment that out in the code if it's generating
too much noise.

Best,
Andy

On Tue, May 10, 2016 at 12:35 AM, Gallagher, Timothy P <
tim.gallagher at gatech.edu> wrote:

> Ah hah, the errors were being sent somewhere else and not caught properly
> by my submission script.
>
>
> I found the error and now it works, I get my images and my VTK files. The
> only warnings now are:
>
>
> 24vtkCPTensorArrayTemplateIdE (0x23b4320): GetVoidPointer called. This is
> very expensive for vtkMappedDataArray subclasses, since the scalar array
> must be generated for each call. Consider using a vtkTypedDataArrayIterator
> instead.
>
>
> which is something I can deal with later. I have to figure out what the
> vtkCPTensorArrayTemplate does exactly -- I believe that was something you
> had added when you took a look at our code awhile back. But it doesn't seem
> to matter for what we're doing now.
>
>
> It feels good to close the ticket in our tracker about getting
> coprocessing on Crays -- it's been open since 2012.
>
> Thanks again for all your help.
>
>
> Tim
>
> ------------------------------
> *From:* Andy Bauer <andy.bauer at kitware.com>
> *Sent:* Monday, May 9, 2016 10:41 PM
> *To:* Gallagher, Timothy P
> *Cc:* paraview at paraview.org
> *Subject:* Re: [Paraview] Linking to Catalyst
>
> Hi Tim,
>
> I assume that writing out a data set works on other machines, correct? Can
> you try the gridwriter.py script at
> https://gitlab.kitware.com/paraview/paraview/blob/master/Examples/Catalyst/SampleScripts/gridwriter.py
> to output the full data set? It should work with PV 4.4. Note that outputs
> a full data set every time step. I'm not sure about Python warnings or
> errors but the C++ warnings and errors will be sent to std::cerr instead of
> std::cout.
>
> <https://gitlab.kitware.com/paraview/paraview/blob/master/Examples/Catalyst/SampleScripts/gridwriter.py>
> Examples/Catalyst/SampleScripts/gridwriter.py · master · ParaView /
> ParaView
> <https://gitlab.kitware.com/paraview/paraview/blob/master/Examples/Catalyst/SampleScripts/gridwriter.py>
> gitlab.kitware.com
> Parallel Data Analysis and Visualization Application based on VTK.
>
>
> Did you make the script with PV 4.4? The PV Python API has some changes
> (especially for rendering) between revisions. I think the best option is to
> just comment out the parts that are causing trouble.
>
> If you're still having issues, maybe send your full script along with
> std::cout and std::cerr outputs so I can get a little deeper into the
> issues.
>
> Good luck,
> Andy
>
> On Mon, May 9, 2016 at 6:14 PM, Gallagher, Timothy P <
> tim.gallagher at gatech.edu> wrote:
>
>> Okay, really close to finally crossing this off my list -- it's been on
>> it for years.
>>
>>
>> I can get the code to generate images but it won't save data files. I
>> don't get any errors and I get my images, just no VTK files. I have tried
>> the default that the catalyst generator creates:
>>
>>
>> parallelMultiBlockDataSetWriter1 =
>> servermanager.writers.XMLMultiBlockDataWriter(Input=slice1)
>>
>> coprocessor.RegisterWriter(parallelMultiBlockDataSetWriter1,
>> filename='slices_%t.vtm', freq=freq_file)
>>
>>
>> and I tried another form that the grid writer you've send me in the past
>> used:
>>
>>
>> writer = coprocessor.CreateWriter(XMLMultiBlockDataWriter,
>> filename='slices_%t.vtm', freq=freq_file)
>>
>>
>> in all cases, freq_file = 1000.
>>
>>
>> Neither one will output any vtk files.
>>
>>
>> Any thoughts?
>>
>>
>> Also, minor note but it is kind of annoying -- both the catalyst
>> generator and the trace generator seem to do this, but they add commands
>> that set attributes that are not part of the classes when run through the
>> python script. For example, I had to comment out:
>>
>>
>> #      slice1Display.SelectInputVectors = ['CELLS', 'Velocity
>> [m/s]']
>>
>> #      slice1Display.WriteLog = ''
>>
>>
>> Any ideas why it does that? Or is there a way (and downfall) to just let
>> it add attributes without manually calling add_attribute() each time?
>>
>>
>> Tim
>>
>>
>> ------------------------------
>> *From:* ParaView <paraview-bounces at paraview.org> on behalf of Gallagher,
>> Timothy P <tim.gallagher at gatech.edu>
>> *Sent:* Monday, May 9, 2016 5:13 PM
>> *To:* Andy Bauer
>>
>> *Cc:* paraview at paraview.org
>> *Subject:* Re: [Paraview] Linking to Catalyst
>>
>>
>> Yeah -- okay, pointing it to the build directory instead of the install
>> made sure everything was found.
>>
>>
>> One step closer to getting this working on Cray.
>>
>>
>> Thanks,
>>
>>
>> Tim
>>
>>
>> ------------------------------
>> *From:* Andy Bauer <andy.bauer at kitware.com>
>> *Sent:* Monday, May 9, 2016 4:54 PM
>> *To:* Gallagher, Timothy P
>> *Cc:* paraview at paraview.org
>> *Subject:* Re: [Paraview] Linking to Catalyst
>>
>> Hmm, that could be a bug in the superbuild. The superbuild arguments
>> aren't all passed to the ParaView build itself and it could be that this
>> option was missing for the PV specific build part. Instead of doing the
>> whole build from scratch, I'd suggest you just search for the
>> ParaViewConfig.cmake file to find where PV was actually built and just
>> point to that.
>>
>> On Mon, May 9, 2016 at 4:50 PM, Gallagher, Timothy P <
>> tim.gallagher at gatech.edu> wrote:
>>
>>> Well shoot, it looks like I didn't build with the install development
>>> files on, even though I set it when I configured the superbuild. I must
>>> have done that part wrong
>>>
>>> My configure line is:
>>>
>>>
>>> cmake \
>>> -DCMAKE_INSTALL_PREFIX:PATH=${HOME}/pv-test/${version}_osmesa \
>>> -DCMAKE_BUILD_TYPE=Release \
>>> ...
>>>
>>> -DPARAVIEW_ENABLE_CATALYST=ON \
>>> -DPARAVIEW_ENABLE_PYTHON=ON \
>>> -DPARAVIEW_INSTALL_DEVELOPMENT_FILES=ON \
>>> -DPARAVIEW_BUILD_QT_GUI=OFF \
>>> -DBUILD_TESTING=OFF \
>>> -DVTK_USE_X=OFF \
>>> ...
>>>
>>> -DVTK_OPENGL_HAS_OSMESA=ON \
>>> -DOSMESA_INCLUDE_DIR:STRING="${INSTALLPATH}/osmesa/include" \
>>> -DOSMESA_LIBRARY:STRING="${INSTALLPATH}/osmesa/lib/libOSMesa.so" \
>>> ./ParaViewSuperbuild
>>>
>>>
>>> and all of the other PARAVIEW* options seemed to pass through properly
>>> when the superbuild made paraview. Any ideas why that option didn't pass
>>> through?
>>>
>>>
>>> Thanks -- I knew it had to be something really obvious!
>>>
>>>
>>> Tim
>>>
>>>
>>>
>>> ------------------------------
>>> *From:* Andy Bauer <andy.bauer at kitware.com>
>>> *Sent:* Monday, May 9, 2016 4:43 PM
>>> *To:* Gallagher, Timothy P
>>> *Cc:* paraview at paraview.org
>>> *Subject:* Re: [Paraview] Linking to Catalyst
>>>
>>> Hi Tim,
>>>
>>> Is the PV directory you're pointing to a build directory or an install
>>> directory? If it's an install directory you'll need to enable
>>> PARAVIEW_INSTALL_DEVELOPMENT_FILES. Other than that, my suggestion would be
>>> to try linking one of the Catalyst examples from
>>> https://github.com/Kitware/ParaViewCatalystExampleCode and see how that
>>> works for you. Another thing you could try is using a newer version of
>>> CMake from https://cmake.org/download/. For the Linux x86_64 tarball,
>>> you can just untar the executables from that and use directly.
>>> <https://github.com/Kitware/ParaViewCatalystExampleCode>
>>> GitHub - Kitware/ParaViewCatalystExampleCode: Example ...
>>> <https://github.com/Kitware/ParaViewCatalystExampleCode>
>>> github.com
>>> ParaViewCatalystExampleCode - Example problems and snippets of code to
>>> demonstrate ParaView's Catalyst.
>>>
>>>
>>> Let us know if none of those ideas don't work for you.
>>>
>>> Best,
>>> Andy
>>>
>>> On Mon, May 9, 2016 at 3:58 PM, Gallagher, Timothy P <
>>> tim.gallagher at gatech.edu> wrote:
>>>
>>>> Hello again,
>>>>
>>>>
>>>> I'm (finally) trying to get Catalyst to successfully link to our
>>>> application code on Excalibur (Cray, ARL HPC). I was able to build paraview
>>>> and all of the dependencies thanks to help I've gotten on the list here in
>>>> the past. However, when I try to link my code to it (and this is a code
>>>> that works fine with Catalyst on other platforms), I get:
>>>>
>>>>
>>>>  CMake Error at
>>>> /p/home/tgallagh/pv-test/4.4.0_osmesa/lib/cmake/paraview-4.4/vtkModuleAPI.cmake:120
>>>> (message):
>>>>    Requested modules not available:
>>>>
>>>>      vtkPVPythonCatalyst
>>>>  Call Stack (most recent call first):
>>>>
>>>> /p/home/tgallagh/pv-test/4.4.0_osmesa/lib/cmake/paraview-4.4/VTKConfig.cmake:80
>>>> (vtk_module_config)
>>>>
>>>> /p/home/tgallagh/pv-test/4.4.0_osmesa/lib/cmake/paraview-4.4/ParaViewConfig.cmake:49
>>>> (include)
>>>>    CMakeLists.txt:218 (find_package)
>>>>
>>>> The section in my CMakeLists that looks for paraview is:
>>>>
>>>> option(LESLIE_USE_COPROCESSING "Turn on CoProcessing with Paraview" OFF)
>>>> if(LESLIE_USE_COPROCESSING)
>>>>   find_package(ParaView REQUIRED vtkPVPythonCatalyst
>>>>                HINTS $ENV{PARAVIEW_CP_ROOT})
>>>>   include(${PARAVIEW_USE_FILE})
>>>>   include_directories(${PROJECT_SOURCE_DIR}/utils)
>>>>   add_definitions(-DPARAVIEW_COPROCESSING)
>>>> endif()
>>>> mark_as_advanced(LESLIE_USE_COPROCESSING)
>>>>
>>>> and it doesn't seem to work. On other machines, I never needed to set
>>>> the PARAVIEW_CP_ROOT variable, it always just found it. When I build my
>>>> code, I point the paraview directory (where it finds the
>>>> paraviewConfig.cmake file) to:
>>>>
>>>>
>>>> /p/home/tgallagh/pv-test/4.4.0_osmesa/lib/cmake/paraview-4.4
>>>>
>>>>
>>>> and it seems to find it okay.
>>>>
>>>>
>>>> Lastly, the vtkPVPythonCatalyst.so library is in the
>>>> lib/paraview-4.4/site-packages/vtk directory and there is
>>>> libvtkPVPythonCatalyst-pv4.4.so* and
>>>> libvtkPVPythonCatalystPython27D-pv4.4.so* in lib/paraview-4.4/
>>>>
>>>>
>>>> I'm at a loss to understand why it is reporting the module is missing
>>>> -- did I miss something somewhere?
>>>>
>>>>
>>>> Thanks as always,
>>>>
>>>>
>>>> Tim
>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>> http://paraview.org/Wiki/ParaView
>>>>
>>>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://public.kitware.com/mailman/listinfo/paraview
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20160510/c0a3a538/attachment-0001.html>