[Paraview] Volume Rendering 17GB 8.5 billion cell volume

Wed Oct 21 09:57:13 EDT 2015

> No GL error this time I am assuming? Just segfault?

Yes. With the stack totally hosed. I also see Valgrind errors deep in Mesa
stack.

On Wed, Oct 21, 2015 at 9:15 AM, Aashish Chaudhary <
aashish.chaudhary at kitware.com> wrote:

> On Wed, Oct 21, 2015 at 7:52 AM, Berk Geveci <berk.geveci at kitware.com>
> wrote:
>
>> Yeah, it is not that easy. If you compile Mesa with llvmpipe, the limit
>> is in something like gallium/drivers/llvmpipe/lp_limits.h. Otherwise it
>> doesn't work. However, I had crashes when I increased that to something
>> like 8 GBs and then volume rendered something large.
>>
>
> No GL error this time I am assuming? Just segfault?
>
>
>> It also looks like other drivers have their own limits that are pretty
>> small. I will talk to some folks doing work with Mesa about this.
>> Hopefully, we can address it in the upcoming OpenSWR driver. We'll have to
>> support streaming for other drivers though...
>>
>
> Absolutely.
>
>>
>> -berk
>>
>>
>> On Tue, Oct 20, 2015 at 5:55 PM, Aashish Chaudhary <
>> aashish.chaudhary at kitware.com> wrote:
>>
>>> Here it is. It would be great if someone else try it as well:
>>>
>>> ----Steps----
>>> 1. In src/mesa/main/config.h, there should MAX_TEXTURE_MBYTES defined. I
>>> believe by default it was 1024 Mbytes. Please change it to 4096 or
>>> something higher.
>>>
>>> 2.  Then Compile and Install MESA again (do not forget to set the
>>> MESA_GL_VERSION_OVERRIDE to 3.2).
>>>
>>> 3. Compile paraview again (server)
>>>
>>>
>>> On Tue, Oct 20, 2015 at 5:48 PM, Aashish Chaudhary <
>>> aashish.chaudhary at kitware.com> wrote:
>>>
>>>> Berk,
>>>>
>>>> On Tue, Oct 20, 2015 at 4:00 PM, Berk Geveci <berk.geveci at kitware.com>
>>>> wrote:
>>>>
>>>>> Hi folks,
>>>>>
>>>>> I wanted to close the loop on this. Here are my findings:
>>>>>
>>>>> * ParaView master (4.4 should also do) + OpenGL2 + NVIDIA Tesla w 12
>>>>> GB memory: I verified that I can volume render data up to the capacity of
>>>>> the card. I could volume render a 1400x1400x1400 volume of floats.
>>>>>
>>>>> * ParaView master (4.4 should also do) + OpenGL2 + Mesa (OSMesa 11,
>>>>> llvmpipe, swrast): Mesa has some fairly small limits on 3D texture size,
>>>>> which is what we use for volume rendering. So, ~ 1000x1000x1000 will be the
>>>>> upper end of what can be done for now. In time, we will implement multiple
>>>>> textures / streaming to enable rendering of larger volumes.
>>>>>
>>>>
>>>> Did you see my other email? You can change the default for OSMesa. I
>>>> sent it last week.
>>>>
>>>> - Aashish
>>>>
>>>>>
>>>>> Best,
>>>>> -berk
>>>>>
>>>>> On Mon, Sep 28, 2015 at 11:00 AM, David Trudgian <
>>>>> David.Trudgian at utsouthwestern.edu> wrote:
>>>>>
>>>>>> Berk,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks very much for looking into this. Look forward to trying things
>>>>>> out whenever they’re ready.
>>>>>>
>>>>>>
>>>>>>
>>>>>> DT
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> David Trudgian Ph.D.
>>>>>> Computational Scientist, BioHPC
>>>>>> UT Southwestern Medical Center
>>>>>> Dallas, TX 75390-9039
>>>>>> Tel: (214) 648-4833
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* Berk Geveci [mailto:berk.geveci at kitware.com]
>>>>>> *Sent:* Monday, September 28, 2015 9:58 AM
>>>>>>
>>>>>> *To:* David Trudgian <David.Trudgian at UTSouthwestern.edu>
>>>>>> *Cc:* ParaView Mailing List <paraview at paraview.org>
>>>>>> *Subject:* Re: [Paraview] Volume Rendering 17GB 8.5 billion cell
>>>>>> volume
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi David,
>>>>>>
>>>>>>
>>>>>>
>>>>>> I have been trying to find some cycles to check this out myself with
>>>>>> ParaView 4.4. Thanks to hardware issues (i.e. my big workstation's disk
>>>>>> dying), I haven't been able to. Good news is that I found issues with
>>>>>> OSMesa + OpenGL2 that we are working through. Give me another 1-1.5 weeks.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Best,
>>>>>>
>>>>>> -berk
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Sep 28, 2015 at 10:46 AM, David Trudgian <
>>>>>> David.Trudgian at utsouthwestern.edu> wrote:
>>>>>>
>>>>>> Hi Berk,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Finally managed to grab an allocation of some Tesla K40 nodes on our
>>>>>> cluster, to check GPU rendering of the full 17GB file with 2 x 12GB GPUs. I
>>>>>> see the same thing as I did with OSMesa rendering.
>>>>>>
>>>>>>
>>>>>>
>>>>>> The 9GB downsampled version works great, across 2 nodes both with a
>>>>>> single K40. Go up to the 17GB original file and nothing is rendered, no
>>>>>> errors. Same behavior with OPENGL or OPENGL2 backends.
>>>>>>
>>>>>>
>>>>>>
>>>>>> This is all on paraview 4.3.1 still – I need to find time to build
>>>>>> OSMesa / MPI versions of 4.4 here. But, does 4.4. have any fixes that would
>>>>>> be expected to affect this?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> David Trudgian Ph.D.
>>>>>> Computational Scientist, BioHPC
>>>>>> UT Southwestern Medical Center
>>>>>> Dallas, TX 75390-9039
>>>>>> Tel: (214) 648-4833
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* Berk Geveci [mailto:berk.geveci at kitware.com]
>>>>>> *Sent:* Tuesday, September 15, 2015 2:43 PM
>>>>>> *To:* David Trudgian <David.Trudgian at UTSouthwestern.edu>
>>>>>> *Cc:* ParaView Mailing List <paraview at paraview.org>
>>>>>> *Subject:* Re: [Paraview] Volume Rendering 17GB 8.5 billion cell
>>>>>> volume
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hey David,
>>>>>>
>>>>>>
>>>>>>
>>>>>> I am hoping to have some time to play around with volume rendering
>>>>>> and hopefully tracking this issue, one thing that I wanted to clarify: it
>>>>>> sounds from you description that you have a short (2 byte) value. Is that
>>>>>> correct?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> -berk
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 9, 2015 at 5:00 PM, David Trudgian <
>>>>>> david.trudgian at utsouthwestern.edu> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We have been experimenting with using Paraview to display very
>>>>>> volumes from very
>>>>>> large TIFF stacks generated by whole-brain microscopy equipment. The
>>>>>> test stack
>>>>>> has dimensions of 5,368x10,695x150. Stack is assembled in ImageJ from
>>>>>> individual
>>>>>> TIFFs, exported as a RAW and loaded into paraview. Saved as a .vti for
>>>>>> convenience. Can view slices fine in standalone paraview client on a
>>>>>> 256GB machine.
>>>>>>
>>>>>> When we attempt volume rendering on this data across multiple nodes
>>>>>> with MPI
>>>>>> nothing appears in the client. Surface view works as expected. On
>>>>>> switching to
>>>>>> volume rendering the client's display will show nothing. There are no
>>>>>> messages
>>>>>> from the client or servers - no output.
>>>>>>
>>>>>> This is happening when running pvserver across GPU nodes with NVIDIA
>>>>>> Tesla
>>>>>> cards, or using CPU only with OSMESA. pvserver memory usage is well
>>>>>> below what
>>>>>> we have on the nodes - no memory warnings/errors.
>>>>>>
>>>>>> Data is about 17GB, 8 billion cells. If we downsize to ~4GB or ~9GB
>>>>>> then we can
>>>>>> get working volume rendering. The 17GB never works regardless of
>>>>>> scaling
>>>>>> nodes/mpi processes. The 4/9GB will work on 1 or 2 nodes.
>>>>>>
>>>>>> Am confused by the lack of rendering, as we don't have memory issues,
>>>>>> or an
>>>>>> other messages at all. Am wondering if there are any inherent
>>>>>> limitation, or I'm
>>>>>> missing something stupid.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Dave Trudgian
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Powered by www.kitware.com
>>>>>>
>>>>>> Visit other Kitware open-source projects at
>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>
>>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>>> http://paraview.org/Wiki/ParaView
>>>>>>
>>>>>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>>>>>
>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>> http://public.kitware.com/mailman/listinfo/paraview
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> *UT** Southwestern*
>>>>>>
>>>>>> Medical Center
>>>>>>
>>>>>> The future of medicine, today.
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Powered by www.kitware.com
>>>>>
>>>>> Visit other Kitware open-source projects at
>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>
>>>>> Please keep messages on-topic and check the ParaView Wiki at:
>>>>> http://paraview.org/Wiki/ParaView
>>>>>
>>>>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>>>>
>>>>> Follow this link to subscribe/unsubscribe:
>>>>> http://public.kitware.com/mailman/listinfo/paraview
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>> *| Aashish Chaudhary | Technical Leader         | Kitware Inc.
>>>>   *
>>>> *| http://www.kitware.com/company/team/chaudhary.html
>>>> <http://www.kitware.com/company/team/chaudhary.html>*
>>>>
>>>
>>>
>>>
>>> --
>>>
>>>
>>>
>>> *| Aashish Chaudhary | Technical Leader         | Kitware Inc.
>>>   *
>>> *| http://www.kitware.com/company/team/chaudhary.html
>>> <http://www.kitware.com/company/team/chaudhary.html>*
>>>
>>
>>
>
>
> --
>
>
>
> *| Aashish Chaudhary | Technical Leader         | Kitware Inc.            *
> *| http://www.kitware.com/company/team/chaudhary.html
> <http://www.kitware.com/company/team/chaudhary.html>*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20151021/303c0f36/attachment.html>