[Paraview-developers] [Paraview] (Ordered) Compositing question.

Wed Nov 21 10:03:10 EST 2012

Good luck :). BTW, indeed, there's scope for providing improved
interface for SetOrderedCompositingInformation(). I implemented for
the use-cases I had at hand. Just the fact that's it's now easier to
add alternative interfaces for the same is a step in the right
direction (the same would have been very tricky with 3.14). For the
redistribution bug, you must be calling
vtkPVRenderView::MarkAsRedistributable() somewhere in your
representation. Unless a representation says it's data is
"redistributable", the vtkPVDataDeliveryManager won't attempt to
redistribute it. It maybe being set in your superclass. I realized I
didn't add any API to unmark as redistributable. Feel free to add that
(and push a patch back).

Utkarsh

On Wed, Nov 21, 2012 at 5:57 AM, Biddiscombe, John A. <biddisco at cscs.ch> wrote:
> 1)      Solved by setting
>
>
> redistributor->SetPassThrough(item.OrderedCompositingInfo.Translator!=NULL);
>
> So we pass through the data if the translator has been set. This fixes the
> blank regions which must have been caused by the D3 filter simply dropping
> cells in the regions which were shrunk, not by IceT as I guessed previously.
>
>
>
> Now the redistribution takes place, but the pass through means the time is
> 0.01s. I’m happy with that. Using the shrunken bounding boxes solves the Kd
> tree build and if the compositor doesn’t know about the blank regions, who
> cares.
>
>
>
> Second problem of redistribution looks like it was the LOD render. Forcing
> LOD to use outline seems to have solved it. Probably the LOD pipeline is
> using its own DataRedistribution which was triggering the D3 separately. I
> only overrode the normal ‘still render’ pipeline, so I will check to see if
> overriding the LOD pipeline too solves it. (they should certainly share the
> redistributed geometry shouldn’t they?)
>
>
>
> Thanks Utkarsh for putting me on the right track. Fingers crossed I don’t
> find any more problems. First big test here did a full opacity rendering on
> all 12 cores for 10million triangles about 7s which seems ok for my current
> needs. Now I’ll try with the big data on the cluster J (closer to a billion
> triangles I think!)
>
>
>
> JB
>
>
>
> From: Biddiscombe, John A.
> Sent: 21 November 2012 09:38
> To: Biddiscombe, John A.; Utkarsh Ayachit
> Cc: paraview-developers at paraview.org; paraview at paraview.org
> Subject: RE: [Paraview] [Paraview-developers] (Ordered) Compositing
> question.
>
>
>
> Doesn’t seem to be working as expected yet. I have found some problems which
> I’ll mention here for general reference.
>
>
>
> 1)      Setting the ExtentTranslator using the function
>
> vtkPVRenderView::SetOrderedCompositingInformation(inInfo, this, input_bet,
> whole_extent, origin, spacing); doesn’t stop the Redistribution from taking
> place. I’m still debugging this, but perhaps the pipeline somehow knows that
> this is not a structured dataset and ignores the information during one
> phases of the DataDelivery pipeline. (NB. input_bet is bounds extent
> translator – I’m using my vtkBoundsExtentTranslator but it inherits from the
> standard ExtentTranslator)
>
>
>
> 2)      I know that 1) isn’t quite true because I can see that the KdTree is
> being constructed from the extents I supply for each bounding region. There
> is a nasty bug in here which is that for an unstructured grid, boundary
> cells are uniquely assigned to each process and this means that the bounding
> boxes will  slightly overlap as a cell on one process may overlap bounds
> with another process. This causes the KdTree construction to abort with an
> error. I can fix this by shrinking the bounds (extents) I pass to the KdTree
> but this in turn causes some parts of the screen to be not rendered. I
> haven’t quite solved what’s going on here. By screen I really mean data,
> there are empty regions between the data corresponding to the shrunken
> extents I’ve used. Seems very odd because the Compositor only needs to know
> which order to composite. Suspect that IceT is using the bounds to generate
> regions internally to composite and blanking out the bits it thinks are
> ‘invisible’. Need to find a hack to fix this (IceT option anywhere?). Even
> though the extents are being used, I don’t yet understand why the
> redistribution is taking place anyway.
>
> 3)      Lock Interactive render option does not work, (I’m using 3.98 from
> last week), used to be able to interact and change rotation several times
> then after 2 seconds it would do a still render. Not any more. (I’m using 12
> core Windows machine for my  debuging).
>
> 4)      Changing any options in window setting causes entire rendering
> pipeline to reexecute. D3 filter takes 80 seconds to redistribute my not
> very big dataset, if I open up settings and then click “disable splash
> screen” or change “lock interactive render” value then refresh the view,
> Data redistribution is done again thus causing a very long wait and paraview
> is basically unusable (for larger datasets) if you also consider the non
> working interactive/still delay feature. No real need to invalidate the
> entire view when something like this changes.
>
> 5)      Error messages are not displayed on windows when using client +
> pvservers. Any vtkErrorMacro(“blah”) causes the app to hang, you can find
> out what the error is by attaching the debugger and viewing the stack trace
> and inspecting the char* passed to the output text.
>
> 6)      App hangs on connection to pvserver if you –use-offscreen-rendering
> on a windows machine. This is strange because the error is actually “Stereo
> window must be set before creation….”, but I didn’t actually ask for a
> stereo window. Fix is to just set SetStereoCapableWindow to 0. Hang is
> caused by 5) error message causing lockup when using pvservers.
>
> 7)      Changing opacity causes immediate re-render. Any way of only
> re-rendering when we’ve finished changing settings?
>
>
>
> I’m working on 1). There is definitely scope for modifying the interface to
> the SetOrderedCompositingInformation, I have to turn my coordinate bounding
> boxes into extents (using some fictitious image resolution) , pass them in,
> where they get used inversely to compute coordinate bounds. This would be ok
> to fix if the KdTree wasn’t so fussy about overlapping bounds. This is a
> real problem. Is there any other way of giving a compositing order – I will
> work on making the tree more robust with my bounds. The
> BoundsExtentTranslator should be added to paraview proper once I finish.
> There are a ton of places we can optimize this DataDelivery phase for non
> structured datasets using actual bounds. I will work on this.
>
>
>
> An help on any of the above gratefully received.
>
>
>
> Thanks
>
>
>
> JB
>
>
>
>
>
> From: paraview-bounces at paraview.org [mailto:paraview-bounces at paraview.org]
> On Behalf Of Biddiscombe, John A.
> Sent: 20 November 2012 14:10
> To: Utkarsh Ayachit
> Cc: paraview-developers at paraview.org; paraview at paraview.org
> Subject: Re: [Paraview] [Paraview-developers] (Ordered) Compositing
> question.
>
>
>
> Utkarsh,
>
>
>
> Depth sort painter representation working ok …. But
>
>
>
> For my first simple implementation I’ll use a D3 filter inside my other
> source so data is partitioned using the standard filters - and I can access
> the BSP cuts etc. Can I pass these cuts down the pipeline as some kind of
> attached information object and then in the ProcessViewRequest of the
> representation get them and pass them into the compositor. Using the D3 Cuts
> might be easier than my Zoltan based bounding boxes and for version 1,
> this’ll do.
>
>
>
> I’m looking at the code now, and can see that vtkOrderedCompositingInfo
> expects wholeextents and other stuff about spacing etc. I suppose I should
> simply invent these numbers based on my whole bounding box and the relative
> offsets of the individual boxes. Seems a shame to go from the BSP cuts to
> extents, presumably so that the compositor can then do the reverse. Is this
> where you said the interface can be improved? (meaning I can pass the bounds
> directly and bypass the structured-extents …)
>
>
>
> Ta
>
>
>
> JB
>
>
>
>
>
>
>
>
>
>
>
> From: Utkarsh Ayachit [mailto:utkarsh.ayachit at kitware.com]
> Sent: 19 November 2012 16:57
> To: Biddiscombe, John A.
> Cc: Moreland, Kenneth; paraview-developers at paraview.org;
> paraview at paraview.org
> Subject: Re: [Paraview] [Paraview-developers] (Ordered) Compositing
> question.
>
>
>
> John,
>
>
>
> Look at vtkStructuredGridVolumeRepresentation, It passes a
> "ExtentTranslator" to the view. The view uses the extent translator to build
> the KdTree. That's one of specifying the cuts. You can indeed provide a
> custom extent translator or if need be, that API can be extended to pass
> some other data-structure to extract the relevant information for ordered
> compositing.
>
>
>
> For using opacity with polygonal data, simply go to the lookup table editor
> dialog and (in the advanced mode) you'll see a checkbox to "Enable Opacity
> Function". Check that and you can specify the opacity function.
>
>
>
> Utkarsh
>
>
>
> On Mon, Nov 19, 2012 at 10:49 AM, Biddiscombe, John A. <biddisco at cscs.ch>
> wrote:
>
> Thanks Ken,
>
>
>
> My suspicions are confirmed. I had a poke around with the ordered
> compositing code (vtkOrderedCompositeDistributor et al) and saw that the BSP
> cuts from the D3 filter are set to match those from the PKDTree internally.
> I notice that in the BSPcuts code it says
>
>   // Description:
>
>   //   Initialize the cuts with arrays of information.  This type of
>
>   //   information would be obtained from a graph partitioning software
>
>   //   package like Zoltan.
>
>
>
> Do you know if there exists such an example. I am infact partitioning the
> data using Zoltan and have the exact bounds etc of the partitions, so I can
> work out what to pass into the BSPcuts  class and could potentially fool the
> IceT composiining into using my partitions. Can you point me to any crucial
> function that I must call. Is it sufficient to do a SetCuts on the
> compositing code – which class do I need to take control of?).
>
>
>
> I shall create a custom representation to handle the depth sort. I’ve done
> the same some time ago for the point sprite renderer so I’ll manage that ok,
> but I’ll need to override the IceT compositing pass too as alluded to above
> , so any pointers welcome. (Could I somehow subclass on of the painters for
> the IceT pass and set the cuts to what I’ve got, then proceed as usual?)
>
>
>
> Thanks
>
>
>
> JB
>
>
>
> From: Moreland, Kenneth [mailto:kmorel at sandia.gov]
> Sent: 19 November 2012 16:13
> To: Biddiscombe, John A.
> Cc: paraview-developers at paraview.org; paraview at paraview.org
> Subject: Re: [Paraview-developers] (Ordered) Compositing question.
>
>
>
> You are correct in that if you disable ordered composite it will disabled
> the distribution and the composite ordering. The problem is that the
> geometry distribution also creates a k-d tree structure that the compositing
> uses to determine the visibility order. So it would not be sufficient to
> simply have geometry that is in non-overlapping pieces. You would also have
> to provide some meta-structure that could be used to determine an
> appropriate ordering given a viewpoint. Even if you had that, there is no
> mechanism to feed that to the compositor in the view. I can't think of any
> way of doing that that would not be a huge hack.
>
>
>
> As far as I know, there is no support for the DepthSortPolyData filter. You
> would probably have to create a custom representation to support that.
>
>
>
> I believe the transparent lookup table for poly data is supported in 3.98,
> but to be honest I have not yet tried it.
>
>
>
> -Ken
>
>
>
> Sent from my iPad so blame autocorrect.
>
>
> On Nov 19, 2012, at 3:46 AM, "Biddiscombe, John A." <biddisco at cscs.ch>
> wrote:
>
> I have geometry distributed into parallel pieces such that they are non
> overlapping and therefore compositing should only need to use a back to
> front sort last frame buffer operation with no redistribution of geometry.
>
>
>
> I’m worried that when I turn on transparency, paraview will try to
> redistribute my data, even though I have already partitioned it nicely for
> this. I’d like to use ordered sort last compositing at the top level, and
> use DepthSortPolyData on each node (no depth peeling for now) to ensure I
> get a true image. What isn’t clear to me is if I “disable ordered
> compositing” if this disables the transfer of geometry AND also disables the
> ordered compositing necessary for correct transparent blending (I may be
> confused here about the blend modes used or the term “ordered compositing”
> as used in the docs).
>
>
>
> Is there written down anywhere a complete description of the pipelines used
> by the various rendering modes so that I can be sure the pipeline is doing
> what I want. I can create a custom representation which does what I want,
> but it’ll take time and if the existing mechanism support it, I shouldn’t
> need to.
>
>
>
> Also, I specifically want to use a transparent lookuptable, does the newer
> paraview 3.98 have anything to support this. I thought I’d seen a thread
> about this recently, but cannot find it now.
>
>
>
> In summary : For the rendering, I’d like Ordered sort-last compositing, no
> geometry redistribution, Depth sort of polygons on each node. Transparent
> lookup table.
>
>
>
> What settings should I use J
>
>
>
> Thanks
>
>
>
> JB
>
>
>
> --
>
> John Biddiscombe,                        email:biddisco @.at.@ cscs.ch
>
> http://www.cscs.ch/
>
> CSCS, Swiss National Supercomputing Centre  | Tel:  +41 (91) 610.82.07
>
> Via Trevano 131, 6900 Lugano, Switzerland   | Fax:  +41 (91) 610.82.82
>
>
>
> _______________________________________________
> Paraview-developers mailing list
> Paraview-developers at paraview.org
> http://public.kitware.com/mailman/listinfo/paraview-developers
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://www.paraview.org/mailman/listinfo/paraview
>
>