FW: [vtkusers] Blending two volumes

[Benjamin King]

Hi John,

I need to do multicomponent volume rendering too and I think that we can 
share our efforts on this. I already modified the existing raycastfunctions 
to get an idea how things are done right now. What I'd like to have in the 
end is a facility that takes multicomponent (let's say n-dimensional) 
samples along a ray and maps them with a transfer function that is defined 
over a n-dimensional domain to RGBA-tuples. These can then be accumulated 
on the way (even early ray termination can be done if the accumulated 
opacity reaches a certain threshold) or the maximal opacity can be found to 
get a MIP. Within this framework, all your blending needs can be fulfilled 
by the tranfer function. For example there can be one that is separable, 
i.e. that finds RGBA-Tuples for each individual component and blends them 
together (like it's done now for scalar and gradient opacity). I would need 
non-separable transfer functions to integrate segmentation information and 
to test ideas from G. Kindlmann et al (http://www.cs.utah.edu/~gk/papers/).

I think it might be a good idea to implement this new mapper in a way that 
closely integrates with ITK because one could make use of the itk::(Vector) 
InterpolateImageFunction's for the volume _and_ the transfer function and 
my multicomponent datasets would be generated with ITK anyway (as perhaps 
do yours if you're interested in registration). Or the mapper could really 
be an itk::ImageFilter<MulticomponentImageType, 
TwoDimensionalRGBAImageType>. In my opinion rendering speed should not be 
the first thing to be concerned about. As Knuth said - 'Early optimization 
is the root of all evil'. (*)

Tell me what you think about it and maybe we can work something out.

Best regards,
  Benjamin

(*) But beware, according to Len Lattanzi, 'belated pessimization is the 
leaf of no good'.


On Tue, 14 Oct 2003 15:01:56 +0100, John Biddiscombe 
<john.biddiscombe at mirada-solutions.com> wrote:

> Fwd'd to vtk users
>
> Lisa,
>
>> Yes - the MIP and composite methods rely on the fact that you
>> can build tables (of at most 65536 entries) to hold the opacity and 
>> color functions. The data values are directly used as an index into 
>> these tables.
>
> Aha! OK then I'll have to abandon the lookup table I guess and map
> scalars through a normal lookup table the hard way. Is there a better
> way of doing it?
>
>> No - the ray caster in VolView 2.0 is not derived from the
>> VTK ray caster.
>
> Does this mean you've just created a new ray cast mapper, or are you not
> using vtk at all? From the comments below it sounds like a new mapper is
> required for proper blending of volumes
>
>> Also, the "blending" of the two channels is done during
>> sampling along the ray, not at the end. If you do it at the end, an 
>> opaque region in one channel would not block stuff behind it in the 
>> other channel, leading to serious depth perception issues.
>
> Understood. For MIP I was specifically asked to blend as a final step.
> It's not clear when you're comparing volumes with different modalities
> which "value" should be higher and therefore appear as the MIP pixel
> value. I have no guidelines on converting for example Standard uptake
> values into an equivalent CT scalar range to make the MIP consistent, so
> blemding the two separate MIPS in the final step seemd the best
> compromise.
>
> For Composite renders than I see exactly what you mean.
>
>> >3) I'd like to be able to move cross hairs etc around on the image
>
>> yes - memory issues. There is no single depth map for volume
>
> Hmm. Yes I confused myself here. Actually, I really only want to move
> cross hairs in an overlay plane, so simply recopying the rendered image
> ought to be fine. Someone else on the list was asking almost the same
> question so putting in a facility to disable the rerender if the volumes
> are not modified and instead recopy the saved buffer might be feasible.
>
>> >4) When rendering in interactive mode with (for example) a
>
>> Interpolating in software we can apply some fancier logic but
>> with a loss of speed.
>
> OK.
>
>
>> >Does anyone have any thoughts on these approaches and advice on how I
>
>> Again, I would not recommend any approach that blends the
>> pixel values after accumulation.  For flexibility you can separate the 
>> sampling from the accumulation, allowing the rays from multiple volumes 
>> to be fed into one accumulator (a bit of a break of the VTK design - 
>> might be tough to implement - but this is what I did in the past in an 
>> old volume visualization system called VolVis which allowed for multiple 
>> overlapping datasets of varying size, dimensions, orientation, etc.)
>
> May I ask how you've apporached the problem in your new renderer? I'm
> thinking that a new mapper is required here if I want good blending (iso
> or composite). The ray casting logic is all fine, but the calls to the
> ray cast functions would need to be changed to handle multiple ray
> blends. One approach would be to fire a ray and build a list of values.
> Fire another ray into the second volume (or just accumulate values from
> a second dataarray component into another value list). Then past the
> filled value lists into the iso/mip/composite function and let it do the
> work of deciding the final pixel value (atleast we can mapscalars all in
> one go for the float mip issue above). This might prevent any
> significant early ray termination though as the whole list would be
> needed each time. Is there any easy mechanism for terminating rays that 
> can be injected in
> a consistent manner? I'm not sure if the existing ray caster terminates
> rays, but it'd be nice to integrate this.
>
> Thanks for your help.
>
> JB
> PS. Can a volumepro card do any kind of clever blending for two volumes
> _______________________________________________
> This is the private VTK discussion list.
> Please keep messages on-topic. Check the FAQ at: 
> <http://public.kitware.com/cgi-bin/vtkfaq>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtkusers
>



-- 
Benjamin King
Institut für Medizinische Informatik
Medizinische Hochschule Hannover
Tel.: +49  511  532-2663