FW: [vtkusers] Blending two volumes

Wed Oct 15 10:07:16 EDT 2003

Benjamin,

Further to what you've written I'm inclined to think that the overall design and utility of itk is so nice that we should branch vtk into an itk compatible variety. There hardly seems any point in have two sets of process objects (by which I mean two pipeline systems), two sets of dataobjects etc, from my point of view, the one thing that vtk has that's not directly echoed (apart from many of the filters of course) in itk is field data - or rather the multi-component add/remove a field data array at will type of flexibility. Mappers that sat directly on top of itk might be quite elegant.

With regard to the volume rendering issues, I've got two major obstacles to overcome - firstly my employer doesn't use itk - we have a large library of registration and segmentation code that's home grown, and try as I might to persuade others to use itk, so far I've been unsucessful. Secondly, I'm going to have to speak to people about doing any large scale open source development in vtk. I haven't been very successful in persuading people to adopt vtk either, I am being encouraged to adapt the in-house mip renderer to the dual channel job as it'll be a lot less work that what you and I have in mind.

I'll get back to you on this one (probably off list)

JB

> -----Original Message-----
> From: Benjamin King [mailto:king.benjamin at mh-hannover.de] 
> Sent: 15 October 2003 10:16
> To: John Biddiscombe; vtkusers at vtk.org
> Subject: Re: FW: [vtkusers] Blending two volumes
> 
> 
> Hi John,
> 
> I need to do multicomponent volume rendering too and I think 
> that we can 
> share our efforts on this. I already modified the existing 
> raycastfunctions 
> to get an idea how things are done right now. What I'd like 
> to have in the 
> end is a facility that takes multicomponent (let's say n-dimensional) 
> samples along a ray and maps them with a transfer function 
> that is defined 
> over a n-dimensional domain to RGBA-tuples. These can then be 
> accumulated 
> on the way (even early ray termination can be done if the accumulated 
> opacity reaches a certain threshold) or the maximal opacity 
> can be found to 
> get a MIP. Within this framework, all your blending needs can 
> be fulfilled 
> by the tranfer function. For example there can be one that is 
> separable, 
> i.e. that finds RGBA-Tuples for each individual component and 
> blends them 
> together (like it's done now for scalar and gradient 
> opacity). I would need 
> non-separable transfer functions to integrate segmentation 
> information and 
> to test ideas from G. Kindlmann et al 
> (http://www.cs.utah.edu/~gk/papers/).
> 
> I think it might be a good idea to implement this new mapper 
> in a way that 
> closely integrates with ITK because one could make use of the 
> itk::(Vector) 
> InterpolateImageFunction's for the volume _and_ the transfer 
> function and 
> my multicomponent datasets would be generated with ITK anyway 
> (as perhaps 
> do yours if you're interested in registration). Or the mapper 
> could really 
> be an itk::ImageFilter<MulticomponentImageType, 
> TwoDimensionalRGBAImageType>. In my opinion rendering speed 
> should not 
> TwoDimensionalRGBAImageType>be
> the first thing to be concerned about. As Knuth said - 'Early 
> optimization 
> is the root of all evil'. (*)
> 
> Tell me what you think about it and maybe we can work something out.
> 
> Best regards,
>   Benjamin
> 
> (*) But beware, according to Len Lattanzi, 'belated 
> pessimization is the 
> leaf of no good'.
> 
> 
> On Tue, 14 Oct 2003 15:01:56 +0100, John Biddiscombe 
> <john.biddiscombe at mirada-solutions.com> wrote:
> 
> > Fwd'd to vtk users
> >
> > Lisa,
> >
> >> Yes - the MIP and composite methods rely on the fact that you can 
> >> build tables (of at most 65536 entries) to hold the 
> opacity and color 
> >> functions. The data values are directly used as an index 
> into these 
> >> tables.
> >
> > Aha! OK then I'll have to abandon the lookup table I guess and map 
> > scalars through a normal lookup table the hard way. Is 
> there a better 
> > way of doing it?
> >
> >> No - the ray caster in VolView 2.0 is not derived from the VTK ray 
> >> caster.
> >
> > Does this mean you've just created a new ray cast mapper, 
> or are you 
> > not using vtk at all? From the comments below it sounds like a new 
> > mapper is required for proper blending of volumes
> >
> >> Also, the "blending" of the two channels is done during sampling 
> >> along the ray, not at the end. If you do it at the end, an opaque 
> >> region in one channel would not block stuff behind it in the other 
> >> channel, leading to serious depth perception issues.
> >
> > Understood. For MIP I was specifically asked to blend as a 
> final step. 
> > It's not clear when you're comparing volumes with different 
> modalities 
> > which "value" should be higher and therefore appear as the 
> MIP pixel 
> > value. I have no guidelines on converting for example 
> Standard uptake 
> > values into an equivalent CT scalar range to make the MIP 
> consistent, 
> > so blemding the two separate MIPS in the final step seemd the best 
> > compromise.
> >
> > For Composite renders than I see exactly what you mean.
> >
> >> >3) I'd like to be able to move cross hairs etc around on the image
> >
> >> yes - memory issues. There is no single depth map for volume
> >
> > Hmm. Yes I confused myself here. Actually, I really only 
> want to move 
> > cross hairs in an overlay plane, so simply recopying the rendered 
> > image ought to be fine. Someone else on the list was asking 
> almost the 
> > same question so putting in a facility to disable the 
> rerender if the 
> > volumes are not modified and instead recopy the saved 
> buffer might be 
> > feasible.
> >
> >> >4) When rendering in interactive mode with (for example) a
> >
> >> Interpolating in software we can apply some fancier logic 
> but with a 
> >> loss of speed.
> >
> > OK.
> >
> >
> >> >Does anyone have any thoughts on these approaches and 
> advice on how 
> >> >I
> >
> >> Again, I would not recommend any approach that blends the pixel 
> >> values after accumulation.  For flexibility you can separate the 
> >> sampling from the accumulation, allowing the rays from multiple 
> >> volumes to be fed into one accumulator (a bit of a break 
> of the VTK 
> >> design - might be tough to implement - but this is what I 
> did in the 
> >> past in an old volume visualization system called VolVis which 
> >> allowed for multiple overlapping datasets of varying size, 
> >> dimensions, orientation, etc.)
> >
> > May I ask how you've apporached the problem in your new 
> renderer? I'm 
> > thinking that a new mapper is required here if I want good blending 
> > (iso or composite). The ray casting logic is all fine, but 
> the calls 
> > to the ray cast functions would need to be changed to 
> handle multiple 
> > ray blends. One approach would be to fire a ray and build a list of 
> > values. Fire another ray into the second volume (or just accumulate 
> > values from a second dataarray component into another value list). 
> > Then past the filled value lists into the iso/mip/composite 
> function 
> > and let it do the work of deciding the final pixel value 
> (atleast we 
> > can mapscalars all in one go for the float mip issue above). This 
> > might prevent any significant early ray termination though as the 
> > whole list would be needed each time. Is there any easy 
> mechanism for 
> > terminating rays that can be injected in a consistent 
> manner? I'm not 
> > sure if the existing ray caster terminates rays, but it'd 
> be nice to 
> > integrate this.
> >
> > Thanks for your help.
> >
> > JB
> > PS. Can a volumepro card do any kind of clever blending for two 
> > volumes _______________________________________________
> > This is the private VTK discussion list.
> > Please keep messages on-topic. Check the FAQ at:
> > <http://public.kitware.com/cgi-bin/vtkfaq>
> > Follow this link to subscribe/unsubscribe:
> > http://www.vtk.org/mailman/listinfo/vtkusers
> >
> 
> 
> 
> -- 
> Benjamin King
> Institut für Medizinische Informatik
> Medizinische Hochschule Hannover
> Tel.: +49  511  532-2663
> 