[vtkusers] Building medical imaging applications from scratch

Andras Lasso lasso at queensu.ca
Sat Jul 1 12:10:36 EDT 2017 

(changed the subject to reflect the slightly shifted topic of the discussion)

Need for a “lightweight” application is a commonly cited reason for starting development of a medical application from scratch (using just VTK+Qt). However, most of the time this is not a well-founded reason.

For example, if you already use Qt and VTK, adding an open-source application framework (3D Slicer, MITK, etc. – if it is not strictly medical application then there are several others, for example ParaView) increases the total amount of third-party libraries (in term of line of source code) by about 5%. This is very little price to pay for the hundreds of man-years of effort that went into development and testing of these frameworks. When you apply for regulatory approval for commercial applications, 3D Slicer is considered software of unknown pedigree, the same way as VTK, and other libraries that you may use. Overall, you get somewhat increased documentation work, but you can save several years of software development effort.

We find that with 3D Slicer you need to write approximately 0.01%-0.1% of application-specific code for a new medical application – see this illustration: https://1drv.ms/b/s!Arm_AFxB9yqHr_hDpop7_CaxXoO4cA.

I know that it is more fun to just start to develop something from scratch, compared to learning from others; it is also useful for understanding details by getting first-hand experience, etc., but when the end result is important – you want to have an application that clinicians can use, try a new method in the operating room, you want to release a product faster, etc. – then it is hard to justify the duplicated effort.

I know that the VTK list is read by many people – VTK developers, users, application developers. I would be interested in hearing about opinion and experiences of others. Not necessarily in the medical imaging domain: for example ParaView developers may see a similar trend of people developing custom scientific visualization applications from scratch, instead of customizing/extending ParaView.

Andras

From: Bill Q [mailto:bill.q.hdp at gmail.com]
Sent: Saturday, July 1, 2017 9:01 AM
To: Andras Lasso <lasso at queensu.ca>
Cc: Elvis Stansvik <elvis.stansvik at orexplore.com>; VTK Users <vtkusers at vtk.org>
Subject: Re: [vtkusers] Issues with coordinate conversion, possible VTK bugs?

Hi Andras,
Thanks a ton!

We are building a software that has a very specific purpose, other than the visualization part, everything else is tailored to the demands. To make it lightweight, we have to build a custom software. That's why we have taken the more difficult approach. Otherwise, we would have chosen an open-source solution.

Thanks again for pointing out the solution.

Many thanks.

Bill

On Sat, Jul 1, 2017 at 8:36 AM, Andras Lasso <lasso at queensu.ca<mailto:lasso at queensu.ca>> wrote:
The highlighted code lines perform point picking in world coordinates. If you need more information then read the entire class and related classes, maybe add a breakpoint and run it step-by-step in a debugger to see how it works exactly.

Playing with these, learning about low-level details is fun and useful experience. However, I would strongly recommend not to develop yet another DICOM viewer from scratch. There are several existing open source medical image computing and visualization applications that are used by thousands of people worldwide, they are used in operating rooms, used as a basis of FDA-approved products, have been around for several years, stable, feature-rich, etc. that can be extended to fit your needs. Most of them use VTK!
Andras

From: Bill Q [mailto:bill.q.hdp at gmail.com<mailto:bill.q.hdp at gmail.com>]
Sent: Thursday, June 29, 2017 8:16 PM
To: Andras Lasso <lasso at queensu.ca<mailto:lasso at queensu.ca>>
Cc: Elvis Stansvik <elvis.stansvik at orexplore.com<mailto:elvis.stansvik at orexplore.com>>; VTK Users <vtkusers at vtk.org<mailto:vtkusers at vtk.org>>

Subject: Re: [vtkusers] Issues with coordinate conversion, possible VTK bugs?

Thanks Andras,

I read through the links, but I wasn't able to identify the part that is relevant to my questions.

Would you be more specific?

Many thanks.
Bill

On Tue, Jun 6, 2017 at 11:01 AM, Andras Lasso <lasso at queensu.ca<mailto:lasso at queensu.ca>> wrote:
See how it is implemented 3D Slicer and MITK:

https://github.com/Slicer/Slicer/blob/master/Libs/MRML/DisplayableManager/vtkThreeDViewInteractorStyle.cxx#L812-L848<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSlicer%2FSlicer%2Fblob%2Fmaster%2FLibs%2FMRML%2FDisplayableManager%2FvtkThreeDViewInteractorStyle.cxx%23L812-L848&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=cIA0AK2NlKNHbrUPGgeTD%2Bo%2FrTfAmm8Fh1cV9iCDvTA%3D&reserved=0>

https://github.com/MITK/MITK/blob/master/Modules/Core/src/Rendering/mitkBaseRenderer.cpp#L600-L640<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMITK%2FMITK%2Fblob%2Fmaster%2FModules%2FCore%2Fsrc%2FRendering%2FmitkBaseRenderer.cpp%23L600-L640&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=CwMad8WlnHrsl%2FCj23TzOskaFtRMmROm3EPxX5iByfo%3D&reserved=0>

Andras

From: vtkusers [mailto:vtkusers-bounces at vtk.org<mailto:vtkusers-bounces at vtk.org>] On Behalf Of Bill Q
Sent: Monday, June 5, 2017 3:56 PM
To: Elvis Stansvik <elvis.stansvik at orexplore.com<mailto:elvis.stansvik at orexplore.com>>
Cc: VTK Users <vtkusers at vtk.org<mailto:vtkusers at vtk.org>>
Subject: Re: [vtkusers] Issues with coordinate conversion, possible VTK bugs?

Hi Elvis,
I did give it a try. But still, it didn't work. I think I must have missed something since there mush have hundreds of people tried to do the same thing, getting the world coordinate from the vtk screen. The only way it worked so far is to use a world picker. All other ways would return different settings with z valued at 88 something. Where is the z value coming from? Does the actor add the z in the renderer?

Would someone know the correct way doing it give some advices?

Many thanks.
Bill

On Sat, May 27, 2017 at 4:06 AM, Elvis Stansvik <elvis.stansvik at orexplore.com<mailto:elvis.stansvik at orexplore.com>> wrote:
2017-05-26 20:08 GMT+02:00 Bill Q <bill.q.hdp at gmail.com<mailto:bill.q.hdp at gmail.com>>:
> Hi Elvis and Dan,
> Thank you very much for the response. After setting the render window
> SetNumberOfLayers(2) and setting the renderer with text SetLayer(1), both
> the layers with the DICOM images and the layer with the text can show up.
> But, the problem still persists. I still cannot get the correct world
> coordinates by using the following function:
>
> def display_to_world2(self, pt):
>     worldPt = [0, 0, 0, 0]
>     vtk.vtkInteractorObserver.ComputeDisplayToWorld(self.renderer_dicom,
> pt[0], pt[1], 0, worldPt)
>     return worldPt[0] / worldPt[3], worldPt[1] / worldPt[3], worldPt[2] /
> worldPt[3]
>
> The z returned is always 86.33.
>
> Meanwhile, I cannot use the world picker either, because it still picks the
> text's world coordinate.
>
> So, either I can make the above function work on the DICOM renderer or I
> have to make the world picker not picking up anything on the text renderer.
> I am not sure how I can get any of these two ways done.
>
> Any suggestions would be greatly appreciated!

Could you try turning interaction off on the renderer you use for your
"foreground" objects. In my code I have:

    auto foregroundRenderer = vtkSmartPointer<vtkRenderer>::New();
    foregroundRenderer->InteractiveOff();
    foregroundRenderer->SetLayer(1);
    ...
    GetRenderWindow()->SetNumberOfLayers(2);

Hope that helps, it's the last idea I have :p

Elvis

>
>
> Many thanks.
>
>
> Bill
>
> On Thu, May 25, 2017 at 12:14 AM, Elvis Stansvik
> <elvis.stansvik at orexplore.com<mailto:elvis.stansvik at orexplore.com>> wrote:
>>
>> Den 24 maj 2017 5:27 em skrev "Bill Q" <bill.q.hdp at gmail.com<mailto:bill.q.hdp at gmail.com>>:
>> >
>> > Hi Dan,
>> > After set the layer, I got this error:
>> >
>> > VTK-7.1.0\Rendering\Core\vtkRendererCollection.cxx, line 61
>> > vtkRendererCollection (000000001192E3A0): Invalid layer for renderer: not
>> > rendered.
>> >
>> > And the text renderer didn't show up.
>>
>> You'll need to call SetNumberOfLayers(2) on your render window first I
>> think.
>>
>> Elvis
>>
>> >
>> > Many thanks.
>> >
>> >
>> > Bill
>> >
>> > On Wed, May 24, 2017 at 11:06 PM, Dan Lipsa <dan.lipsa at kitware.com<mailto:dan.lipsa at kitware.com>>
>> > wrote:
>> >>
>> >> Bill,
>> >> Try using SetLayer(1) for the renderer that has the text.
>> >>
>> >> Dan
>> >>
>> >>
>> >> On Wed, May 24, 2017 at 10:34 AM, Bill Q <bill.q.hdp at gmail.com<mailto:bill.q.hdp at gmail.com>> wrote:
>> >>>
>> >>> Hi Dan,
>> >>> Thanks a ton for the example. The example used 4 different viewports.
>> >>> However, for my application, I need to set all renderers under the same
>> >>> viewport. The problem is that if I add a second renderer into the render
>> >>> window and using the default viewport, it will cover the first renderer and
>> >>> nothing on the first renderer will show up. I am wondering if I did the
>> >>> whole thing wrong.
>> >>>
>> >>> My purpose is to display some DICOM images, some texts, and scales on
>> >>> the same screen. Also, I would like to make the scales zoom with the DICOM
>> >>> images but the texts stay put. Most importantly, I would like to get the
>> >>> world coordinates on the DICOM image when users click on the screen, so I
>> >>> can mark the click properly.
>> >>>
>> >>> What should be done in order to get this working?
>> >>>
>> >>>
>> >>> Many thanks.
>> >>>
>> >>>
>> >>> Bill
>> >>>
>> >>> On Wed, May 24, 2017 at 12:02 AM, Dan Lipsa <dan.lipsa at kitware.com<mailto:dan.lipsa at kitware.com>>
>> >>> wrote:
>> >>>>
>> >>>> Not sure what is going on. Do the objects you are rendering have the
>> >>>> same world coordinates? Are both renderers added to the rendering window?
>> >>>> Here is an example with multiple renderer (they are over different
>> >>>> viewports but they could have the same viewport as well)
>> >>>>
>> >>>> http://www.vtk.org/Wiki/VTK/Examples/Cxx/Visualization/MultipleViewports<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.vtk.org%2FWiki%2FVTK%2FExamples%2FCxx%2FVisualization%2FMultipleViewports&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=T9b3S%2FqHV0Y5FQ6JUnMbUrVUVBidB1PYxLRXsejZNA0%3D&reserved=0>
>> >>>>
>> >>>> On Tue, May 23, 2017 at 11:39 AM, Bill Q <bill.q.hdp at gmail.com<mailto:bill.q.hdp at gmail.com>>
>> >>>> wrote:
>> >>>>>
>> >>>>> I just gave it a try. But when I added another renderer into the
>> >>>>> window that I used for text actors, it overlaps and covered the first
>> >>>>> renderer. So the first renderer won't show up. Is there a solution?
>> >>>>>
>> >>>>> Many thanks.
>> >>>>>
>> >>>>>
>> >>>>> Bill
>> >>>>>
>> >>>>> On Tue, May 23, 2017 at 10:16 PM, Dan Lipsa <dan.lipsa at kitware.com<mailto:dan.lipsa at kitware.com>>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> I think moving those text actors on a different renderer will solve
>> >>>>>> your problem.
>> >>>>>>
>> >>>>>> Dan
>> >>>>>>
>> >>>>>>
>> >>>>>> On Tue, May 23, 2017 at 9:40 AM, Bill Q <bill.q.hdp at gmail.com<mailto:bill.q.hdp at gmail.com>>
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>> Hi Dan,
>> >>>>>>> Thanks a lot for the reply. Yes, that's exactly the case. If I
>> >>>>>>> remove the text actors, the world picker would return the correct
>> >>>>>>> coordinates. So, what method should I use in order to get the correct
>> >>>>>>> coordinates no matter how many actors I put in with the dicom image reader
>> >>>>>>> actor?
>> >>>>>>>
>> >>>>>>> Many thanks.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Bill
>> >>>>>>>
>> >>>>>>> On Mon, May 22, 2017 at 10:04 PM, Dan Lipsa
>> >>>>>>> <dan.lipsa at kitware.com<mailto:dan.lipsa at kitware.com>> wrote:
>> >>>>>>>>
>> >>>>>>>> Bill,
>> >>>>>>>> For display_to_world, the picker picks up only what is visible
>> >>>>>>>> (closes to the camera). You might have several objects (several values in
>> >>>>>>>> world coordinates) that end up over the same display coordinate. This might
>> >>>>>>>> explain why adding those text actors changes what you pick.
>> >>>>>>>>
>> >>>>>>>> Dan
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Sun, May 21, 2017 at 5:25 AM, Bill Q <bill.q.hdp at gmail.com<mailto:bill.q.hdp at gmail.com>>
>> >>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>> Can anybody help? Many thanks.
>> >>>>>>>>>
>> >>>>>>>>> Many thanks.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Bill
>> >>>>>>>>>
>> >>>>>>>>> On Fri, May 19, 2017 at 11:20 PM, Bill Q <bill.q.hdp at gmail.com<mailto:bill.q.hdp at gmail.com>>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Hello Everyone,
>> >>>>>>>>>> I encountered a weird problem in coordinate conversion.
>> >>>>>>>>>>
>> >>>>>>>>>> In order to check the accuracy, I converted from display to
>> >>>>>>>>>> world, and from world back to display. Using the following ways:
>> >>>>>>>>>>
>> >>>>>>>>>> self.world_picker = vtk.vtkWorldPointPicker()
>> >>>>>>>>>>
>> >>>>>>>>>> def display_to_world(self, pt):
>> >>>>>>>>>>     self.world_picker.Pick(pt[0], pt[1], 0, self.ren)
>> >>>>>>>>>>     picker_pt = self.world_picker.GetPickPosition()
>> >>>>>>>>>>     picker_pt = (picker_pt[0], picker_pt[1], 0)
>> >>>>>>>>>>     return picker_pt
>> >>>>>>>>>>
>> >>>>>>>>>> def world_to_display(self, pt):
>> >>>>>>>>>>     if len(pt) == 3:
>> >>>>>>>>>>         pt = pt + (1.0,)
>> >>>>>>>>>>     elif len(pt) == 2:
>> >>>>>>>>>>         pt = pt + (0.0, 1.0)
>> >>>>>>>>>>     self.ren.SetWorldPoint(pt)
>> >>>>>>>>>>     self.ren.WorldToDisplay()
>> >>>>>>>>>>     pt = self.ren.GetDisplayPoint()
>> >>>>>>>>>>     pt = (pt[0], pt[1])
>> >>>>>>>>>>     return pt
>> >>>>>>>>>>
>> >>>>>>>>>> pt = [5, 5]
>> >>>>>>>>>>
>> >>>>>>>>>> world_pt = self. world_to_display(pt)
>> >>>>>>>>>>
>> >>>>>>>>>> display_pt = self. world_to_display(world_pt)
>> >>>>>>>>>>
>> >>>>>>>>>> In some cases, it's quite accurate, which means pt equals
>> >>>>>>>>>> display_pt. But, in some cases, the result is simply wrong. For example, for
>> >>>>>>>>>> (5, 5), after conversions back to display, the coordinate becomes (20.89,
>> >>>>>>>>>> 20.89).
>> >>>>>>>>>>
>> >>>>>>>>>> Any ideas?
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> Many thanks.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> Bill
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> _______________________________________________
>> >>>>>>>>> Powered by www.kitware.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.kitware.com&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=C%2FCjHwZ0BUq%2F2s2V21jZc68m3nmuiyC8%2Fjr9gmZAZc4%3D&reserved=0>
>> >>>>>>>>>
>> >>>>>>>>> Visit other Kitware open-source projects at
>> >>>>>>>>> http://www.kitware.com/opensource/opensource.html<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.kitware.com%2Fopensource%2Fopensource.html&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=s4xbyO4EWFK%2BEs2WCwkDKxi5IXO%2F0jBXQ%2FyLG3piftE%3D&reserved=0>
>> >>>>>>>>>
>> >>>>>>>>> Please keep messages on-topic and check the VTK FAQ at:
>> >>>>>>>>> http://www.vtk.org/Wiki/VTK_FAQ<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.vtk.org%2FWiki%2FVTK_FAQ&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=Ls6%2B%2BnosA8Lys3aXITQWlsJf%2Bc404mrGD4%2B29wziWyU%3D&reserved=0>
>> >>>>>>>>>
>> >>>>>>>>> Search the list archives at:
>> >>>>>>>>> http://markmail.org/search/?q=vtkusers<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmarkmail.org%2Fsearch%2F%3Fq%3Dvtkusers&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=qyF%2F1xQRxFzzzvr%2BG7ggSQXXcRWvkatbPlPTcQRgF3U%3D&reserved=0>
>> >>>>>>>>>
>> >>>>>>>>> Follow this link to subscribe/unsubscribe:
>> >>>>>>>>> http://public.kitware.com/mailman/listinfo/vtkusers<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpublic.kitware.com%2Fmailman%2Flistinfo%2Fvtkusers&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=RPE1rFrl5%2BtmcNsgHyfS4zAL5frL5uYVbBsEWQARTKo%3D&reserved=0>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
>> >
>> > _______________________________________________
>> > Powered by www.kitware.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.kitware.com&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=C%2FCjHwZ0BUq%2F2s2V21jZc68m3nmuiyC8%2Fjr9gmZAZc4%3D&reserved=0>
>> >
>> > Visit other Kitware open-source projects at
>> > http://www.kitware.com/opensource/opensource.html<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.kitware.com%2Fopensource%2Fopensource.html&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=s4xbyO4EWFK%2BEs2WCwkDKxi5IXO%2F0jBXQ%2FyLG3piftE%3D&reserved=0>
>> >
>> > Please keep messages on-topic and check the VTK FAQ at:
>> > http://www.vtk.org/Wiki/VTK_FAQ<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.vtk.org%2FWiki%2FVTK_FAQ&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=Ls6%2B%2BnosA8Lys3aXITQWlsJf%2Bc404mrGD4%2B29wziWyU%3D&reserved=0>
>> >
>> > Search the list archives at: http://markmail.org/search/?q=vtkusers<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmarkmail.org%2Fsearch%2F%3Fq%3Dvtkusers&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=qyF%2F1xQRxFzzzvr%2BG7ggSQXXcRWvkatbPlPTcQRgF3U%3D&reserved=0>
>> >
>> > Follow this link to subscribe/unsubscribe:
>> > http://public.kitware.com/mailman/listinfo/vtkusers<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpublic.kitware.com%2Fmailman%2Flistinfo%2Fvtkusers&data=02%7C01%7Classo%40queensu.ca%7C041978750cdf4cd4227d08d4bf1aef23%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C636343569784881914&sdata=RPE1rFrl5%2BtmcNsgHyfS4zAL5frL5uYVbBsEWQARTKo%3D&reserved=0>
>> >
>
>



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtkusers/attachments/20170701/eae65e3e/attachment.html>

More information about the vtkusers mailing list