[Insight-users] Re: Tumor Measurements..

[Lino Ramirez]

Hi Luis,

Thank you very much for your comprehensive explanation.

I am designing the experiments to validate a segmentation approach I am
developing. The final goal of this segmentation is to provide clinicians
with a set of measurements they use in their daily practice.

Now, I have some questions about comparing the results obtained by the
expert(s) and the computer method:

1.- Should I pool together the results obtained by the same expert at
different time intervals with the results obtained by different experts
(to get let’s say at least 30 results) and then compare those results with
the computer generated results using for example paired t-tests (or other
type of statistical test)?

2.- Should I keep the inter-observer measurements separated from the
intra-observer measurements and do the individual comparisons of these
results with the computer-generated results?

3.- Should I repeat the comparisons for each parameter I can set up in the
computer segmentation method?

4.- Do you know of any paper in which the authors present a comprehensive
comparison of expert-made measurements with computer-generated
measurements? I want to produce a tool that could be used in clinical
practice and I think that the best way of convincing clinicians of the
validity of the tool is providing a sound statistical analysis.

Thank you very much for all your help.

Have a nice day

Lino


>
> Hi Lino,
>
> When measuring a value there is always variability
> in the outcome of the measurement.
>
> This is due to the fact that the measurement process
> itself is regulated by parameters. Some of those
> parameters are under control and can be regulated,
> some other parameters are not under control, and
> may even be unknown.
>
>
> Let's consider both cases:
>
> 1) Parameters not under control.
>
>    When dealing with parameters of the measurement
>    process that are not under control, the only way to
>    estimate values is to take the measure multiple times
>    and attempt to estimate the statistical distribution
>    of the outcome
>
>    In most cases, Gaussian distributions are assumed,
>    usually for the lack of any other better model.
>
>    When you follow this method, the set of values
>    resulting from your measurements will have a Mean
>    and a Standard Deviation. In a Gaussian distribution
>    you can claim that if you randomly sample the
>    distribution, X% of the samples will be at a distance
>    of K*(Standard Deviations) from the Mean.  The relation
>    ship between X and K is given by the erf() function.
>    (the integral of the Gaussian).
>
>    X% is called the confidence and K*STD define your interval.
>    A typical common confidence value to use is 95% which
>    corresponds to +-3*STD. Meaning that if you integrate the
>    area of the Gaussian from  (Mean - 3*STD) to (Mean+3*STD)
>    you get 95% of the total area under the full Gaussian.
>
>
>    This is the typical process to follow when segmentation
>    is done under human supervision, since it allows to account
>    for the variability of human judgment, both in the same
>    individual over time, and between multiple different
>    individuals.
>
>
>
> 2) Parameters under control.
>
>    For parameters that are under control, the uncertainty of
>    the final measured value is related to the sensitivity of
>    the measurement as response to perturbations of the parameters.
>
>    For a trivial example:
>
>    If you use a thresholding method in order to segment a tumor
>    that has been made visible by Gadolinium enhancement in  an
>    MRI image, the value of the threshold is a critical parameter.
>    Small changes in the value of the threshold will produce
>    variations on the segmentation result and therefore will affect
>    the volume estimation of the tumor.
>
>    From the point of view of Measurement Theory, the preferred
>    value for the threshold should be the one that at which perturbations
>    result in minimal variations of the value to be measured (in this
>    case the tumor volume).
>
>    In order to bound the range of the thresholding value, you must
>    establish what are the limits in which the resulting segmentation
>    is still plausible. For example, let's say that you find that
>    plausible segmentations are produced when the thresholding value
>    is between 150 and 180. That means that at threshold values less
>    than 150 you observe that the segmentation is missing regions that
>    are clearly part of the tumor, or that it is including regions of
>    healthy tissue.
>
>    At this point you can generate a population of threshold values
>    that follow a uniform statistical distribution inside the plausible
>    interval. For each one of those thresholding values the segmentation
>    will result in a specific tumor volume.  If you collect those volumes
>    and compute their Mean and Standard deviation, then you will be
>    able to generate an interval around the mean that correspond to the
>    confidence level that you selected.
>
>
> What is challenging in most segmentation methods is that there are
> many input parameters that may affect the outcome of the measurement.
> Therefore, plausible ranges must be established for each parameters
> and using their collective ranges, a population of segmentation can
> be generated.
>
>
> There are a number of tools that can help you to explore the parameter
> space and estimate the statistical distribution of the measured value.
>
>
>
>     Please let us know if you have further questions.
>
>
>       Thanks
>
>
>          Luis
>
>
> --------------------
> Lino Ramirez wrote:
>
>> Hi Luis,
>>
>> Since you pointed out that
>>
>> ----------------------------------------
>> Do not forget also that when you report measurement
>> they must be accompanied by the uncertainty in the
>> measure and the confidence of this interval.
>>
>> For example,
>> the tumor volume should be reported as:
>>
>>       30 mm^3  +/-  5 mm^3  with  95% confidence.
>> ---------------------------
>>
>> Could you please provide some references on how to compute the
>> uncertainty
>> and confidence interval of the measurements obtained after using a
>> segmentation method?
>>
>> Thank you,
>>
>> Lino
>>
>>
>> _______________________________________________
>> Insight-users mailing list
>> Insight-users at itk.org
>> http://www.itk.org/mailman/listinfo/insight-users
>>
>>
>
>
>
>
>