<html>
<body>
<font size=3>Hi Jim,<br><br>
You observed well. I think this is the expected behaviour. I do not think
the truncation issues are the cause.<br><br>
M = the number of pixels in the image<br>
N = the number of loops over the image (200 in the example)<br>
R = The number of pixels selected by the RandomIterator (= 100M in this
example)<br><br>
image2:<br>
after one loop through the image:<br>
<x-tab> </x-tab>P( pixel_i
= 0x sampled ) = 1/2 for i = 1...M<br>
<x-tab> </x-tab>P( pixel_i
= 1x sampled ) = 1/2<x-tab> </x-tab>for i = 1...M<br>
after N loops through the image: <br>
<x-tab> </x-tab>P (pixel_i
= K times sampled) = \sum_{k=1...N} binom_coeff(N,K) (1/2)^K
(1/2)^{N-K}<x-tab> </x-tab>for i =
1...M<br>
<x-tab> </x-tab>(which is
of course the binomial distribution)<br>
<x-tab> </x-tab>Expectation:
E( K ) = N/2 = 200/2 = 100<br>
<x-tab> </x-tab>Variance:
Var( K ) = N/4 = 200/4 = 50<br>
<x-tab> </x-tab>Standard
Deviation: SD( K ) = sqrt (N/4) = 7. as you observed.<br><br>
image 4:<br>
after one jump of the RandomIterator<br>
<x-tab> </x-tab>P( pixel_i
= 0x sampled ) = (M-1)/M for i =
1...M<br>
<x-tab> </x-tab>P( pixel_i
= 1x sampled ) =
1/M<x-tab> </x-tab> for i =
1...M<br>
after R jumps of the RandomIterator:<br>
<x-tab> </x-tab>P( pixel_i
= K times sampled ) = \sum_{k=1...R} binom_coeff(R,K) (1/M)^K
((M-1)/M)^{R-K} for i = 1...M<br>
<x-tab> </x-tab>(so again
a binomial distribution)<br>
<x-tab> </x-tab>Expectation:
E(K) = R/M = 100M / M = 100<br>
<x-tab> </x-tab>Variance:
Var(K) = R/M * (M-1)/M = approximately R/M (if M is big) = 100.<br>
<x-tab> </x-tab>Standard
Deviation: SD(K) = sqrt( var(K) ) = 10. as you observed.<br><br>
My statistic math classes were some time ago, so correct me if i made a
mistake :)<br><br>
Regards,<br>
Stefan.<br><br>
At 16:43 16/03/05, Miller, James V (Research) wrote:<br>
</font><blockquote type=cite class=cite cite><font face="verdana" size=2 color="#0000FF">An
interesting observation:</font><font size=3><br>
<br>
</font><font face="verdana" size=2 color="#0000FF">The standard deviation
of the pixels in image2 and image4 are suprisingly different.
image2 has a standard deviation of 7 and image4 has a standard deviation
of 10. </font><font size=3><br>
<br>
</font><font face="verdana" size=2 color="#0000FF">If I understand the
original post correctly, both images were created using the same random
number generator. image2 was created by drawing a random number 200
times at each pixel and counting the number of times the random number
was greater than 0.5. image4 was created using a
ImageRandomIteratorWithIndex to visit pixels in the image at random,
performing enough random visits so that each pixel was on average visited
100 times (the same expected value as the pixels in
image2).</font><font size=3><br>
<br>
</font><font face="verdana" size=2 color="#0000FF">Naively, I would
expect the mean and standard deviations of these two images to be very
close.</font><font size=3><br>
<br>
</font><font face="verdana" size=2 color="#0000FF">The RandomIterator
pretty much just draws a random number between 0 and the number of pixels
and unrolls the resulting linear index into an index in the image.
The truncation of the random number to an integer and then to an index
increases the standard deviation of how many times a pixel is visited
from 7 to 10.</font><font size=3><br>
<br>
</font><font face="verdana" size=2 color="#0000FF">Probably not a useful
observation.... but interesting.</font><font size=3><br>
<br>
</font><font face="verdana" size=2 color="#0000FF">Jim</font><font size=3><br>
</font>
<dl>
<dd><font face="tahoma" size=2>-----Original Message-----<br>
<dd>From:</b> insight-developers-bounces@itk.org
[<a href="mailto:insight-developers-bounces@itk.org" eudora="autourl">mailto:insight-developers-bounces@itk.org</a>]On
Behalf Of </b>Blezek, Daniel J (Research)<br>
<dd>Sent:</b> Wednesday, March 16, 2005 10:05 AM<br>
<dd>To:</b> Stefan Klein; Insight-users@itk.org;
insight-developers@itk.org<br>
<dd>Subject:</b> [Insight-developers] RE: [Insight-users] random number
generator<br><br>
</font>
<dd><font face="arial" size=2 color="#0000FF">Stefan, I started a
thread on this a while ago relating to non-deterministic behavior across
platforms. If the ITK design committee approves the move, it would
be great to standardize the random number generator to be sufficiently
random, fast and generate the same sequences (from a given seed) across
platforms. The discussion culminated with Brad's post:
<a href="http://www.itk.org/mailman/private/insight-developers/2005-January/006220.html">http://www.itk.org/mailman/private/insight-developers/2005-January/006220.html</a>
I'm not sure any action happened on the
suggestions.</font><font size=3><br>
<dd> <br>
</font>
<dd><font face="arial" size=2 color="#0000FF">As you point out, many
registration algorithms depend on random sampling, exactly where I came
across the problem.</font><font size=3><br>
<dd> <br>
</font>
<dd><font face="arial" size=2 color="#0000FF">Cheers,</font><font size=3><br>
</font>
<dd><font face="arial" size=2 color="#0000FF">-dan</font><font size=3><br>
</font>
<dd><font face="tahoma" size=2>-----Original Message-----<br>
<dd>From:</b> insight-users-bounces@itk.org
[<a href="mailto:insight-users-bounces@itk.org" eudora="autourl">mailto:insight-users-bounces@itk.org</a>]On
Behalf Of </b>Stefan Klein<br>
<dd>Sent:</b> Wednesday, March 16, 2005 9:16 AM<br>
<dd>To:</b> Insight-users@itk.org<br>
<dd>Subject:</b> [Insight-users] random number generator<br><br>
</font><font size=3><br>
<dd>Dear itk-users,<br><br>
<dd>I did some tests with the underlying random generator of the
itkImageRandomIteratorWithIndex and it seems that, in Windows, it is 'not
very random'. <br><br>
<dd>The itkImageRandomIterator uses the following random-generator:<br>
<dd><ITKSOURCE>\Utilities\vxl\core\vnl\vnl_sample.<h/cxx><br><br>
<dd>In Linux the drand48 random-generator is used, which is a good
choice.<br>
<dd>In Windows however a "simple congruential random number
generator" is implemented, since drand48 is not available in
Windows. This gives inferior results, in my experience.<br><br>
<dd>To get a feel for the result look at the image1.<mhd/raw>,
which is in the file: randomtestresults.zip, which you can download from:
<a href="http://www.isi.uu.nl/People/Stefan/" eudora="autourl">http://www.isi.uu.nl/People/Stefan/</a><br>
<dd>The gray-values in this image show how many times a pixel was
sampled. The sampling<br>
<dd>process works was defined as follows:<br>
<dd> "An ImageRegionIterator walks N=200 times through
the image and tests<br>
<dd> at each voxel whether to sample it or not. The test is
performed by <br>
<dd> drawing a number between 0 and 1 using the random
generator defined in<br>
<dd> vnl_sample.h; a value >=0.5 means that the voxel is
sampled"<br>
<dd>As you can see in the image the pixels are not really selected at
random...<br><br>
<dd>If we use the itkImageRandomIteratorWithIndex to sample the image,
the result (image3.<mhd,raw>) may look better at first sight, but
the histogram looks terrible.<br><br>
<dd>On the internet I found the following link:<br>
<dd><a href="http://paine.wiau.man.ac.uk/pub/doc_vxl/core/vnl/html/classvnl__random.html" eudora="autourl">http://paine.wiau.man.ac.uk/pub/doc_vxl/core/vnl/html/classvnl__random.html</a><br>
<dd>It describes the files vnl_random.<h,cxx>; those files are not
included in the vxl-version that comes with ITK. I tried to use these as
a random generator. The results are now much better. Image2, which is
generated in the same was as image1, but with the random generator
defined in vnl_random, does not show any structure anymore. Image4, which
is generated using an ImageRandomIterator that uses the vnl_random as its
underlying random number generator, has the histogram that you would
expect.<br><br>
<dd>For more details please look at the source of the test program, which
you can also download from
<a href="http://www.isi.uu.nl/People/Stefan/" eudora="autourl">http://www.isi.uu.nl/People/Stefan/</a>
: itkrandomtestsource.zip. You may reproduce the results with this program. Note that only in Windows bad results will be obtained. In Linux there is no problem.<br><br>
<dd>Algorithms that rely on a good random generator may fail if you use the itkRandomImageIteratorWithIndex under Windows. For example, in my research on nonrigid registration I tried to use a stochastic gradient descent optimisation method for minimising the MattesMutualInformation in combination with a B-spline transform (it may speed up your registration algorithms; if you are interested: <br>
<dd><a href="http://www.isi.uu.nl/Research/Publications/publicationview.php?id=1011" eudora="autourl">http://www.isi.uu.nl/Research/Publications/publicationview.php?id=1011</a> ). When I used the itkImageRandomConstIteratorWithIndex for selecting spatial samples the registration results got significantly worse than when I used the itkImageMoreRandomConstIteratorWithIndex (which uses the vnl_random as underlying random number generator).<br><br>
<dd>Any comments on this would be appreciated!<br><br>
<dd>Stefan<br><br>
<br><br>
<br><br>
</font>
</dl></blockquote></body>
<br>
</html>