[vtk-developers] IRC meet log (Aug 6, 2002)

Tue Aug 6 11:00:08 EDT 2002

[10:06] *** KenMartin has joined #vtk
[10:06] <SebastienB> Hi Paul, Jeff
[10:06] <prabhu> hi all.
[10:06] <KenMartin> Hey Folk,
[10:06] <SebastienB> What do you want to address today ?
[10:06] <KenMartin> DOes anyone have any issues for the agenda today?
[10:07] <SebastienB> do'h
[10:07] <prabhu> I will have something for the next meet I hope.  Nothing 
for now.
[10:07] <paulsen> hmm. I was just planning to listen
[10:07] <KenMartin> The dashboard is a mess today but it should clean up 
pretty quickly.
[10:08] <KenMartin> A few machines needed to updated CMake to patch 3 to 
take advantage of the new etst driver
[10:08] <prabhu> just curious, why is it a mess?
[10:08] *** amy_ has joined #vtk
[10:08] <prabhu> oh ok.
[10:08] *** hoffman has joined #vtk
[10:08] *** legoandy has joined #vtk
[10:08] <SebastienB> the tests have been collapsed into a smaller test 
drivers, which require CMake 1.4
[10:08] <KenMartin> A change in syntax of the CREATE_TEST_SOURCELIST 
command in patch 3
[10:08] <legoandy> Hello
[10:08] *** kingb has joined #vtk
[10:08] *** kingb has quit IRC (Client Quit)
[10:09] <SebastienB> + some push pipeline I gues
[10:09] <KenMartin> I am hoping that patch3 will be the key 1.4 version of 
CMake and that any other patches will just be minor bug fixes but not 
required for VTK
[10:09] *** kingb has joined #vtk
[10:09] <KenMartin> The push pipline is just warnings, same as FTGL stuff
[10:10] <jeffL> hi andy, the cmake stuff you did for java works great
[10:10] <prabhu> what's push pipeline?
[10:10] <SebastienB> I do agree, patch 3 looks really nice
[10:10] <KenMartin> I'll try to post something about it, but in summary it 
is a data driven pipeline execution...
[10:11] <KenMartin> in contrast to VTKs normal demand driven exection
[10:11] <prabhu> hmm.
[10:11] <KenMartin> It can be useful for data driver souces such as a video 
camera, CT scanner etc
[10:12] <KenMartin> I don't recall if there were any open issues from last 
time?
[10:12] <SebastienB> I'm looking at it right now
[10:12] <SebastienB> I remember: vtkString, + SetRefCount
[10:12] <legoandy> SetRefCount is fixed
[10:12] <KenMartin> The SetRefCount was resolved, yes?
[10:13] <prabhu> SetRefcount is a non-issue now.
[10:13] <prabhu> yes
[10:13] <SebastienB> Good
[10:13] *** berk has joined #vtk
[10:13] <SebastienB> Python issues anymore ?
[10:14] <KenMartin> I still think vtkString should be removed but Andy and 
I can discuss that at some point. It seems that others do not have a strong 
opinion
[10:14] <prabhu> yes but I need python users to chat here.
[10:14] <legoandy> Well, I am still fixing some little python issues on Sun
[10:14] <prabhu> I've been wanting to add testing support but I've been busy.
[10:14] <legoandy> Like for example - it does not work without hacking
[10:14] <SebastienB> Ken: I missed the arguments against vtkString. You 
mean, vtkString as an object, or as a kind of namespace ?
[10:14] <prabhu> I hope that I'll be able to do something next meet.
[10:14] <KenMartin> As a namespace
[10:15] <legoandy> I am against vtkString as an object.
[10:15] <prabhu> By test support I mean a way to do unittests with Python 
for VTK.
[10:15] <legoandy> (Strongly...)
[10:15] <legoandy> I mean every toolkit adds string class and they are all 
non compatible
[10:15] <SebastienB> Ken: you mean you are against string-related funcs or 
their location/namespaces ?
[10:16] <legoandy> And they are usually slow
[10:16] <legoandy> But having platform independent string manipulation 
functions ...
[10:16] <SebastienB> for internal use
[10:16] <legoandy> Of course
[10:16] <KenMartin> I am against creating our own string functions to 
replace functions that are standard and almost the same thing
[10:16] <legoandy> String class is not wrapped
[10:16] <legoandy> (or at least should not be)
[10:17] <KenMartin> Such as strlen() versus vtkString::Length()
[10:17] <legoandy> But they are just as readable and there is almost no 
problem of using either
[10:18] <legoandy> Except that for using let say strlen on all platforms 
you have to put cast around, which makes code strange
[10:18] <legoandy> strdup does wrong things
[10:18] <SebastienB> Ken: I agree with that, actually what is just a bit 
painful is alloc/dealloc and all that jazz, that's why a vtkString object 
for internal use might be helpful, just like, say, I would like to have 
whatever part of a filename without worrying about alloc/dealloc
[10:18] <legoandy> I kind of agree with that
[10:19] <legoandy> But you can do all that without vtkString object
[10:19] <SebastienB> and do some concat stuff with bits of vtkString...
[10:19] <legoandy> Again, no object all char*
[10:19] <kingb> I believe "int len = strlen(s)" is the motivating example 
for vtkString::Length().
[10:19] <kingb> Standard C specifies that strlen returns "size_t".
[10:20] <KenMartin> it should be size_t len = strlen(foo);
[10:20] <kingb> Does anyone know if "size_t len = strlen(s)" doesn't work 
on a particular platform?
[10:20] <SebastienB> I mean, it's no big deal, but each time I have a piece 
of code that handles files in VTK, it's the same pb. It's quickly resolved, 
but still a slight accumulating pain each time :)
[10:20] <kingb> I guess I should add it to the "TestCxxFeatures" test.
[10:21] <KenMartin> But it is a pain ALL C++/C programmers know about. 
Anyone can look at the code and knows what is going on.
[10:21] <legoandy> I don't know about platform issues regarding size_t, but 
does anybody actually use size_t?
[10:21] <legoandy> People like using int for everything
[10:22] <paulsen> As a new developer I would like to raise a more 
philosophical question about code reviewing. Have it been considered to 
make a formalised review process when people are checking code in. I mean 
that another developer should check that 1. the code is correct. 2. It does 
what it is supposed to? (yes, I know this is not string related)
[10:22] <hoffman> Common/vtkObjectBase.cxx:void* vtkObjectBase::operator 
new(size_t nSize)
[10:22] <KenMartin> We should, size_t is an important type, that is why so 
many std routines use it.
[10:22] <SebastienB> Ken: sure. I mean, I thought we were focusing on more 
advanced stuff than just C++/C :) Anyway...
[10:22] <legoandy> So, what is the difference between size_t and vtkIdType?
[10:23] <KenMartin> Id type may be smaller than size_t
[10:23] <kingb> or larger
[10:23] <KenMartin> depending on how they build VTK
[10:23] <legoandy> But it does the same thing does not it?
[10:24] <KenMartin> Not close enough IMHO
[10:24] <kingb> size_t is unsigned
[10:24] <kingb> vtkIdType is signed
[10:24] <KenMartin> Re: code reviews, there is no formal process...
[10:24] <SebastienB> paulsen: what do you mean ? Do you think like it 
should be reviewed more closely ?
[10:25] <KenMartin> Big changes are usually discussed first for architecture
[10:26] <paulsen> What I mean is that when I checked my first class in some 
of you guys where nice enough to look through the code for typos etc. But 
that is no guarentee that it really does what it should do (Principal 
component analysis in this case)
[10:26] <KenMartin> Most VTK developers know how to write good code and the 
dashboard helps with this.
[10:26] <SebastienB> paulsen: that's the reason it is important that you 
provide tests
[10:26] <KenMartin> And look at the coverage to make sure your class is > 
70% covered
[10:27] <SebastienB> paulsen: is the PCA tested ?
[10:27] <legoandy> And look at purify results for all tests to see if you 
missed something
[10:27] <paulsen> Yes, that will guarentee that the implementation is 
correct, but not the math.
[10:27] <legoandy> If you write good test, you can check the math too
[10:27] <SebastienB> sure
[10:28] <KenMartin> Code reviews are a great tool. You just need to find 
someone to donate their time
[10:28] <paulsen> You mean that I should perform the calculations in 
Matlab, and then check it against the C++
[10:28] <SebastienB> paulsen: it looks like your test is "visual" only, but 
you can provide a pure computational test only, provided that it returns a 
true/false value depending on the solution you are expect (and that you 
would hardcode)
[10:29] <legoandy> paulsen: For example...
[10:29] <paulsen> Yes, but it depends on correct baseline data. Which in 
that case should not be made with my own implementation.
[10:29] <SebastienB> paulsen: well, make your class solve a known pb, 
hardcode the solution +- margin/errors, and test the result of your clas
[10:29] <KenMartin> Matlab would be fine for a sanity check
[10:29] <SebastienB> paulsen: you can provide your own data into VTKData 
(but not too large)
[10:29] <legoandy> Hmm, matlab installed on test system and ctest would 
check same calculation with MatLab....
[10:30] <KenMartin> Nooo
[10:30] <SebastienB> nope
[10:30] <legoandy> IF(MATLAB) .... ENDIF(MATLAB)
[10:30] <KenMartin> Solve a few problems with Matlab.. and then hard code 
one or two of them into a C== test
[10:30] <KenMartin> (C++ test)
[10:31] <paulsen> I agree with all your comments, but I am having a hard 
time making my point clear. I will try once more ;)
[10:31] <KenMartin> There is no need to run matlab every night, I assume 
its answer will not change
[10:31] <prabhu> On the topic of tests the VTK tests do not seem to be unit 
tests. They seem more like create a pipeline and test the image.
[10:32] <SebastienB> I hope so, that would be scary
[10:32] <prabhu> Or am I wrong?
[10:32] <SebastienB> paulsen: go ahead
[10:32] <legoandy> prabhu: test is essentially - do something if 
successfull, pass, otherwise no
[10:32] <KenMartin> The pipeline is designed to test a unit (white box)
[10:33] <prabhu> true but it is not fine grained enough.
[10:33] <SebastienB> prabhu: again, you are free to do anything you want in 
the test, provided that it returns a true/false success value
[10:33] <prabhu> yes I know
[10:33] <paulsen> Imagine a mad student providing a class which he claims 
solve a statistical problem. He also provides baseline data calculated with 
that class. The problem is that the student do not know math and has made a 
lot of mathematical errors so the class is giving false results always 
(like saying 2+2=5)
[10:33] <prabhu> but is there some sort of policy on this?
[10:33] <SebastienB> prabhu: you want a fuzzy dashboard ?
[10:34] <KenMartin> Use matlab as a sanity check, does that solve your 
problem or am I missing something?
[10:34] <prabhu> SebastienB: fuzzy as in?
[10:34] <KenMartin> Or solve a test case that has a known analytical answer
[10:34] <paulsen> No Ken you are absolutely correct!
[10:34] <SebastienB> prabhy: what's your point with granularity ?
[10:35] <KenMartin> Once your code and Matlab agree, create a test to make 
sure that your codes answer doesn't change
[10:35] <KenMartin> That way future edits on the class can be monitored to 
make sure they don't change the answer
[10:35] <paulsen> But another option is that another person with knowledge 
of the field made a sanity of the code. Like in extreme programming
[10:35] <paulsen> (i meant sanity check)
[10:35] <prabhu> SebastienB: I'm used to unit tests and they seem nice in 
that they test the code in many ways ensuring a particular behaviour.
[10:36] <KenMartin> Actually doing both is the best case
[10:36] <paulsen> exactly. I have made my point :)
[10:36] <SebastienB> prabhu: I don't get it. What's your "unit test" ? You 
mean, group of tests ?
[10:36] <KenMartin> Do a code review (of the math and impl) and verify 
against another tool or analytic solution
[10:36] <legoandy> paulsen: I don't think there is enough time to check 
every code ...
[10:37] <legoandy> I mean most code will break at some point...
[10:37] <prabhu> SebastienB: unit tests as in check each and every member 
and test and ensure that the behaviour is what is advertized etc.
[10:37] <paulsen> Ugh! That was a hard one andy.
[10:37] <SebastienB> paulsen: moreover, PCA is something I don't think we 
use internally in any of our projects.
[10:37] <legoandy> So, what we want is to minimize the amount of code that 
will break
[10:38] <jeffL> the only way to do that is to write less codd
[10:38] <prabhu> SebastienB: unittest is what the xp folks call 
testing.  If you want I can dig up a few urls or mail you some code.
[10:38] <paulsen> No but there are other classes with very similar 
behaviour. OBB trees etc. But I was not refereing to a specific class.
[10:38] <legoandy> Let me rephrase my previous statement: I don't think 
there is time for every picece of code to be tested by another person every 
time something is commited
[10:39] <SebastienB> prabhu: nothing prevents you from doing that, i.e. 
more elaborate tests, or a test made of several tests, the result of the 
test being the fact that all passed (if not, just send something to cerr 
and we will see it in the dashboard)
[10:39] <SebastienB> andy: can you rephrase it again :) ?
[10:39] <prabhu> SebastienB : OK.
[10:39] <paulsen> I agree with you Andy. But the more things are tested, 
the more you can trust them.
[10:40] <KenMartin> This has been a good discussion, but are there any key 
issues left here?
[10:40] <prabhu> Another thought.
[10:40] <KenMartin> OK
[10:40] <prabhu> The current tests use images
[10:40] <KenMartin> Some do
[10:40] <KenMartin> Some don;t
[10:40] <prabhu> can we use an md5sum of an image or something like that
[10:40] <prabhu> that eliminates the need to create images?
[10:40] <prabhu> Yes, some dont use images.
[10:41] <SebastienB> prabhu: an extreme example would be the "complex" test 
in CMake: 
http://www.cmake.org/Testing/Dashboard/20020806-0200-Nightly/TestDetail/__Source_complex.html
[10:41] <KenMartin> Danger, do not look at the complex test in CMake, you 
might go blind, I'm serious, that is a nasty test
[10:41] <jeffL> are there plans to test wrapped languages?
[10:42] <prabhu> What I'm getting at is that could two images be 
characterized without storing the image.
[10:42] <prabhu> I understand that you cant do a diff but you do save space.
[10:42] <SebastienB> pra: no, actually md5 would not allow us I guess to 
handle the fact that the match between valid image and test image is not an 
ëxact" match (due to precision issues and stuff like that)
[10:42] <KenMartin> Re: md5 sum, the problem is that the valid image and 
test image are very useful in debugging
[10:42] <SebastienB> Ken: yep, it's an extreme test, I'm not in favor of 
going this way :)
[10:42] <prabhu> jeffL : There should be and I'm planning on adding some 
support for Python unittests soon.
[10:43] <prabhu> md5: yes, but md5 is what i thought of, is there some 
other way?
[10:43] <prabhu> I'm no imaging expert.
[10:43] <KenMartin> When you do not have access to the other persons 
computer, being able to see the two images can often be the difference in 
solving the problem remotely
[10:43] <jeffL> testing under cmake/dart or a python thing?  I have been 
using junit for java which works quite well for smoke tests
[10:43] <SebastienB> pra: and I guess hard-disk space is not the issue (and 
should not versus the fact that these images are very handy for debugging)
[10:43] <prabhu> KM: Yes I understand.
[10:43] <legoandy> jeffL: we have to start testing Java
[10:44] <legoandy> Write some tests for Java VTK
[10:44] <prabhu> jeffL: pyunit along with cmake so that it can be darted.
[10:44] <prabhu> SebastienB: It was a thought, and i agree hard disk space 
is not an issue now.
[10:44] <legoandy> I added build for java, so now we can try to execute 
some tests
[10:44] <prabhu> But imagine testing every little method you apply
[10:44] <prabhu> you cant test everything with an image.
[10:45] <SebastienB> your test should try to stress the whole class code 
(thus increasing the coverage)
[10:45] <KenMartin> There are other tests such as SetGet which tests every 
Set/Get method in every class
[10:45] <prabhu> I mean if you want to rigorously test a class and all its 
methods and want to make sure that everything does it "right" you cant 
create 10 images per class.
[10:45] <KenMartin> other tests like that could be written
[10:45] <prabhu> then hdd space does make a huge difference.
[10:45] <prabhu> so, is there a quick test that can be applied.
[10:45] <KenMartin> There is no need for every test to be a image based test
[10:46] <prabhu> I'm not saying eliminate images.
[10:46] <legoandy> You can create grid of multiple images
[10:46] <prabhu> I'm asking to see if there are other ways of testing.
[10:46] <SebastienB> pra: if this is *really* an issue, I guess you could 
be nasty and stress different thing in your test, thus produce different 
small 32x32 images, paste them into a mosaic to create a single valid test 
image (wow)
[10:46] <KenMartin> There are other ways of testing. See 
VTK/Common/Testing/Cxx it has over 20 tests none of which use images. Same 
idea in VTK/COmmon/Texting/Tcl
[10:47] <jeffL> there are many ways to test, I think what we want is a 
framework which allows the developer to define pass/fail - images, diffs, 
fuzzy diffs, any logic
[10:47] <SebastienB> pra: this is done in some tests already 
(vtkImageMathematics for ex)
[10:48] <prabhu> hmm you mean diffs done without images?
[10:48] <jeffL> if you are testing numerics, yes
[10:48] <SebastienB> pra: you are free to do what you want in the test, 
read whatever baseline data, compare it, diff it, but it should return 
true/false
[10:48] <KenMartin> All Dart/ctest requires is that at the end of the day 
you return 0 or 1, inside your test you are welcome to do etst diffs, 
tolerance based math comparisons etc, there isn't a framework for all of 
those, but nothing is wrong with creating them
[10:49] <prabhu> I know that. :)
[10:49] <prabhu> I am not very knowledgeable about the imaging pipeline and 
wanted to see if testing could be done in other ways. i.e. is this a 
feasible idea etc.
[10:50] <prabhu> I know that every test does not require an image.  Thats 
obvious.
[10:50] <KenMartin> Sure, I like the md5 idea with respect to tests that 
generate large data etc
[10:50] <prabhu> But sometimes the ultimate test is to use an image
[10:50] <KenMartin> I think we are all very open to other types of test
[10:50] <prabhu> Of course.
[10:51] <legoandy> One of the problem of md5 is that the image has to be 
exactly what the test image was. If one pixel is wrong, test will fail
[10:51] <KenMartin> My only request is that the tests be fast (especially 
for purify) and not require large data files in VTKData
[10:51] <prabhu> I'll see if I can do something like that.
[10:51] <prabhu> But do you folks know of anything like md5 that will be 
good for images?
[10:51] <legoandy> What is wrong with images ...
[10:52] <prabhu> Maybe get the images and diff the bits and  get the 
resulting numbers
[10:52] <legoandy> They compress well, and comparing is a fast operation
[10:52] <KenMartin> Not really, opengl has a lot of variety in what pixel 
values a representation will produce
[10:52] <legoandy> Exactly
[10:52] <legoandy> That is why MD5 would fail every time
[10:52] <KenMartin> So you need a complex metric like what is used in 
ImageDifference
[10:53] <SebastienB> pra: no, do not forget that there is a tolerance built 
in the function that compares two images. This is really mandatory, and 
sometimes it's not enough that's why two or more valid images had to be 
created in VTKData (like for the old text tests)
[10:53] <prabhu> tolerances cant be used?
[10:53] <KenMartin> And for that metric to work you ned the valid image
[10:53] <prabhu> hmm.
[10:53] <SebastienB> pra: tolerance *are* used, but they would not 
translate into MD5, since MD5 does not care about image-domain problem
[10:53] <KenMartin> tolerances can be used, but the tolerance needs the 
valid image, I don;t know of a metric that can use a global tolerace
[10:54] <prabhu> ok
[10:54] <KenMartin> But for some other tests that produce data that does 
not depend on OpenGL, somehting like md5 might be great
[10:55] <legoandy> So, Java testing (change the subject)
[10:55] <KenMartin> Also have to watch for floating point diffs
[10:55] <KenMartin> (quickly)
[10:55] <prabhu> ok.
[10:55] <SebastienB> ken: I don't remember of the MD5 implem, but I think 
that as most checksum, it would statistically fail detecting some % of 
errors. I mean, we are doing test here, so the aim is to test correctly, 
not to spare hard-disk space. Just my opinion
[10:55] <jeffL> i like junit and if i can integrate it with cmake, that 
would be great - i'm guessing cmake can just call junit or ant to run the 
java tests
[10:56] <prabhu> jeffL : I'm planning on using the pyunit code and return 
1/0 to indicate success/failure.
[10:56] <legoandy> Why not just directly call ${JAVA_RUNTIME} ${blabla}/Test
[10:57] <jeffL> junit provides the framwork to specify the test and its 
pass/fail criteria
[10:57] <prabhu> yes but cant you trap it and return the appropriate exit 
status when its done?
[10:58] <legoandy> But there is regression test written in java ain so Ken?
[10:58] <jeffL> but you could just call call ${JAVA_RUNTIME} ${blabla}/Test 
and if it doesn't die, then return pass/fail
[10:58] <KenMartin> OK Folks, I don't want to to stop the discussion, 
but.... I'm turning into a pumpkin
[10:58] <KenMartin> We have had some good discussion...
[10:58] *** jeffL has quit IRC ("later")
[10:58] <paulsen> Wauw. Thanks for the good info
[10:59] *** kingb has quit IRC ("Client Exiting")
[10:59] <KenMartin> but why don't we take a break until next time?
[10:59] <prabhu> thanks folks for being patient.
[10:59] <legoandy> Later
[10:59] <SebastienB> Thanks. Bye
[10:59] *** amy_ has left #vtk
[10:59] <prabhu> bye
[10:59] *** legoandy has quit IRC ("Client Exiting")
[10:59] <KenMartin> Thanks Again
[10:59] *** berk has quit IRC ("Client Exiting")
[10:59] <paulsen> bye

--
Sebastien Barre