[Midas] midas batchmake plugin error

j at dicomfly.com j at dicomfly.com
Thu May 26 17:08:04 EDT 2011 

Hi Michael,
   It Works!

Many thanks...

J

> Hi J,
>
> There is a bug in the ImageMath.bms script, thanks for finding this.
>
> You can see that in the Condor script (ImageMath16.dagjob in your
> attachment) generated by BatchMake, the path to the mha file is incorrect
> (it appears to be duplicated) :
>
> transfer_input_files =
> /var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha//var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha
>
>
>
> If you change your ImageMath.bms to the following (in
> $MIDASROOT/midas/plugins/batchmake/examples/tasks) code this should fix
> this
> problem.
>
> Remember that these scripts are just examples for you to base your own
> work
> on.  Even so, they should work correctly and we aim for that.
>
> Thanks for your patience.
>
>
>
>
>
>
>
>
>
> # Should the script end in case of errors
> ExitOnError(1)
>
> # Imported parameters: cfg_input_cart, cfg_sigma
> Include(ImageMath.config.bms)
>
>
> # Set Application Properties
>
> SetApp(imageMath @ImageMath)
> SetAppOption(imageMath.GaussianBlur 1)
> SetAppOption(imageMath.GaussianBlur.Sigma ${cfg_sigma})
> SetAppOption(imageMath.OutputFileType 1)
> SetAppOption(imageMath.OutputFileType.FileType 3)
>
>
> Foreach(file ${cfg_input_cart})
>   SetAppOption(imageMath.InputFile ${file})
>
>   GetFilename(name ${file} NAME_WITHOUT_EXTENSION)
>
>   Set(outputFile ${name}.${cfg_output1}.mha)
>
>   SetAppOption(imageMath.OutputFile "${outputFile}")
>   CondorPostScript(imageMath ${cfg_condorpostscript} ${cfg_midas_baseURL}
> ${cfg_itemid} ${cfg_email} ${cfg_appname} ${cfg_apikey}
> ${cfg_output_directory} ${outputFile})
>   Run(output ${imageMath})
> Endforeach(file)
>
>
>
>
>
>
> On Tue, May 24, 2011 at 6:08 AM, <j at dicomfly.com> wrote:
>
>> Hi Michael,
>>   is there something that continue to goes wrong.
>>
>> Following your istruction I successfully executed, outside Midas,
>> ImageMath on ChestCT-SHORT.mha (from VTKData/Data/MetaIO/) with the
>> following command:
>>
>> /var/www/opt/apps/bin/ImageMath ~/data/ChestCT-SHORT.mha -b 2 -W 3
>> ~/data/3_ChestCT-SHORT.sum.mha
>>
>> and it works!
>>
>> But, unfortunately, running inside Midas with
>> $MIDAS_BATCHMAKE_USE_CONDOR =
>> 1
>>
>> [ImageMath.16.dagjob]
>>  Script generated by BatchMake (c) Insight Software Consortium
>> # (c) Kitware Inc 2009
>> # More information at: http://www.batchmake.org
>>
>> Universe       = vanilla
>> Output         = bmGrid.16.out.txt
>> Error          = bmGrid.16.error.txt
>> Log            = bmGrid.16.log.txt
>> Notification   = NEVER
>> Executable    = /var/www/opt/apps/bin/ImageMath
>> Arguments     =
>>
>> "'/var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha//var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha'
>> -b '2' -W '3' ''3_ChestCT-SHORT.sum.mha''"
>> transfer_input_files =
>>
>> /var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha//var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha
>> should_transfer_files = yes
>> when_to_transfer_output = ON_EXIT_OR_EVICT
>> Queue 1
>>
>> we have at least the following error:
>>
>> [from ImageMath.dagjob.dagman.out]
>> 05/24/11 01:04:41 From submit: ERROR: Can't open
>>
>> "/var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha/var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha"
>>  with flags 00 (Not a directory)
>>
>> Is there an error passing imagepath from midas/cart to
>> batchmake/condor/ImageMath?
>>
>> How can I solve it?
>>
>> In the attached file, you can find debug and error midas logs, the
>> complete SSP directory created by batchmake plugin to perform the
>> requested job(s) (#18)...
>>
>> Thanks in advance
>>
>> Ciao
>>
>> J
>>
>>
>>
>>
>> > Hi J,
>> >
>> > At this time, executing locally (without Condor) isn't fully supported
>> on
>> > Midas, so you should be using
>> >
>> > $MIDAS_BATCHMAKE_USE_CONDOR = 1
>> >
>> > in your config_batchmake.php or config.batchmake.local.php files.
>> >
>> >
>> >
>> > Do you have your Condor grid setup?
>> >
>> >
>> >
>> > Since these are just example applications, you don't need to have them
>> > running to use Midas and BatchMake.  If you have the PixelCounter
>> > application running this should give you an example for sending files
>> into
>> > a
>> > BatchMake application/script from within Midas.  These examples just
>> give
>> > you a basis for developing your own BatchMake scripts and
>> applications.
>> >
>> >
>> >
>> > As far as debugging, once you have the MIDAS_BATCHMAKE_USE_CONDOR
>> option
>> > set
>> > to 1, there should appear some condor outputs and logs in your
>> BatchMake
>> > temporary (batchmake-basedir temp) execution directory, which you can
>> use
>> > to
>> > see any problems.
>> >
>> >
>> >
>> > One problem in this case, judging from your output of "Error:
>> > SetAppOption:
>> > Cannot find parameter: GaussianBlur.", is probably that incorrect .bmm
>> > files
>> > (or none at all) were packaged with Midas.  I'm attaching the
>> appropriate
>> > .bmm file for the ImageMath application.  Place this in the directory
>> you
>> > have defined for MIDAS_BATCHMAKE_APPLICATION_DIRECTORY, and change the
>> > path
>> > in the .bmm file to the correct path to the ImageMath application.
>> >
>> >
>> > I hope this helps.
>> >
>> >
>> >
>> >
>> >
>> > On Wed, May 18, 2011 at 6:55 AM, <j at dicomfly.com> wrote:
>> >
>> >> Hi Michael,
>> >>   following your info I was able to successfully execute PixelCounter
>> >> (without and with condor=execute_in_server). I saw the following echo
>> >> and I found output xml:
>> >>
>> >> #use_condor=0
>> >> Executing ...
>> >> Status: Start /var/www/opt/batchmake/bin/PixelCounter
>> >> /var/www/BASEDIR-batchmake/data/user_1/cart_2/10_AddTest_DOUBLE.mha
>> 10
>> >> --ofxml 10_AddTest_DOUBLE.output.xml
>> >> run: '/var/www/opt/batchmake/bin/PixelCounter'
>> >> '''/var/www/BASEDIR-batchmake/data/user_1/cart_2/10_AddTest_DOUBLE.mha'
>> >> '10' --ofxml '10_AddTest_DOUBLE.output.xml'''
>> >> Status: Finish: Execution time 315ms
>> >>
>> >> Unfortunately, trying to run ImageMathAdder (following rigorously the
>> >> info
>> >> in the bms file: i.e. png filenames in cart1 and cart2, etc.),
>> something
>> >> goes wrong: I cannot find any output image.
>> >>
>> >> I saw the following echo (only one line):
>> >> Executing ...
>> >>
>> >> I tried to understand what happens checking in logs and in
>> >> batchmake-basedir temp folder but without success (apparently no
>> >> errors...)
>> >>
>> >> For ImageMath a different error is returned:
>> >> Executing ...
>> >> Error: SetAppOption: Cannot find parameter: GaussianBlur.
>> >>
>> >>
>> >> Any advice?
>> >>
>> >> Thanks in advance...
>> >>
>> >> J
>> >>
>> >>
>> >>
>> >> > Hi J,
>> >> >
>> >> > Can you tell me more about what you are trying to accomplish with
>> >> Midas?
>> >> > I
>> >> > might be able to give you some better advice if I know this.
>> >> >
>> >> >
>> >> >
>> >> > PixelCounter won't work with DICOM images, it is really more of an
>> >> example
>> >> > or demo application for testing out the BatchMake/Condor
>> >> infrastructure.
>> >> >  This is useful to test before you do any other development, so you
>> >> can
>> >> be
>> >> > sure the infrastructure works correctly.
>> >> >
>> >> >
>> >> > Here is a link that covers some of this information.  If anything
>> is
>> >> > unclear
>> >> > or you would like more details in a particular area, please let us
>> >> know
>> >> as
>> >> > we are always trying to improve the documentation.
>> >> >
>> >> >
>> >>
>> http://www.kitware.com/midaswiki/index.php/Plugins:BatchMake#Adding_a_BatchMake_script_to_the_Midas_BatchMake_Plugin
>> >> >
>> >> >
>> >> >
>> >> > For the PixelCounter application, you can use some sphere image
>> files
>> >> as
>> >> > inputs.  In the BatchMake build directory, under bin, if you
>> >> configured
>> >> > BatchMake with BUILD_EXAMPLES = on, you should have a
>> GenerateSpheres
>> >> > executable.  If you run this, it will generate several sphere
>> images
>> >> files
>> >> > for you.  You can upload some of these to Midas, then create a cart
>> >> and
>> >> > add
>> >> > the images to the cart.  Use this cart as the input to the
>> >> PixelCounter
>> >> > application.
>> >> >
>> >> >
>> >> >
>> >> > Once you run the PixelCounter, you can use the Midas logs to help
>> >> debug
>> >> > problems.  Also, look in the directory defined for
>> >> >
>> >> > $MIDAS_BATCHMAKE_TEMP_DIRECTORY
>> >> >
>> >> > in your BatchMake Midas plugin configuration file.  Here you will
>> see
>> >> the
>> >> > temporary directories associated with BatchMake processing in
>> Midas,
>> >> and
>> >> > the
>> >> > generated Condor output files there can help you debug further
>> >> problems.
>> >> >
>> >> >
>> >> >
>> >> > I think the development version of Midas taken out of the Git
>> >> repository
>> >> > (2.8.1) is the best one to use for doing BatchMake plugin
>> development,
>> >> as
>> >> > this area of Midas is under active development.
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On Sat, May 7, 2011 at 12:26 PM, J <j at dicomfly.com> wrote:
>> >> >
>> >> >>  Hi Michael,
>> >> >>         batchmake plugin now seems to work even if PixelCounter
>> >> example
>> >> >> runs but does not produce any output: any idea? I tried to process
>> >> DICOM
>> >> >> images...
>> >> >>
>> >> >> Which is the better Midas version to use? (2.8.1, 2.9, ??)
>> >> >>
>> >> >> Thanks in advance
>> >> >>
>> >> >> J
>> >> >>
>> >> >>
>> >> >>
>> >> >> At 16.48 05/05/2011, you wrote:
>> >> >>
>> >> >> Hi J,
>> >> >>
>> >> >> First off, you will want to upgrade your BatchMake to the current
>> >> >> development version to take advantage of integration with Midas
>> and
>> >> >> Condor:
>> >> >>
>> >> >>  http://batchmake.org/batchmake/resources/software.html
>> >> >>
>> >> >> You can clone from git and then build the source using Cmake.
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> A couple things to think about before running any BatchMake
>> >> applications
>> >> >> from within Midas:
>> >> >>
>> >> >> -Is your condor grid status correctly viewable from the "Midas
>> Grid
>> >> >> Status"
>> >> >> link of the BatchMake plugin?
>> >> >> -Are all of the configuration settings correct (green colored)
>> from
>> >> the
>> >> >> "Check Configuration" link of the BatchMake plugin?
>> >> >>
>> >> >>
>> >> >>
>> >> >> There should be more detailed error messages available in the
>> Midas
>> >> >> logs.
>> >> >> Look in the MIDAS_INSTALL_DIR/midas/tmp/logs/error.log , this will
>> >> help
>> >> >> you
>> >> >> with your specific problem.
>> >> >>
>> >> >>
>> >> >> Let me know if this helps.
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> > --
>> >> > Thanks,
>> >> > Michael Grauer
>> >> > R & D Engineer
>> >> > Kitware, Inc.
>> >> > 919 969 6990 x322
>> >> >
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > Thanks,
>> > Michael Grauer
>> > R & D Engineer
>> > Kitware, Inc.
>> > 919 969 6990 x322
>> >
>>
>
>
>
> --
> Thanks,
> Michael Grauer
> R & D Engineer
> Kitware, Inc.
> 919 969 6990 x322
>

More information about the Midas mailing list