[Midas] midas batchmake plugin error
Michael Grauer
michael.grauer at kitware.com
Thu May 26 16:26:49 EDT 2011
Hi J,
There is a bug in the ImageMath.bms script, thanks for finding this.
You can see that in the Condor script (ImageMath16.dagjob in your
attachment) generated by BatchMake, the path to the mha file is incorrect
(it appears to be duplicated) :
transfer_input_files =
/var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha//var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha
If you change your ImageMath.bms to the following (in
$MIDASROOT/midas/plugins/batchmake/examples/tasks) code this should fix this
problem.
Remember that these scripts are just examples for you to base your own work
on. Even so, they should work correctly and we aim for that.
Thanks for your patience.
# Should the script end in case of errors
ExitOnError(1)
# Imported parameters: cfg_input_cart, cfg_sigma
Include(ImageMath.config.bms)
# Set Application Properties
SetApp(imageMath @ImageMath)
SetAppOption(imageMath.GaussianBlur 1)
SetAppOption(imageMath.GaussianBlur.Sigma ${cfg_sigma})
SetAppOption(imageMath.OutputFileType 1)
SetAppOption(imageMath.OutputFileType.FileType 3)
Foreach(file ${cfg_input_cart})
SetAppOption(imageMath.InputFile ${file})
GetFilename(name ${file} NAME_WITHOUT_EXTENSION)
Set(outputFile ${name}.${cfg_output1}.mha)
SetAppOption(imageMath.OutputFile "${outputFile}")
CondorPostScript(imageMath ${cfg_condorpostscript} ${cfg_midas_baseURL}
${cfg_itemid} ${cfg_email} ${cfg_appname} ${cfg_apikey}
${cfg_output_directory} ${outputFile})
Run(output ${imageMath})
Endforeach(file)
On Tue, May 24, 2011 at 6:08 AM, <j at dicomfly.com> wrote:
> Hi Michael,
> is there something that continue to goes wrong.
>
> Following your istruction I successfully executed, outside Midas,
> ImageMath on ChestCT-SHORT.mha (from VTKData/Data/MetaIO/) with the
> following command:
>
> /var/www/opt/apps/bin/ImageMath ~/data/ChestCT-SHORT.mha -b 2 -W 3
> ~/data/3_ChestCT-SHORT.sum.mha
>
> and it works!
>
> But, unfortunately, running inside Midas with $MIDAS_BATCHMAKE_USE_CONDOR =
> 1
>
> [ImageMath.16.dagjob]
> Script generated by BatchMake (c) Insight Software Consortium
> # (c) Kitware Inc 2009
> # More information at: http://www.batchmake.org
>
> Universe = vanilla
> Output = bmGrid.16.out.txt
> Error = bmGrid.16.error.txt
> Log = bmGrid.16.log.txt
> Notification = NEVER
> Executable = /var/www/opt/apps/bin/ImageMath
> Arguments =
>
> "'/var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha//var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha'
> -b '2' -W '3' ''3_ChestCT-SHORT.sum.mha''"
> transfer_input_files =
>
> /var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha//var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha
> should_transfer_files = yes
> when_to_transfer_output = ON_EXIT_OR_EVICT
> Queue 1
>
> we have at least the following error:
>
> [from ImageMath.dagjob.dagman.out]
> 05/24/11 01:04:41 From submit: ERROR: Can't open
>
> "/var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha/var/www/BASEDIR-batchmake/data/user_1/cart_3/3_ChestCT-SHORT.mha"
> with flags 00 (Not a directory)
>
> Is there an error passing imagepath from midas/cart to
> batchmake/condor/ImageMath?
>
> How can I solve it?
>
> In the attached file, you can find debug and error midas logs, the
> complete SSP directory created by batchmake plugin to perform the
> requested job(s) (#18)...
>
> Thanks in advance
>
> Ciao
>
> J
>
>
>
>
> > Hi J,
> >
> > At this time, executing locally (without Condor) isn't fully supported on
> > Midas, so you should be using
> >
> > $MIDAS_BATCHMAKE_USE_CONDOR = 1
> >
> > in your config_batchmake.php or config.batchmake.local.php files.
> >
> >
> >
> > Do you have your Condor grid setup?
> >
> >
> >
> > Since these are just example applications, you don't need to have them
> > running to use Midas and BatchMake. If you have the PixelCounter
> > application running this should give you an example for sending files
> into
> > a
> > BatchMake application/script from within Midas. These examples just give
> > you a basis for developing your own BatchMake scripts and applications.
> >
> >
> >
> > As far as debugging, once you have the MIDAS_BATCHMAKE_USE_CONDOR option
> > set
> > to 1, there should appear some condor outputs and logs in your BatchMake
> > temporary (batchmake-basedir temp) execution directory, which you can use
> > to
> > see any problems.
> >
> >
> >
> > One problem in this case, judging from your output of "Error:
> > SetAppOption:
> > Cannot find parameter: GaussianBlur.", is probably that incorrect .bmm
> > files
> > (or none at all) were packaged with Midas. I'm attaching the appropriate
> > .bmm file for the ImageMath application. Place this in the directory you
> > have defined for MIDAS_BATCHMAKE_APPLICATION_DIRECTORY, and change the
> > path
> > in the .bmm file to the correct path to the ImageMath application.
> >
> >
> > I hope this helps.
> >
> >
> >
> >
> >
> > On Wed, May 18, 2011 at 6:55 AM, <j at dicomfly.com> wrote:
> >
> >> Hi Michael,
> >> following your info I was able to successfully execute PixelCounter
> >> (without and with condor=execute_in_server). I saw the following echo
> >> and I found output xml:
> >>
> >> #use_condor=0
> >> Executing ...
> >> Status: Start /var/www/opt/batchmake/bin/PixelCounter
> >> /var/www/BASEDIR-batchmake/data/user_1/cart_2/10_AddTest_DOUBLE.mha 10
> >> --ofxml 10_AddTest_DOUBLE.output.xml
> >> run: '/var/www/opt/batchmake/bin/PixelCounter'
> >> '''/var/www/BASEDIR-batchmake/data/user_1/cart_2/10_AddTest_DOUBLE.mha'
> >> '10' --ofxml '10_AddTest_DOUBLE.output.xml'''
> >> Status: Finish: Execution time 315ms
> >>
> >> Unfortunately, trying to run ImageMathAdder (following rigorously the
> >> info
> >> in the bms file: i.e. png filenames in cart1 and cart2, etc.), something
> >> goes wrong: I cannot find any output image.
> >>
> >> I saw the following echo (only one line):
> >> Executing ...
> >>
> >> I tried to understand what happens checking in logs and in
> >> batchmake-basedir temp folder but without success (apparently no
> >> errors...)
> >>
> >> For ImageMath a different error is returned:
> >> Executing ...
> >> Error: SetAppOption: Cannot find parameter: GaussianBlur.
> >>
> >>
> >> Any advice?
> >>
> >> Thanks in advance...
> >>
> >> J
> >>
> >>
> >>
> >> > Hi J,
> >> >
> >> > Can you tell me more about what you are trying to accomplish with
> >> Midas?
> >> > I
> >> > might be able to give you some better advice if I know this.
> >> >
> >> >
> >> >
> >> > PixelCounter won't work with DICOM images, it is really more of an
> >> example
> >> > or demo application for testing out the BatchMake/Condor
> >> infrastructure.
> >> > This is useful to test before you do any other development, so you
> >> can
> >> be
> >> > sure the infrastructure works correctly.
> >> >
> >> >
> >> > Here is a link that covers some of this information. If anything is
> >> > unclear
> >> > or you would like more details in a particular area, please let us
> >> know
> >> as
> >> > we are always trying to improve the documentation.
> >> >
> >> >
> >>
> http://www.kitware.com/midaswiki/index.php/Plugins:BatchMake#Adding_a_BatchMake_script_to_the_Midas_BatchMake_Plugin
> >> >
> >> >
> >> >
> >> > For the PixelCounter application, you can use some sphere image files
> >> as
> >> > inputs. In the BatchMake build directory, under bin, if you
> >> configured
> >> > BatchMake with BUILD_EXAMPLES = on, you should have a GenerateSpheres
> >> > executable. If you run this, it will generate several sphere images
> >> files
> >> > for you. You can upload some of these to Midas, then create a cart
> >> and
> >> > add
> >> > the images to the cart. Use this cart as the input to the
> >> PixelCounter
> >> > application.
> >> >
> >> >
> >> >
> >> > Once you run the PixelCounter, you can use the Midas logs to help
> >> debug
> >> > problems. Also, look in the directory defined for
> >> >
> >> > $MIDAS_BATCHMAKE_TEMP_DIRECTORY
> >> >
> >> > in your BatchMake Midas plugin configuration file. Here you will see
> >> the
> >> > temporary directories associated with BatchMake processing in Midas,
> >> and
> >> > the
> >> > generated Condor output files there can help you debug further
> >> problems.
> >> >
> >> >
> >> >
> >> > I think the development version of Midas taken out of the Git
> >> repository
> >> > (2.8.1) is the best one to use for doing BatchMake plugin development,
> >> as
> >> > this area of Midas is under active development.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Sat, May 7, 2011 at 12:26 PM, J <j at dicomfly.com> wrote:
> >> >
> >> >> Hi Michael,
> >> >> batchmake plugin now seems to work even if PixelCounter
> >> example
> >> >> runs but does not produce any output: any idea? I tried to process
> >> DICOM
> >> >> images...
> >> >>
> >> >> Which is the better Midas version to use? (2.8.1, 2.9, ??)
> >> >>
> >> >> Thanks in advance
> >> >>
> >> >> J
> >> >>
> >> >>
> >> >>
> >> >> At 16.48 05/05/2011, you wrote:
> >> >>
> >> >> Hi J,
> >> >>
> >> >> First off, you will want to upgrade your BatchMake to the current
> >> >> development version to take advantage of integration with Midas and
> >> >> Condor:
> >> >>
> >> >> http://batchmake.org/batchmake/resources/software.html
> >> >>
> >> >> You can clone from git and then build the source using Cmake.
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> A couple things to think about before running any BatchMake
> >> applications
> >> >> from within Midas:
> >> >>
> >> >> -Is your condor grid status correctly viewable from the "Midas Grid
> >> >> Status"
> >> >> link of the BatchMake plugin?
> >> >> -Are all of the configuration settings correct (green colored) from
> >> the
> >> >> "Check Configuration" link of the BatchMake plugin?
> >> >>
> >> >>
> >> >>
> >> >> There should be more detailed error messages available in the Midas
> >> >> logs.
> >> >> Look in the MIDAS_INSTALL_DIR/midas/tmp/logs/error.log , this will
> >> help
> >> >> you
> >> >> with your specific problem.
> >> >>
> >> >>
> >> >> Let me know if this helps.
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Thanks,
> >> > Michael Grauer
> >> > R & D Engineer
> >> > Kitware, Inc.
> >> > 919 969 6990 x322
> >> >
> >>
> >>
> >>
> >
> >
> > --
> > Thanks,
> > Michael Grauer
> > R & D Engineer
> > Kitware, Inc.
> > 919 969 6990 x322
> >
>
--
Thanks,
Michael Grauer
R & D Engineer
Kitware, Inc.
919 969 6990 x322
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/midas/attachments/20110526/c477c729/attachment.html>
More information about the Midas
mailing list