Batchmake tutorial

< Main BatchMake

Tutorial: Getting Started with BatchMake

Introduction

This section explains the purpose of the BatchMake tool, and describes the screen layout. This section is organized as a series of questions and answers about the general capabilities and structure of the BatchMake tool.

Running an application with BatchMake

To begin, we are going to do a very simple example. It consists of listing all the files in a directory. First, you have to indicate the directory that you want to list. You do this with the command 'Set'. The first argument is the name of the variable, and the second is the path of your directory.

Set(directory 'C:\Matthieu\BatchMake\Examples')

After that, BatchMake knows your directory, and so, it can list all files.

ListFileInDir(Files ${directory})

The last thing to do is print the names of the files. It is easy with the commad 'echo' You can print the list which is "directory", or you can print each file.

foreach(file ${Files})
echo(${file})
endforeach(file)
echo(${Files})

If you want to print each file, you need to create a loop bracketing the loop commands with 'foreach...endeach'

Complete Example : Converter

Converter.cxx

This example runs an application which converts a tif image into an mha one.

First you have to use MetaCommad class in order to receive parameters from the commad line. If you don't know how to use MetaCommand, you can find some help at the end of this tutorial.

We have this code :

MetaCommand command;
command.SetName("Converter");
command.SetVersion("1.0");
command.SetAuthor("Kitware Inc");
command.SetDescription("Convert tif->mha");
command.AddField("InputImage","InputImage",MetaCommand::STRING);
command.AddField("OutputImage","OutputImage",MetaCommand::STRING);
if(!command.Parse(argc,argv))
{
return 1;
}

std::string InputImage = command.GetValueAsString("InputImage"); 
std::string OutputImage = command.GetValueAsString("OutputImage"); 

reader->SetFileName(InputImage);
writer->SetFileName(OutputImage);

writer->SetInput(reader->GetOutput());
writer->UseCompressionOn();
writer->Update();

return 1;
}

At the beginning of our code, we fill insome fields like the name, the version etc... The most important fields to add, are the input image field and the output image one. The first parameter is the name of the field, the second its "tag", and the last one, its type.

Therefore, our application have one input (InputImage), and one output (OutputImage) which will be the two arguments of the program.

Now, we can run our program with BatchMake.

Converter.bms

Before beginning to write your BatchMake code, we need to specify a new application in the wraper

Create a new application with the button "New" :

Use the button "Browse and Run" to select your application (Converter.exe), and you will see some information about it, like metacommand filelds.

At the bottom of the image, you can see the Command line of the application. The two <string> indicate you have to specify two arguments to run Converter => the input image and the ouput one. Now you can click "OK"

"Converter" application is now known by BatchMake.

Here is the code of our BatchMake file :

SetApp(converter @Converter)       #"converter" is the variable which defines the application

When you type @, BatchMake shows you all the applications it knows.

Set(outputdir 'C:/Example/TifImages')       #Here, we define the output directory where new images will be stored
MakeDirectory(${outputdir})                 #If the output directory doesn't exist, it will be created

Set(inputdir 'C:/Example/MhaImages')      #Here, we define the intput directory where tiff images are
ListFileInDir(files ${inputdir})          #Then we list all files which are in that directory

foreach(i $(files))        #For each file of the directory inputdir

Set(inputimage ${inputdir}/${i})                      #We create a new variable which contains the inputimage file.
SetAppOption(converter.InputImage ${inputimage})      #We sepcify what is the input image for the first field

GetFilename(j ${i} NAME_WITHOUT_EXTENSION)
Set(outputimage ${outputdir}/${j}.mha)                 #We create a new variable which contains the outputimage file.
SetAppOption(converter.OutputImage ${outputimage})     #And the output image for the second field of Metacommand

echo(${converter})          #We can see the line command with echo
Run(output ${converter})    #Running the application...

endforeach(image)

Tutorial: Sending Data to the central DataBase

Sending results:

Firstly, we are going to send the name of our inputimage and our ouputimage.

To send data to the central DataBase of BatchMake, you have to register you on the website :

http://insight-journal.org/batchmake/index.php

When it's done, log you on the DataBase, go to "My Profile", and click on the button "Generate Key" and remember this one.

Now, you can add this code at the begining of your script :

DashboardHost(http://www.insight-journal.org/batchmake)
DashboardUser('Your_FirstName Your_LastName')
DashboardKey('Your_Key')

#The name of your project must be created by a super-admin of BatchMake : Ask him if you want to create a new project
CreateExperiment(exp 'Your project name' 'Title of your new experiment' 'This is a description for this experiment')

# We add a method 
CreateMethod(ConvertMeth exp 'Converter' 'Convert a mha image into a mha compress')

The following cammand should be add after that the variables are seted :

# We add two inputs that will be displayed in the DataBase
AddMethodInput(InputImage ConvertMeth 'InputImage Name') #InputImage is the same variable that we used before.
AddMethodInput(InputImage ConvertMeth 'OutputImage Name') #OutputImage is the same variable that we used before.

# All the path will be displayed, but if you prefre display only the imagename => replace InputImage and OutputImage by i.

The entries of the methods are now created, you can send your new method. with the command :

DashboardSend(ConvertMeth)

The Dashboard has another option that is very interisting : You can display an image in the DataBase.

The only condition is that your image is a png.

But maybe, you would like to display a slice from a 3D image (like a mha) : you can do this too with BatchMake. You just have to write the following command :

 Set(InputLiver 'C:/Matthieu/Tests/Liver.mha')
 #We take the first slice after the middle of the image (2 = axial axe)
 ExtractSlice('${InputLiver}' '${InputLiver}1.png' 2 1 FROM_MIDDLE) 

Liver.bms

The following lines will display diferent slices of the liver image in the database.

Set(InputLiver 'C:/Matthieu/Tests/Liver.mha')
sequence(seq 0 20 5)

foreach(x ${seq})
ExtractSlice('${InputLiver}' '${InputLiver}${x}.png' 2 ${x} FROM_MIDDLE) 
SetIdealOutput(slice '${InputLiver}${x}.png')
AddMethodInput(x ConvertMeth 'Slice #')
AddMethodIdealOutput(slice ConvertMeth 'Slice' png)
DashboardSend(ConvertMeth)
endforeach()

Tutorial: Using BatchMake inside a C++ application

BatchMake is a standalone application. It can be run as a command line but also has a front-end GUI. But BatchMake is also a collection of C++ classes that can be used directly inside your C++ applications. This tutorial shows how run a script from your application and how to access the values of your script variable. You first have to include the BatchMake parser. Make sure you link the BatchMake libraries with your project.

#include "bmScriptParser.h"

Then you should instantiate a ScriptParser object

bm::ScriptParser m_Parser;

Optionally set the wrapped application directory or/and the application path

m_Parser.LoadWrappedApplication( "c:/my_apps/" );
m_Parser.SetBatchMakeBinaryPath( "c:/my_apps/" ); 

And finally compile and run the script. Execute() returns true if it successfully compiled and run with no errors, false otherwise.

m_Parser.Execute( "my_script.bms" );

After the script is executed, you can retrieve the values of your script variables.

std::vector<BMString> value = m_Parser.GetScriptActionManager()->GetVariable( "my_variable" );

Tutorial: Adding a test with CTest

First you have to write a test script (.bms). You then need an executable that takes your .bms file and executes it. This is done by bmActionsTest in Testing/Code/bmActionsTest.cxx.
In your testing directory, open your CMakeLists.txt file and add the lines:

SET(BATCHMAKE_TESTS ${CXX_TEST_PATH}/BatchMakeTests)
SET(TEST_DATA ${WhereYourBMSFilesAre} )
ADD_TEST(MyActionsTest   ${BATCHMAKE_TESTS} ${WhereActionTestIs}/ActionsTest ${TEST_DATA}/MyActionsTest.bms)
ADD_TEST(MyActionsTest2  ${BATCHMAKE_TESTS} ${WhereActionTestIs}/ActionsTest ${TEST_DATA}/MyActionsTest2.bms)
...

You can give a look at the Testing/Code/CMakeLists.txt file; some BatchMake tests are there.

Tutorial: Running BatchMake scripts within Slicer

You can run BatchMake scripts in Slicer. The idea is to generate a script with the inputs of the Slicer module, compile the script and run it (see Tutorial: Using BatchMake inside a C++ application
You can take one of the three existing modules in Slicer as example. They are located in Slicer3/Applications/CLI/BatchMakeApplications.

If you want to use Condor with BatchMake, you have to have condor installed on your machine. You can either create a condor user on your machine, either be a user with condor privileges. In both cases, the user that run Slicer must have the condor privileges. In the first case, supposing that 'condor' is the name of the condor user, launch Slicer by:

$ su condor -m -c where/slicer/is/Slicer3

Once Slicer is started, select the BatchMake module from the module list ( in BatchProcessing ) and make sure you check the "condor" checkbox in the Advanced properties.
It is recommended to set DataDirectory(), GridExecutableDirectory() and GridTransferFile() in your script. Transfering the applications might lead to some path and dependency problems. You probably would like to have the same address for your executables: use GridExecutableDirectory( ) and GridTransferFiles() with one of the following values, NONE, INPUT_FILES or OUTPUT_FILES. You would also probably like to share the directory where all your input data are with all of your condor machines: use DataDirectory( ) where the path can be a network path accessed by all the condor machines.

Note: slicer modules can be run from Slicer but also from the command line.
Example of EMSegmenter launched from the command line:

./EMSegmentBatchMakeModule --mrmlSceneFileName /mnt/midas/batchmake_data/images/EMSegmenterInputs/VolumeData_Input/EMSeg-Brain-MRT1T2.mrml /mnt/midas/batchmake_data/images/EMSegmenterInputs/VolumeData_Input --dirmask Target_NewPatient* --t1m *1*.nhdr --t2m *2*.nhdr /mnt/midas/batchmake_data/images/EMSegmenterOutputs --condor