BatchMake: Difference between revisions

From KitwarePublic
Jump to navigationJump to search
Line 11: Line 11:
* Cross platform
* Cross platform


= BatchMake =
==List of available commands:==


*'''''AppendFile''''' Add a string to the end of a file
*'''''CloseTCPSocket''''' Close a TCP/IP socket 
*'''''DashboardExperiment''''' Specify the encrypted name of the experiment to report on 
*'''''DashboardHost''''' Specify the hostname of the central database manager 
*'''''DashboardNotify''''' Specify the hostname of the central database manager 
*'''''DashboardSend''''' Send the results of a program to the central database for data collection 
*'''''DeleteFile''''' Delete a file on disk
*'''''Echo''''' Print messages on the standard output
*'''''ExtractSlice''''' Extract a slice from a 3D image volume
*'''''ExtractString''''' Extract a string from a string chain
*'''''ForEach''''' Create a "for" loop 
*'''''GetParam''''' Access directly a parameter from a sequence 
*'''''If''''' Defines a conditional statement 
*'''''Inc''''' Increment a number by a certain amount 
*'''''Include''''' Include another batchmake script 
*'''''Int Convert''''' a variable to integer
*'''''ListDirInDir''''' List all the directories in a specified directory 
*'''''ListFileInDir''''' List all the files in a specified directory 
*'''''OpenTCPSocket''''' Open a TCP/IP socket 
*'''''Randomize''''' Create a random number 
*'''''Run''''' Execute a program in a thread 
*'''''Sequence''''' Create a sequence of numbers 
*'''''Set''''' Assign a value to a variable
*'''''SetApp''''' Assign a variable to an Application defined by the Application Wrapper 
*'''''SetAppOption''''' Assign a value to a predefined option of a program 
*'''''SendTCP''''' Send a message via TCP socket 
*'''''Sin''''' Evaluate the sin() function 
*'''''WriteFile''''' Write variables into a file
==How to use each command:==
===AppendFile - Add a string to the end of a file===
'''AppendFile( <filename> <value1> <value2> ... )'''
The first parameter is the name of the file and all the others parameters will be added at the end of this file.
'''Example :'''
sequence(seq 1 3 1)
foreach(value ${seq})
appendFile('C:/bmtest.txt' 'value: ${value}\n')
endforeach(value)
This script will add the following strings at the end of the file entitled "C:/bmtest.txt" :
"value: 1"
"value: 2"
"value: 3"
===CloseTCPSocket - Close a TCP/IP socket===
'''CloseTCPSocket( <SocketName> )'''
'''Example :'''
CloseTCPSocket(MySocket)
===DashboardExperiment - Specify the encrypted name of the experiment to report on===
'''DashboardExperiment( <encryptedExperiment> )'''
The encryptedExperiment is given when creating a new experiment online (this requires a login/pass)
'''Example :'''
DashboardExperiment( 7YF04vELWW4m1B88t)
===DashboardHost - Specify the hostname of the central database manager===
'''DashboardHost( <hostname> )'''
'''Example :'''
DashboardHost(www.kitware.com)
===DashboardNotify - Add a new string to the central database===
'''DashboardNotify( <message> )'''
'''Example :'''
DashboardNotify('End of first experiment')
===DashboardSend - Send the results of a program to the central database for data collection===
'''DashboardSend( <methodid> <value1> <value2> )'''
'''Example :'''
DashboardSend(1,${value1},${value2})
===DeleteFile - Delete a file on disk ===
'''DeleteFile( <filename> )'''
'''Example :'''
DeleteFile('myfile.txt')
===Echo - Print messages on the standard output===
'''Echo( <value1> <value2> ... )'''
'''Example :'''
sequence(seq 1 2 1)
sequence(seq2 1 2 1)
foreach(value ${seq})
foreach(value2 ${seq2})
Echo(value: ${value} ${value2})
endforeach(value2)
endforeach(value)
This script will print the following lines on the screen :
value: 1 1
value: 1 2
value: 2 1
value: 2 2
===ExtractSlice - Extract a slice from a 3D image volume===
'''ExtractSlice( <3D image filename> <2D image filename> <orientation> <slice> )'''
'''Example :'''
ExtractSlice('head.mha','slice.jpg',0,23)
This command extracts the saggital slice #23 from the volume head.mha and write the extracted slice to slice.jpg
===ExtractString - Extract a string from a string chain===
'''ExtractString( <output> <input> <lenght> [FROMEND]  [KEEPEND] )'''
The parameter <output> will contain the <lenght> first characters of <input> if FROMEND is not set. With FROMEND the <lenght> last characters won't be copied. if KEEPEND is specified then the end of the string is returned.
'''Example :'''
set(example 'hello')
extractstring(result ${example} 1)
extractstring(result2 ${example} 2 FROMEND)
extractstring(result3 ${example} 2 KEEPEND)
extractstring(result4 ${example} 2 FROMEND KEEPEND)
echo(${result})
echo(${result2})
echo(${result3})
echo(${result4})
This script will print the following lines on the screen :
h
hel
llo
lo
===ForEach / EndForEach  - Create a "for" loop===
'''ForEach( <variable> <sequence> )'''
'''EndForEach( <variable> )'''
During the ith iteration, <variable> contains the ith parameter of <sequence>. The EndForEach command allows to define the end of the loop.
'''Example :'''
set(seq a b c d e f)
ForEach(letter ${seq})
echo(${letter})
EndForEach(letter)
This script will print the following lines on the screen :
a
b
c
d
e
f
===GetParam - Access directly a parameter from a sequence===
'''GetParam( <new variable> <sequence> <number> [<number2>] ...  )'''
<number> corresponds to the argument of <sequence> to copy into <new variable>. We can copy as many arguments of <sequence> as we want adding at the end the rank of the argument in <sequence>.
'''Example :'''
sequence(seq 1 7 2)
echo(${seq})
GetParam(new1 ${seq} 0)
GetParam(new2 ${seq} 3)
echo(${new1})
echo(${new2})
GetParam(new ${seq} 0 2)
echo(${new})
This script will print the following lines on the screen :
1 3 5 7
1
7
1 5
===If / EndIf - Defines a conditional statement====
'''If( <variable1> == or <= or >= or != <variable2> )'''
'''EndIf( <variable> )'''
'''Example :'''
set(var1 2)
set(var2 4)
If(${var1} <= ${var2})
echo(${var1} <= ${var2})
EndIf(${var1})
set(char1 test)
set(char2 test)
If(${char1} == ${char2})
echo(${char1} == ${char2})
EndIf(${char1})
This script will print the following lines on the screen :
2 <= 4
test == test
===Inc  Increment a number by a certain amount===
'''Inc( <variable> <number> )'''
'''<variable> = <variable> + <number>'''
'''Example :'''
Set(value 2)
Inc(${value} 5)
echo(2 + 5 = ${value})
This script will print the following lines on the screen :
2 + 5 = 7.000000
===Include - Include another batchmake script===
'''Include( <ScriptFilename> )'''
'''Example :'''
#Script1 : F:\Script1.bms
echo('Script1')
#Script2 : F:\Script2.bms
echo('Script2')
#Script3 :
Include(F:\Script1.bms)
Include(F:\Script2.bms)
When we execute Script3 the following lines will be printed on the screen :
Script1
Script2
===Int - Convert a variable to integer===
'''Int( <variable> )'''
'''Example :'''
Set(value 2)
Inc(${value} 5)
Int(${value})
echo(2 + 5 = ${value})
This script prints the following lines on the screen:
2 + 5 = 7  ( instead of 7.000000 if we Int() is not used)
===ListDirInDir - List all the directories in a specified directory===
'''ListDirInDir( <dirlist> <directory> [ *.* ] )'''
<dirlist> contains the name of all the subdirectories included in <directory>. We can also list only the subdirectories with a specific string in their names : for instance if 'doc*' is added at the end of ListDirInDir only the subdirectories started wtih 'doc' will be listed.
'''Example :'''
Structure of the Hard Drive :
  F:\
    Example
          number1
          number2
          number3
set(dir F:\Example)
ListDirInDir(example ${dir})
echo(${example})
This script will print the following lines on the screen:
number1/ number2/ number3/
===ListFileInDir - List all the files in a specified directory===
'''ListFileInDir( <filelist> <directory> [ *.* ] )'''
<filelist> contains the name of all the files included in <directory>. We can also list only the files with a specific string in their names : for instance if *.txt' is added at the end of ListFileInDir only the files wtih the '.txt' extension will be listed.
'''Example :'''
Structure of the Hard Drive :
  F:\
    Example
        number1.txt
        number2.txt
        number3.txt
set(dir F:\Example)
ListFileInDir(example ${dir})
echo(${example})
This script will print the following lines on the screen:
number1.txt number2.txt number3.txt
===OpenTCPSocket - Open a TCP/IP socket===
'''OpenTCPSocket( <SocketName> <IPAddress> <Port> )'''
'''Example :'''
OpenTCPSocket(MySocket,10.0.0.1,8081)
===Randomize - Create a random number===
'''Randomize( <variable> <uniform | gaussian | categorical> <min> <max> )'''
<variable> contains the result. The distribtion can be set thanks to one of these 3 words : uniform, gaussian or categorical. The range an be set thanks to <min> and <max>.
'''Example :'''
Randomize(result uniform 0 1)
echo(${result})
This script will print on the screen a random number between 0 and 1 following a uniform distribution.
===Run - Execute a program in a thread===
'''Run( <output variable> <'program.exe param1 param2 ....'> )'''
The program called 'program.exe' will be launched with the parameters 'param1 param2' ...
Note that the Application Wrapper can be used to defined the parameters as names.
'''Example :'''
Run( output 'MyProgram.exe -in input -out output' )
This script will run the program called MyProgram.exe with the parameters input and output.
===SendTCP - Send a message thru a TCP socket===
'''SendTCP( <SocketName> <message> )'''
The socket should be open using OpenTCPSocket and closed via CloseTCPSocket
'''Example :'''
SendTCP(MySocket 'my message to send')
===Sequence - Create a sequence of numbers===
'''Sequence( <variable> <From> <To> <Step> )'''
A sequence starting at <From> ending at <To> with a step of <Step> is generated and stored in <variable>.
'''Example :'''
sequence(seq 0 10 2)
echo(${seq})
This script will print the following lines on the screen :
0 2 4 6 8 10
===Set - Assign a value to a variable===
'''Set( <variable> <name1> <name2> ...  )'''
'''Example :'''
Set(example name1 name2 name3)
echo(${example})
This script will print the following lines on the screen :
name1 name2 name3
===SetApp - Assign a variable to an Application defined by the Application Wrapper===
''''SetApp( <variable> <application> )
Before using this command we must declare the program we want to execute in Application Wraper ( clic on the Application Wrapper button -> new ).
'''Example :'''
SetApp( prog @MyProgam)
Run(output ${prog})
 
This script will run the software called MyProgram.
'''NOTE''' : When the @ key is pressed the list of the programs known by Application Wrapper is displayed.
===SetAppOption - Assign a value to a predefined option of a program===
'''SetAppOption( <application.option> <value> )'''
Before using this command we must use the SetApp command to set an application.
Warning : We can only use this command if the software use the MetaCommand library to parse its arguments.
'''Example :'''
SetApp(prog @MyProgam)
SetAppOption( prog.input MyInput )
SetAppOption( prog.output MyOutput )
Run(output ${prog})
This script is equivalent to : MyProgram.exe -in MyInput -out MyOutput.
'''NOTE''' : When the "." key is pressedafter the nam eof the application the list of the option available is displayed.
===Sin - Evaluate the sin() function===
'''Sin( <output variable> <input> )'''
The input number must be given in radian.
'''Example :'''
Sin(result 1.57)
echo(${result})
This script wiill print the following lines on the screen :
1.000000
===WriteFile - Write variables into a file===
'''WriteFile( <filename> <variable1> <variable2> ...  )'''
'''Example :'''
sequence(seq 1 10 1)
set(end 'end')
WriteFile(F:\example.txt ${seq} ${end})
This script will print the following lines into the file : F:\example.txt :
12345678910end


= Distributed Computing =
= Distributed Computing =


= BatchBoards =
= BatchBoards =

Revision as of 15:13, 15 October 2007

BatchMake is a cross platform tool for batch processing of large amount of data. BatchMake can process datasets locally or on distributed systems using Condor.

Common Features:

  • BSD License
  • CMake like scripting langage
  • Distributed Scripting via Condor
  • Central remote website for online statistical analysis
  • User Interface using FLTK
  • Cross platform


Distributed Computing

BatchBoards