Submit job to remote server¶

Documentation¶

http://www.vistrails.org/usersguide/v2.2/html/tej.html

http://www.vistrails.org/usersguide/v2.2/examples/tej-primes.vtl

Possible run problem¶

Follow the documentation, download and run tej-primes.vt in VisTrails. An error occurs at first.

This appears to be a problem with pkg_resources.resource_filename in the Mac binary, because it uses py2app. We will fix this for the next release, in the meantime you can try to extract tej so that it can be reached by resource_filename using:

$ cd /Applications/VisTrails/VisTrails.app/Contents/Resources/lib/python2.7
$ unzip site-packages.zip tej/*

Stage folder¶

After your job runs, a stage folder will be created. e.g when I run this job in my local computer

/Users/jinghuage/.tej/jobs/vistrails_module_17ccbae56272c17a06d68654ddab9307b677958e/stage

If job is running remotely, this stage folder will be created in remote computer’s file ystem.

Running a script¶

The script file can be used when invoking the Wolfram Language kernel from the command line, assuming that the MathKernel or math executables are on your path and can be found.

$ MathKernel -script file.m <argument-list>

A shell script to run Mathematica script findBestShift.m¶

DATA_DIR="/Users/jinghuage/Documents/VisTrails/bullet-reconstruct-vistrails"
SCRIPTS_DIR="/Users/jinghuage/Documents/VisTrails/bullet-reconstruct-vistrails/m"

/Applications/Mathematica.app/Contents/MacOS/MathKernel \
-script $SCRIPTS_DIR/findBestShift.m \
$DATA_DIR/sinograms/    \
100 11

Save this content to a file “runscript” and make it executable

Read parameters from argument-list¶

1
2
3
4

   (* input *)
   sinoDir = ARGV[[1]];
   shiftSlice = eval[ARGV[[2]]];
   shift = eval[ARGV[[3]]];

Computation¶

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

   (* find out sinogram numberFiles, numberColumns, numberAngles *)
   filenamesSinograms = FileNames["sino_*", sinoDir];
   numberFiles = Length[filenamesSinograms];
   Print[numberFiles];

   sinogram = Import[filenamesSinograms[[1]], {"Datasets", "sinogram"}];
   {numberColumns, numberAngles} = Dimensions[sinogram];
   Print[Dimensions[sinogram]];


   (* shift using ImagePad and reconstruction of a slice *)
   sinogram = Import[filenamesSinograms[[shiftSlice]], {"Datasets", "sinogram"}];
   sinogram = Image[sinogram, "Real"];

   bestShift = {shift, 0};
   gPad = InverseRadon[ImagePad[sinogram, {{0, 0}, bestShift}, 0], numberColumns];

Output¶

Normal output : save to working data directory

output need to transfer back to local: save to current job directory ./

Which is a running job’s Stage folder

1
2

   padCOR = StringJoin["./", "padCOR.png"];
   Export[padCOR, gPad, "PNG"];

Parameters(String) modules¶

input port: value:

datadir: /melete-nas01/lsu/scratch/BunnyData/feetdown_Hgrating/
mdir: /melete-nas01/lsu/scratch/BunnyData/scripts/
mathKernel: /Applications/Mathematica.app/Contents/MacOS/MathKernel
shiftSlice: 100
shift: 11
appdata: sinograms/
appscript: findBestShift.m

CreateRunScript(PythonSource) module¶

Configure the pythonsource:

workpath = 'DATA_DIR='
workpath += datadir
workpath += '\n'
workpath += 'SCRIPTS_DIR='
workpath += mdir
workpath += '\n'

appscript = "$SCRIPTS_DIR/"+appscript
appdata = "$DATA_DIR/"+appdata
plist = [mathkernel, '-script', appscript, appdata, shiftSlice, shift]
cmd = " ".join(map(str, plist))
print cmd

output = self.interpreter.filePool.create_file()
with open(output.name, 'w') as fp:
    fp.write(workpath)
    fp.write("\n")
    fp.write(cmd)

self.set_output('runscript', output)

Queue module¶

Set hostname and username inputs

SubmitJob module¶

Configure this module to have a file “runscript” input, connect to “CreateRunScript” output.

Set “script” input to a name, such as “runscript”. This script will later be copied to running job’s Stage folder

DownloadFile module¶

Specify “filename” input, to indicate which file to download. The file must be present in running job’s Stage folder

ImageViewerCell module¶

connect to downloaded file and show in spreadsheet cell

More about spreadsheet cells:

http://www.vistrails.org/usersguide/v2.2/html/spreadsheet.html#sending-output-to-the-spreadsheet