How to combine outputs of multiple tasks performed using Matlab SGE?

Question

I have the following batch file launching some m-files ( main.m and fm which are scripts) 4 times (4 tasks).

#$ -S /bin/bash
#$ -l h_vmem=2G
#$ -l tmem=2G
#$ -cwd
#$ -j y

#Run 4 tasks where each task has a different $SGE_TASK_ID ranging from 1 to 4
#$ -t 1-4

#$ -N example


#Output the Task ID
echo "Task ID is $SGE_TASK_ID"

cat main.m f.m | matlab -nodisplay -nodesktop -nojvm -nosplash 

At the end I obtain 4 outputs that are example.o[...].1,example.o[...].2, example.o[...].3, example.o[...].4 . Each of them looks like

...
Task ID is ...

                        < M A T L A B (R) >
                                ...
>> >> >> >> >> >> >> >> >> >> >> 
output =

    4.0234   -3.4763

How can I combine these 4 outputs in a matrix 4x2 and save it?

Answer 1

You should save the relevant output from within fm using MATLAB's save or something similar.

If you use the -r flag to call main and f from the command line you can add a variable which will contain the task ID and you can then access that from within fm

matlab -nodisplay -nodesktop -nojvm -nosplash -r "main; ID = $SGE_TASK_ID; f; exit"

Then within fm

% You theoretically generate some numeric result
result = rand(1, 2);

filename = sprintf('Result.%d.mat', ID);
save(filename, 'result')

This will save Result.0.mat , Result.1.mat etc.

Alternately, you could modify fm such that it loads the data from the file, appends to it, and re-saves it every time

result = rand(1,2);

filename = 'Results.mat';

% If this is the first task, then create a new file, otherwise append to the old
if ID == 1
    data = result;
else
    tmp = load(filename, '-mat');
    data = tmp.data;
    data(ID,:) = result;
end

save(filename, 'data')