Matlab Distributed Computing Server on OctansCluster

Using the Matlab parallel and distributed computing environment on Octans.

There is 8 Matlab Distributed Computing Engine licenses on Octans. In order to use them, users have to meet the following requirements (assuming an MS windows environment on the client side):

Requirements:

1. Account on Octans(request one here)
2. Matlab and Matlab Distributed Computing Toolkit of the same version like Matlab Distributed Computing Server on Octans
3. Putty ssh client

1. Getting your account on OctansCluster
   Georgia State University researchers and affiliates may request their accounts online. Please specify Octans as the resource name and provide all requested information. If have questions or need assistance with you account request, please contact us via HelpDesk or directly using the ACS staff contact information.
2. Upgrading your Matlab installation
   In order to use Matlab DCS on Octans, you have to use the same version of Matlab. AdvancedCampusServices? ? upgrades MatlabDCS shortly after a new version is released. Users are notified via email. The current version (April 2009) is 2009a.
3. Customizing submission functions
   Overwrite lsfNonSharedSimpleSubmitFcn.m and runCmdOnCluster.m in your C:\Program Files\MATLAB\R2008a\toolbox\distcomp\examples\integration\lsf\nonshared directory with the files attached to this post. (If you need to submit parallel jobs or job arrays, please contact ACS as further customization of the default scripts might be needed.) Add the directory with the customized scripts to Matlab PATH (Start at the File menu.)
4. Replace the copyJobFilesIfFinished.m file with the patched version attached to this post. (Find the file using the which command: which copyJobFilesIfFinished.m ) Link to the patched file: copyJobFilesIfFinished.m: Fixed copy file function
5. Putty ssh client
   Get putty from http://www.putty.org/ - the windows installer file containing all main putty services is recommended. Follow the Putty documentation to install all services (plink, pscp ...) on your system. Add the putty directory to your PATH variable.

Set up:

1. Set up passwordless access to octans.gsu.edu
   1. Generate a public/private SSH key pair using the Putty puttygen utility or load and use an existing key. (Type puttygen on the command line and hit enter or start the utility from the Start menu.) To achieve passwordless access, use no key passphrase.
      Save your private key to a safe location on your PC. Please be carefull, whoever gets access to this key will also have access to your octans account.
      
      
   2. Login to octans.gsu.edu using your account information and add the public key you just generated to your authorized_keys file:
      Using putty from command line to connect to octans:
      

      putty you_user_name@octans.gsu.edu

      
      Supply your password. If your connection is refused, please contact ACS via HelpDesk to have your password reset. If your login is successful, execute the following statements:
      cd .ssh
      
      vi authorized_keys2
      
      
      Press "i" to enter the insert mode, navigate to the end of the text file, and paste your public key generated in step 1 at the end of the file starting at a new line. Press the "Esc" key. Press ":wq" and hit the enter key.
      
      Alternatively, you can transfer your public key file to octans via ftp and add it to the autorized_keys2 file like this:
      
      cat your_public_key_file >> ~/.ssh/authorized_keys2
      Regardless of which method you used, make certain that the permissions of the authorized_keys2 file are 644:
      
      chmod 644 ~/.ssh/authorized_keys2
   3. Set up a Putty session with the name octans.gsu.edu
      
      Setting up the Putty session:
      
      
   4. Make sure to specify the path to your private key in the Connection - SSH - Auth section and your octans login name in the Connection - Data - Auto-Login username section:
      • Specifying the path to user's private key:
        

• • Specifying auto-login name:
    

Test your passwordless conection

From within Matlab execute the following statements

>>!plink octans.gsu.edu

If you get an octans shell prompt, you have a successful connection. Type exit and hit enter.
If you are prompted for a password or get an error message, revisit the previous steps or contact ACS via HelpDesk.

>>!pscp -load octans.gsu.edu C:\tmp\a.txt octans.gsu.edu:/tmp/deletme.txt

This test assumes there is a C:\tmp\a.txt file on your computer. You may use an arbitrary small file. If the file gets copied over to octans with no password prompts, your putty settings are correct and you can start submitting matlab jobs using your putty ssh client.

Modify your Matlab environment

Change directory to $MATLABROOT\toolbox\distcomp\@distcomp\@abstractscheduler

Rename

pGetJobState.m to OLD_pGetJobState.m_OLD
pDestroyJob.m to OLD_pDestroyJob.m_OLD

Copy

$MATLABROOT\toolbox\distcomp\examples\integration\lsf\nonshared\windows\ pGetJobState.m
to
$MATLABROOT\toolbox\distcomp\@distcomp\@abstractscheduler

$MATLABROOT\toolbox\distcomp\examples\integration\lsf\nonshared\windows\ pDestroyJob.m
to
$MATLABROOT\toolbox\distcomp\@distcomp\@abstractscheduler

Start a MATLAB session.

The files in
$MATLABROOT\toolbox\distcomp\examples\integration\lsf\nonshared\windows
must be present on the MATLAB path. (File-SetPath-Add Folder)

Submitting a simple Matlab job to OctansCluster

Create Matlab data locations on both your PC and Octans. This example will use C:\mldata as the local data directory and /home/jklc/matlabdata as the remote Matlab directory (on Octans). The user name used here is jklc. Please adjust the test statements to reflect your user name and data locations.

Enter the following statements at the Matlab command prompt:

clusterHost = 'octans.gsu.edu';
remoteDataLocation = '/home/jklc/matlabdata';
sched = findResource('scheduler', 'type', 'generic');
set(sched, 'DataLocation', 'C:\mldata');
set(sched, 'ClusterMatlabRoot', '/usr/apps/matlab');
set(sched, 'HasSharedFilesystem', true);
set(sched, 'ClusterOsType', 'unix');
set(sched, 'GetJobStateFcn', @lsfGetJobState);
set(sched, 'SubmitFcn', {@lsfNonSharedSimpleSubmitFcn, clusterHost, remoteDataLocation});
job10 = createJob(sched);
createTask(job10, @sum, 1, {[1 1]});
createTask(job10, @sum, 1, {[1 2]});
submit(job10)
waitForState(job10)
results = getAllOutputArguments(job10)

Your results have been successfully retrieved from Octans if you see output like this:
results =

[2]
[3]

Submitting a Parallel Matlab Job to OctansCluster

Create Matlab data locations on both your PC and Octans. This example will use C:\mldata as the local data directory and /home/jklc/matlabdata as the remote Matlab directory (on Octans). The user name used here is jklc. Please adjust the test statements to reflect your user name and data locations.

Enter the following statements at the Matlab command prompt:

clusterHost = 'octans.gsu.edu';
remoteDataLocation = '/home/jklc/matlabdata';
sched = findResource('scheduler', 'type', 'generic');
set(sched, 'DataLocation', 'C:\mldata');
set(sched, 'ClusterMatlabRoot', '/usr/apps/matlab');
set(sched, 'HasSharedFilesystem', true);
set(sched, 'ClusterOsType', 'unix');
set(sched, 'ParallelSubmitFcn', {@lsfNonSharedParallelSubmitFcn, clusterHost, remoteDataLocation});
pj=createParallelJob(sched);
createTask(pj, @labindex, 1, {});
set(pj, 'MaximumNumberOfWorkers', 3);
set(pj, 'MinimumNumberOfWorkers', 3);
submit(pj)
waitForState(pj, 'finished', 60)
results = getAllOutputArguments(pj)

If everything works correctly, your results will look like this:

results =

[1]
[2]
[3]

We will soon provide a test job with a file dependency.

Accessing Matlab jobs after the client session was interrupted

sched=findResource('scheduler', 'type', 'generic');

4. Set the directory of your jobs as DataLocation?

set(sched, 'path to your local job directory');5. Search in a scheduler object for finished jobs

finished_jobs = findJob(sched, 'State', 'finished')finished_jobs contains the array of all your finished jobs.

6. Retrieve job results

results = getAllOutputArguments(finished_job())

Files

All customized files necessary for setting up the client environment are attached to the topic MatlabDCS

-- JaroKlc - 28 Apr 2009