Larger computing clusters consist of many nodes where each node is made up of many CPU cores. Moreover, special software such as Mathematica is often available on the cluster. Users request these resources by logging into a head node and submitting batch jobs to the cluster manager. The job executes when the resources become available. Two common cluster managers include TORQUE and Slurm.
Parallelization in Mathematica uses the hub-and-spoke model where a controlling kernel manages a number of subordinate kernels (subkernels). In a cluster environment, the client runs the controlling kernel and the hosts provide subkernels. The cluster manager determines which node acts as the client and which nodes act as hosts.
The benefits of running Mathematica on a cluster are twofold: The number of available CPU cores, even on a single node, is usually more than on a desktop computer. Also, the speed of each CPU core is individually faster than those in a desktop computer.
Running a remote Mathematica front end
A remote front end requires the user to remain in control of the job’s resources. This is known as an interactive session.
Though there is usually a benefit in CPU speed over using a desktop computer, the front end via an interactive session is slower than running a local version of the front end. This is because the front end runs on the cluster while the interface is forwarded to the remote user’s computer.
An interactive session is not intended to run CPU-intensive calculations, but rather it is used to test and diagnose code. It is typical to request only one node’s resources.
- Log in to the head node via SSH with X-windows forwarding enabled.
- Launch an interactive session, requesting all of the resources on a single node.
- Start a Mathematica session.
Running a remote Mathematica script
It is assumed that
- you are familiar with launching remote subkernels
- the cluster is a Unix environment
- the cluster uses a cloned file system such that Mathematica is run by the same executable on all nodes
If any of the above assumptions are not true, then the following will need to be modified, but the general outline remains:
- query the system to find the names of the nodes that are assigned to the job, and how many cores per node are available
- manually launch remote kernels
For example, on a Torque managed cluster, in a Mathematica or Wolfram Language script you would have:
At this point, parallel functions can now be used that utilize the whole set of available resources. When the parallel code is complete, it is good practice to close the Wolfram kernels.