Create a Single Node Slurm Cluster
In this blog, we'll guide you through the seamless upload of Flight solo image on alces cloud and then followed by the installation of a slurm standalone cluster installation. Our setup involves a single nodes that has slurm manager installed on it and that can be further used to execute HPC workloads.
Let's start with a step-by-step process, starting from uploading flight solo image, then creating virtual machines on the Alces Cloud platform, followed by verification of setup by running a simple HPC job.
Upload Flight Solo Image
- Download the Flight Solo OpenStack image here
- Upload the image by following the steps provided in the link
Launch the Instance
All the steps to launch and connection to instance is provided in link.
Create Slurm Cluster Using Flight Solo
-
Parse your node(s) with the command
flight hunter parse
.-
This will display a list of hunted nodes, for example
[flight@myinstance.novalocal ~]$ flight hunter parse Select nodes: (Scroll for more nodes) ‣ ⬡ myinstance.novalocal - 10.10.0.1
-
Select the desired node to be parsed with Space, and you will be taken to the label editor
Choose label: login-node.novalocal
-
Here, you can edit the label like plain text
Choose label: login1
Tip
You can clear the current node name by pressing Down in the label editor.
-
When done editing, press Enter to save. The modified node label will appear next to the ip address and original node label.
Select nodes: login-node.novalocal - 10.10.0.1 (login1) (Scroll for more nodes) ‣ ⬢ myinstance.novalocal - 10.10.0.1 (login1)
-
From this point, you can either hit Enter to finish parsing and process the selected nodes, or continue changing nodes. Either way, you can return to this list by running
flight hunter parse
. -
Save the node inventory before moving on to the next step.
Tip
See
flight hunter parse -h
for more ways to parse nodes.
-
Add genders
- Optionally, you may add genders to the newly parsed node. For example, in the case that the node should have the gender
cluster
andall
then run the command:flight hunter modify-groups --add cluster,all login1
SLURM Standalone Configuration
-
Configure profile
flight profile configure
- This brings up a UI, where several options need to be set. Use up and down arrow keys to scroll through options and enter to move to the next option. Options in brackets coloured yellow are the default options that will be applied if nothing is entered.
- Cluster type: The type of cluster setup needed, in this case select
Slurm Standalone
. - Cluster name: The name of the cluster.
- Default user: The user that you log in with.
- Set user password: Set a password to be used for the chosen default user.
- IP or FQDN for Web Access: This could be the public IP or public hostname.
- Cluster type: The type of cluster setup needed, in this case select
- This brings up a UI, where several options need to be set. Use up and down arrow keys to scroll through options and enter to move to the next option. Options in brackets coloured yellow are the default options that will be applied if nothing is entered.
-
Apply an identity by running the command
flight profile apply
, E.g.flight profile apply login1 all-in-one
Tip
You can check all available identities for the current profile with
flight profile identities
-
Wait for the identity to finish applying. You can check the status of all nodes with
flight profile list
.Tip
You can watch the progress of the application with
flight profile view login1 --watch
Success
Congratulations, you've now created a SLURM Standalone environment.
Run Slurm Workload
-
Create a file called
simplejobscript.sh
, and copy this into it:#!/bin/bash -l echo "Starting running on host $HOSTNAME" sleep 30 echo "Finished running - goodbye from $HOSTNAME"
-
Run the script with
sbatch simplejobscript.sh
, and to test all your nodes try queuing up enough jobs that all nodes will have to run. -
In the directory that the job was submitted from there should be a
slurm-X.out
whereX
is the Job ID returned from thesbatch
command. This will contain the echo messages from the script created in step 1