Text size
Line height
Text spacing
Section outline
-
-
Exercise 1: Tools for working with a cluster
a) Registration on the cluster
Sign in to the cluster. If necessary, we install the appropriate program for access to the remote computer via the SSH protocol.b) File transfer programs
Install a program for transferring files between a personal computer and a cluster and connect to the cluster with it.
Create a text file file-or.txt on the desktop of a personal computer and write a few words in it. Create a new exercise1b folder on the cluster and upload the file-or.txt file to it. Open and edit the file on the remote computer (change the text) and save the change. Then we apply the file to the file-g.txt and transfer it back to the desktop of the personal computer. Check if any changes have been made to it.
Using a file transfer program on a remote computer, create a text.txt file in the exercise1b folder, open it, type a few words, and save it.c) Work directly on the cluster
In the terminal:- use the cd command exercise1b to move to the folder from the previous exercise,
- use the pwd command to check which folder you are in,
- use the cat text.txt command to look at the contents of the file,
- if desired, we can use the nano text.txt command to open the file with the editor and
- finally, with the cd .. command, return to the basic folder of the user account.
-
Exercise 2: Cluster information
a) Print cluster information
Run the commands from the chapter on the sinfo command and review the printouts. Remember the name of one of the computational nodes (column NODELIST), we will need it in Exercise 2c.b) Printout of information on jobs in line
Run the commands from the squeue command section and review the printouts. In the printout we get after calling the squeue command without switches, remember the user and the identifier (JOBID) one of his jobs. The first information will be needed when calling the squeue command with the switches in this exercise, and the second information in Exercise 2c for more detailed information).c) Printout of more detailed information
Run the commands from the scontrol command section and review the printouts. For individual commands, we use the node name, user and job identifier from the previous tasks, which we memorized from exercises 2a and 2b. -
Exercise 3: Creating and running cluster jobs
a) The srun command
Start the following operations:- Four instances of the hostname program on one computing node on the fri reservation. Set the job name to my_job. Use the srun command.
- Two instances of the hostname program on each of the two computing nodes on the fri reservation. Set the amount of memory per processor to 10 MB. Use the srun command.
b) The sbatch and scancel commands
Using the sbatch command, run the following operations:- Run four instances of the hostname program on a single node with the sbatch command. Follow the example in chapter Starting jobs on a cluster, sbatch command.
- Run the sleep 600 instance with the sbatch command. The program will wait 600 seconds after startup and finish without printing. We use the reservation fri. As a basis for our script, we use the example presented, in which we extend the time limit accordingly. Wait for the job to start (state R), then terminate it prematurely with the scancel command.
c) The salloc command
With the salloc command, we occupy enough capacity for one task (one processor core) on the fri reservation. Connect to the appropriate node with the ssh command and run the cat /proc/cpuinfo command on it, which tells us the details of the built-in processor. When we are done, we release the occupied capacities. -
Exercise 4: Video Processing
In this exercise, we use the examples presented in the chapter on video processing.
a) Using ffmpeg
Load the ffmpeg module and run the ffmpeg -version command, first on the input node and then on the computing node.b) Convert the record
Upload a short videollama.mp4to the cluster, convert it to a mov format, and transfer the output file back to our computer. There we watch it in our favorite player.c) Use of filters
- The image from the previous exercise is processed with the help of ffmpeg with one of the listed filters. Be sure to run the transaction on the computing node (use the srun command and the appropriate reservation, if you have one).
- Repeat the process with another filter, specifying another output file. Transfer the processed images back to our computer and view them.
-
Exercise 5: Parallel video processing
In this exercise, we use the examples presented in the chapter on parallel video processing.
a) Video segmentation
Download thebig-buck-bunyvideo to your PC and transfer it to the NSC cluster. Then cut it into pieces lasting 80 seconds each. Although this operation is not demanding, as it only copies parts of the snapshot to new files, we still run it on the computing node with the srun command. Use the reservation fri. Remember to load the FFmpeg module.b) Processing
Increase the resolution by a factor of 1.5 for the pieces of video we created in the previous task. We use the sbatch command and array jobs. We can help ourselves with the script ffmpeg3.sh.c) Assembling
We combine the pieces from exercise 5b into one clip, download it to our computer and watch it in the player. -
Exercise 6: Processing videos with containers
In this exercise, we use the examples presented in the chapter on the use of graphical process units.a) Prepare a container for graphic processing units
Prepare the container for graphic processing units according to the instructions in the Use of graphical process units chapter.b) Processing
Run the ffmpeg program using a graphical processing unit. Increase the resolution of bbb.mp4 by a factor of 1.5. We use the sbatch command and job strings. More detailed instructions can be found in chapter Video processing on a graphics processing unit.
-