Summary and Setup

This workshop is an introduction to using high-performance computing systems effectively. We can’t cover every case or give an exhaustive course on parallel programming in just two days’ teaching time. Instead, this workshop is intended to give students a good introduction and overview of the tools available and how to use them effectively.

Prerequisites

Command line experience is necessary for this lesson. We recommend the participants to go through shell-novice, if new to the command line (also known as terminal or shell).

By the end of this workshop, students will know how to:

  • Identify problems a cluster can help solve
  • Use the UNIX shell (also known as terminal or command line) to connect to a cluster.
  • Transfer files onto a cluster.
  • Submit and manage jobs on a cluster using a scheduler.
  • Observe the benefits and limitations of parallel execution.

Getting Started

To get started, follow the directions in the “Setup” tab to download data to your computer and follow any installation instructions.

Note that this is the draft HPC Carpentry release. Comments and feedback are welcome.

For Instructors

If you are teaching this lesson in a workshop, please see the Instructor notes.

Setup

There are several pieces of software you will wish to install before the workshop. Though installation help will be provided at the workshop, we recommend that these tools are installed (or at least downloaded) beforehand.

  1. A terminal application or command-line interface
  2. A Secure Shell application

Bash and SSH

This lesson requires a terminal application (bash, zsh, or others) with the ability to securely connect to a remote machine (ssh).

Where to type commands: How to open a new shell

The shell is a program that enables us to send commands to the computer and receive output. It is also referred to as the terminal or command line.

Some computers include a default Unix Shell program. The steps below describe some methods for identifying and opening a Unix Shell program if you already have one installed. There are also options for identifying and downloading a Unix Shell program, a Linux/UNIX emulator, or a program to access a Unix Shell on a server.

Unix Shells on Windows

Computers with Windows operating systems do not automatically have a Unix Shell program installed. In this lesson, we encourage you to use an emulator included in MobaXterm for Windows, which gives you access to both Bash shell commands and SSH.

Once installed, you can open a terminal by running the program MobaXterm from the Windows start menu.

Shell Programs for Windows

Alternatives to MobaXterm and Git for Windows

Other solutions are available for running Bash commands on Windows. There is now a Bash shell command-line tool available for Windows 10. Additionally, you can run Bash commands on a remote computer or server that already has a Unix Shell, from your Windows machine. This can usually be done through a Secure Shell (SSH) client. One such client available for free for Windows computers is PuTTY. See the reference below for information on installing and using PuTTY, using the Windows 10 command-line tool, or installing and using a Unix/Linux emulator.

For advanced users, you may choose one of the following alternatives:

Warning

Commands in the Windows Subsystem for Linux (WSL), Powershell, or Cygwin may differ slightly from those shown in the lesson or presented in the workshop. Please ask if you encounter such a mismatch — you’re probably not alone.

Unix Shell on macOS

On macOS, the default Unix Shell is accessible by running the Terminal program from the /Application/Utilities folder in Finder.

To open Terminal, try one or both of the following:

  • In Finder, select the Go menu, then select Utilities. Locate Terminal in the Utilities folder and open it.
  • Use the Mac ‘Spotlight’ computer search function. Search for: Terminal and press Return.

For an introduction, see How to Use Terminal on a Mac

Unix Shell on Linux

On most versions of Linux, the default Unix Shell is accessible by running the (Gnome) Terminal or (KDE) Konsole or xterm, which can be found via the applications menu or the search bar.

Special cases

If none of the options above address your circumstances, try an online search for: Unix shell [your operating system].

Setup an SSH client

Details

All attendees should have an SSH client installed. SSH is a tool that allows us to connect to and use a remote computer as our own. Please follow the directions below to install an SSH client for your system if you do not already have one.

Modern versions of Windows have SSH available in Powershell. You can test if it is available by typing ssh --help in Powershell. If it is installed, you should see some useful output. If it is not installed, you will get an error. If SSH is not available in Powershell, then you should install MobaXterm as described below.

An alternative is to install MobaXterm from https://mobaxterm.mobatek.net. You will want to get the Home edition (Installer edition). However, if Powershell works, you do not need this.

macOS comes with SSH pre-installed, so you should not need to install anything. Use your “Terminal” app.

Linux users do not need to install anything, you should be set! Use your terminal application.