You are here
High-Performance Computing (HPC) is the use of large-scale, off-site computers and parallel processing techniques for solving complex computational problems. HPC technology focuses on using parallel processing algorithms and systems. HPC is typically used for solving advanced problems and performing research activities through computer modelling, simulation and analysis. The terms High-Performance Computing and Supercomputing are used interchangeably.
HPC systems are most often useful for researchers:
- with parallelisable tasks (parallelism can be achieved on a fine or coarse grain level — on a fine level you have a program which is intrinsically parallel while on a coarse grain level you might run many instances of the same program for parameter studies)
- to take advantage of fine/coarse grain parallelism, embarrassingly parallel and parameter studies techniques
- with computing jobs needing more memory (RAM) than available on their local systems (if you run out of memory on your desktop system or can no longer process your data quickly enough)
needing access to software packages offered by the HPC facilities.
HPC jobs run in a non-interactive batch mode. These jobs are submitted to a queue, and when the time comes for them to be executed they can run without any user intervention. This means that HPC systems rarely sit idle but utilise the hardware fully. It has the benefit that you can queue your jobs and get on with other things while you wait for them to execute. If you are accustomed to using your program interactively, you will have to learn how to use its batch mode in order to use it for HPC.
Intersect offers a series of HPC courses to meet the needs of all researchers, whether you are at a Beginner, Intermediate or Advanced HPC level. Check our Training page for more details.
- Intensive Beginner to Intermediate HPC - Introduction to UNIX for HPC
- Intermediate HPC - From Unix to HPC
Advanced HPC - Parallel Programming
As a researcher you have access to a number of facilities:
- The National Facility at NCI (National Computing Infrastructure), raijin.NCI.org.au, which serves researchers nationally
- The NeCTAR and OwnTime Research Clouds, over 4,500 local and 32,000 distributed computing cores
- Facilities local to your institution or faculty
For details about the last option, please contact your institution.
The NeCTAR Research Cloud, rc.NeCTAR.org.au is powered by OpenStack.org, a cloud orchestration system that controls large pools of compute, storage, and networking resources throughout multiple Australian data centres, all managed through a dashboard that gives administrators control while empowering consumers with web self-serve resource provisioning. Pre-baked virtual machine images are catalogued at: https://support.rc.nectar.org.au/docs/images
Connecting to Raijin
To connect to Raijin you'll need a Secure Shell (SSH) client. If you're on Linux or Mac OS X, chances are that you can just open a terminal and issue one of the following command:
When prompted, enter your username and password. If you're running Windows, you'll need to download an SSH client. A good free option is Putty.
To transfer files to and from your machine, you'll need a File Transfer Protocol (FTP) client that supports Secure-FTP (SFTP). You can do this from the command line on Linux, Mac OS X, and Windows:
But you might find it easier to use a graphical FTP program, such as FileZilla.
HPC Software Registries
A complete lists of all software installed are available at https://opus.nci.org.au/display/Help/Application+Software
Please note that software packages marked with a yellow dot carry license restrictions. Please contact us to establish whether you can use this software.
Setting up software environments
To set up the environment for a software package you need to use the module system. This is identical on both machines but the module names might differ. To see the exact names of the modules visit the links given above or to get a list use the command:
To load, for example, the now latest version of the Intel compilers on Raijin use
module load intel-fc/188.8.131.52
This sets up your environment (variables and path). The reason for using this module system is to allow for different versions of the same software package.
Other useful module commands include:
A list of already loaded modules: module list
To show you what a module does: module show package-name
To unload a package module unload <package> ()
Software installation and requests
If you would like to have new software installed on Raijin, please email to firstname.lastname@example.org and specify the download site. You are also welcome to install software yourself in your home directory.