CHPC

Switch to desktop Register Login

CentOS based OpenStack Cloud Computing platform boosts CHPC’s virtual environment service offering

The Centre for High Performance Computing (CHPC) has introduced a CentOS-based OpenStack cloud-computing platform to boost support to users running jobs in the cloud environment.

OpenStack is a set of software tools for building and managing cloud-computing platforms for public and private clouds and is managed by the OpenStack Foundation, a non-profit organisation that oversees both development and community building around the project. The addition enhances the CHPC’s significant portfolio of hardware and software applications that are available to the research community to allow them to perform world-class research.

The CentOS based OpenStack Cloud is a self-service Virtual Machine (VM) provisioning portal where common administrative tasks like VM creation, recouping of unused resources, and infrastructure maintenance tasks are automated and capacity analysis, utilisation, and end-user costing reports can be generated. It has replaced the centre’s VMware virtual environment or IT-Shop cluster that had limited memory resources due to the large demand of numerous projects requiring high-specification virtual machines and was no longer able to serve the requirements of CHPC users when its capacity deteriorated over time.

Improved service offering

The CHPC’s new virtual environment will offer the following benefits and functionalities:

  • Self-Service Portal:cloud users will now have the ability to deploy application on-demand with limited technical support to promote rapid and efficient IT Service;
  • Metered Service and Resource Monitoring: theCHPC will now be able to monitor resource utilisation from individual users or projects to prepare billing statement as per the centre’s cost-recovery model;
  • Avoidance of Vendor Lock-in: the OpenStack solution is open source. CHPC will reduce on cost related to proprietary software such as the VMware vSphere Solution; and
  • Enable Rapid Innovations (DevOps):the CHPC staff can significantly reduce on development and testing periods and have more freedom to experiment with new technology or even do customisation to expand the capabilities of the OpenStack Cloud.

“The cloud platform further enable CHPC to gather the necessary expertise both technical and operational to develop, provision and operate a national federated OpenStack platform, and allow for global connectivity in a virtual environment for mega project like the Square Kilometre Array (SKA) and those similar in stature”, said Ms Dorah Thobye, Technical Manager: CHPC.

Technical specifications of the CHPC’s virtual environment

The new solution is built on Supermicro server and storage systems, using the Supermicro TwinPro servers to provide 320cores/640threads (2.50 - 3.90GHz) and over 3TB DDR4 memory providing some 9GB RAM per core and all in just 4U of rack space. It is connected through Mellanox 100GB ethernet networking to Supermicro Ultraand Supermicro Simply Double Servers,providing a CEPH Storage cluster with over 1.5PB (1500TB) of mechanical disk storage and more than 220TB of flash storage.

The addition of the cloud-computing platform to the CHPC service suite, is not only in-line with international standards of High Performance Computing (HPC) centres, but will assist the centre to support its users that are running virtual jobs in the cloud environment such as custom workflows, pleasingly parallel workloads and webhosting. Furthermore, the OpenStack service enables the CHPC to be a road-header for virtual HPC application provisioning and configuration in the future. “OpenStack provides a different offering for other users of HPC and this implementation is a step in the right direction to revolutionise HPC and drive towards converging platforms,” said CHPC Acting Director, Dr Happy Sithole.

 

 

 

Last Updated on Wednesday, 22 April 2020 15:59

Hits: 265

South Africa wins International Student Cluster Competition for the fourth time

A team of six South African undergraduate students has taken first prize against 13 teams at the International Supercomputing Conference (ISC) in Frankfurt, Germany. The spectacular success follows four days of working on a selection of tests and applications to optimise and run their computer cluster to demonstrate the performance of their chosen design. The competition took place from 16 to 19 June 2019.

The team of four University of Cape Town students and two University of the Witwatersrand students entered the very rigorous and fiercely contested competition following their success at the national round, where they beat nine other South African teams. The team was one of the few to be made up of 50% men and 50% women. Stefan Schröder, Dillon Heald, Jehan Singh, Clara Stassen, Anita de Mello Koch and Kaamilah Desai, under the supervision of team advisors  and computer engineers David Macleod and Matthew Cawood of the Centre for High Performance Computing (CHPC), took on 102 members of teams from the United Kingdom, United States of America, China, Taiwan, Spain, Switzerland, Estonia and Singapore.  The team took first place with the highest overall score for all the benchmarks they were given.

Competition sponsors

The team received a number of sponsorships for this competition including hardware, software and training. The total value of the South African team’s cluster was about R6 million and comprised of sponsorships from Dell EMC, Intel, Nvidia and Mellanox.

The CHPC holds the national round annually and sends different teams of undergraduate computer science and computer engineering students to participate in the international competition. Team South Africa has won the competition four times, in 2013, 2014, 2016 and 2019, and has taken second place twice, in 2015 and 2017. The teamcame third in 2018.

It all began in 2012, when a team of CHPC staff attended ISC in Germany and realised the potential of the Student Cluster Competition as an opportunity to create awareness for high-performance computing as a career and to develop the high-performance computing pipeline in the country. The team started planning a national competition that would culminate in a team of six who would represent South Africa on the international stage.

Getting started: Selecting teams for participation in the South African round

The team selection phase coincides with the CHPC’s Winter School. It is designed to impart critical knowledge for building a cluster, which includes using Linux systems, the basic software stack of a cluster, and considerations which should be taken into account when choosing hardware. Universities around the country are invited to send teams of four and the CSIR ensures that teams are entered from as many universities as possible, so that students from all backgrounds are given a chance to learn and compete. Team selection concludes with an assignment that requires each team to build a prototype cluster in the cloud. The teams selected from this round proceed to the national round of the Student Cluster Competition.

Getting to Team South Africa: Competing nationally

In the national CHPC Student Cluster Competition, participants build small high-performance clusters out of hardware provided by the CHPC and its industrial partners. The contest takes place at the annual CHPC National Conference. The participants are given a selection of applications to optimise and run on their cluster to demonstrate the performance of their design. 

Each team is assigned a budget and a parts list from one of the CHPC's industry partners. With this budget and parts list, the team must design a cluster, taking into consideration the set of applications that will be used to benchmark the cluster. Once the cluster's design is finalised, the hardware specification is submitted to the CHPC's partners for manufacturing.

The hardware, as specified in the cluster design, is delivered to the CHPC National Conference. There, the teams unpack their equipment, construct their cluster, install the software stack and perform benchmarks. The teams are judged on a combination of the performance of the applications and the design of the cluster.

South African team representatives go through extensive training after winning the national round, including travelling to the Texas Advanced Computing Centre in the United States of America, where they receive training from experienced administrators of supercomputing clusters on the design and administration of supercomputers.

Winning formula

According to the team advisor and manager of the CHPC’s Advanced Computer Engineering Lab, David Macleod, the South African team’s winning formula is to have dedicated students and sponsors. “Our sponsors are excellent and allowed the team to choose equipment without restriction or compromise. In turn the students put in a lot of time and effort before the competition and arrived at the competition well prepared,” he says.

“It really is excellent national progress. We have demonstrated consistently that talent and skills abound in our country. These teams come from different universities and provinces – showing that this is now national DNA,” says Dr Happy Sithole, Acting Director of the CHPC and Centre Manager of the National Cyberinfrastructure System (NICIS).

Last Updated on Thursday, 27 June 2019 06:35

Hits: 2175

CHPC's Lengau hangs-on in the Top500 List

The CHPC's Lengau supercomputer has placed 496th on the computing comnunity's Top500 List. The list was announced at the International Supercomputing Conference in Frankfurt, Germany in June 2019. 

For the first time, all 500 systems deliver a petaflop or more on the High Performance Linpack (HPL) benchmark, with the entry level to the list now at 1.022 petaflops. Lengau has appeared on the Top500 List since her launch in June 2016 and is currently at 1.029 petaflops. In her first appearance in June 2016, she was at number 121. 

The Top 500 List lists computers ranked by their performance on the LINPACK benchmark (The LINPACK Benchmarks measure a system's floating point computing power. Introduced by Jack Dongarra, they measure how fast a computer solves a dense n by n system of linear equations, which is a common task in science and engineering). The list is announced in June and in November each year. With over 32000 cores, Lengau remains one of the fastest computers on the African continent, with a utilisation that averagares at 90%.

Lengau continues to put the country in the company of leading supercomputing nations. She has over 1500 registered users, 500 of which are actively engaged in over 200 research programmes. 

 

Last Updated on Thursday, 20 June 2019 19:22

Hits: 1076

CHPC starts the year with Linux and Python training

The Centre for High performance Computing (CHPC) had a busy start to the year, with human capital development collaborations involving two of its stakeholders, North West University (NWU) and the National Institute for Theoretical Physics (NITheP).

NWU- CHPC Programming with Linux and Python collaboration:

As one of the universities that make use of the CHPC supercomputer, NWU sent a special request to the CHPC to send high performance computing specialists to conduct a five-day programming workshop aimed at providing basic to intermediate programming skills to the university’s teaching staff and postgraduate students.

The training took place from 14 to18 January at the university’s Potchefstroom campus.

The training consisted of two-days of Introduction to Linux (Ubuntu) and three days of Introduction to Python programming. The university received 75 applications for this workshop, of which 34 were accepted. The hands-on training was aimed at staff and postgraduate students in the fields of chemistry, physics, mathematics, applied mathematics, biology, bioinformatics, computer science and engineering, who had no prior knowledge of the programming languages.

Professor Du Toit Strauss of the NWU’s Centre for Space Research (CSR) says, “Our students need to be prepared to tackle the contemporary challenges posed by the fourth industrial revolution. Therefore, computational skills are essential. These hands-on training courses provide the participants with the necessary skills, and we are grateful to the CHPC for giving us and our students the opportunity be a part of this”.

“Based on the number of applications we received, it is clear that there is a need for similar training workshops and to branch out and become more inclusive across all disciplines in academia,” says Katlego Moloto, also associated with the CSR.

2019 CHPC-NITheP Summer School

In late January 2019, the CHPC and NITheP combined the 9th CHPC Introductory Programming School and the30th Chris Engelbrecht Summer School in a joint venture aimed at bridging the gap between theoretical studies and high performance computing. The workshop took place from 26 January 2019 until 6 February 2019, at the Premier Resort. The school that was attended by 70 students and speakers targeted Master’s and Doctoral students engaged in science and engineering degrees in South African Universities. The syllabus covered an introduction toLinux, which covered a basic introduction to the Linux command line, bash scripting and Introduction to PBS Pro and job submission at CHPC. It was conducted by Dr Krishna Govender, CHPC Research Scientist, with the support of Dr Daniel Moeketsi, CHPC Research Scientist and HPC Technician, Zama Mtshali.

Dr Andrew Gill, CHPC Research Scientist, then led the session on Python Programming and was supported by HPC Technicians William Phukungwane and Sakhile Masoka. Their session covered the basics of python and syntax, advanced function, using matplotlib with python and the use of advanced mathematical packages, such as numpy and scipy.

The school was complimented by other institutions that conducted tutorials and talks around related topics, such as Foundations of Theoretical and Computational Physics, Foundations of Theoretical and Computational Biology, Foundations of Theoretical and Computational Chemistry, Foundations of Quantum Information Processing and Computation and Machine Learning as a Tool for Theoretical and Computational Science.

Why is training in Linux and Python so important in the world of big data?

CHPC’s scientists Drs Daniel Moeketsi, Krishna Govender and Andrew Gill are some of the CHPC staff who run training courses in Python and Linux regularly, both in South Africa and in the Square Kilometre Array’s African Very Long Baseline Interferometry Network partner countries.

Dr Moeketsi explains that almost all of the top 500 supercomputers in the world are using Linux, including the CHPC’s Lengau machine. “In this intervention, the centre is assisting in training students and researchers to be proficient in Linux and Python, so that they can maximise their use of Lengau. This training is crucial for high performance computing and big data science,” he says.

Last Updated on Thursday, 21 February 2019 13:51

Hits: 1845