I recently presented on the project at the Hartree Centre's HPC As a Service for Industry event. You can see my slides above, and I have broken out some specific comments in this blog post.
UK e-Infrastructure Programme
HPC Midlands was funded by EPSRC as a regional HPC centre of excellence, under the £158m e-Infrastructure initiative announced by BIS in Autumn 2011. In my talk I made the point that this has been a long time coming, and builds upon prior work by the OSI e-Infrastructure Working Group (2007) and the RCUK e-Infrastructure Advisory Group (2011). People may be more familiar with the Tildesley Report, whose publication coincided with the announcement of the programme.
More recently the Chancellor's 2012 Autumn Statement allocated £600m for what we now refer to as the Eight Great Technologies, including £189m specifically earmarked for Big Data and Energy Efficient Computing. We are expecting to hear in the next few days how this funding will be allocated, but it would be natural to build upon the successes of the e-Infrastructure programme. There are, of course, particular synergies with the move to Open Access for publishing and research data. See my blog posts "Suddenly, Everything has Changed" and "Describing the Elephant" for my take on this.
Hera, the HPC Midlands cluster
We are very proud of Hera, which was one of the first HPC systems in the UK to be constructed around the Intel Sandy Bridge architecture. I have pasted in the system specifications below, and would note in particular that Hera has a large number of high memory compute nodes, with 8GB of RAM per CPU core.
- 3,000 cores (48 Teraflops)
- 11 chassis (18 blades each)
- 15TB RAM
- 120TB Lustre storage
- Non-blocking QDR Infiniband
- 188 compute node blades, each with:
- 2 x 2.0GHz (8 core) Sandy Bridge CPU
- 140 with 64GB RAM (4GB/core)
- 48 with 128GB RAM (8GB/core)
Please do get in touch if you have an interesting application for Hera - we would love to hear from you! You can find out more and contact us via the HPC Midlands website.
Open For Business
In my Hartree Centre talk I made the point that the industrial engagement component of the e-Infrastructure initiative was a non-negotiable part of the package. In the past we might have expected to receive RCIF funding and use it towards an institutional facility for our own exclusive use. The new model is very much for a shared service used by (and bringing together) University researchers and industrial collaborators. At the close of our project in 2016 we will be judged on our success at accomplishing this.
Something else that for me is significantly different about the e-Infrastructure services is the degree of engagement with both suppliers and software vendors. Just as STFC are working with IBM to deliver the Hartree Centre, we are working very closely with Bull on HPC Midlands. I have also been delighted that key ISVs like ANSYS and CD-adapco are eager to work with us on flexible licensing models for a multi-tenant on demand environment. CD-adapco's Power-on-Demand model is shown in the figure below:
Software licenses are often a significant proportion of the overall cost of an HPC project. This flexibility will be key to both "cloudbursting" beyond internal HPC capacity and also bringing new users onboard. I hope that we will be able to build on the HPC Midlands work to develop community wide agreements with the leading ISVs in a similar fashion to JANET(UK)'s Model Contracts for cloud email services from Google and Microsoft.
Industrial Engagement
I teased people a little in my talk by showing them images of HPC based visualizations, and asking them to take a guess as to what the image described. For example, could you tell what the image below represented without the paper title? (I was impressed by how many people successfully guessed what they were looking at!)
T. Coates, G. J. Page, 30th AIAA Applied Aerodynamics Conference
DOI: 10.2514/6.2012-2775
But there was another element to this exercise - the images I showed the audience were from some of our industrial collaborations with firms like Airbus and Rolls Royce. If you see something which is billed as a journal paper or conference presentation, the tendency is to assume that it is a purely academic piece of work. The reality is that research intensive Universities like Loughborough and Leicester have extensive industrial collaborations, and a large proportion of research output is directly linked to work with industry.
These collaborations take the form of joint projects like Leicester's Mintweld, and higher level strategic initiatives such as the Rolls Royce University Technology Centre at Loughborough. Rolls Royce alone are involved in over 400 doctoral level projects across the UK, and it will be very interesting to see whether there is a role for the e-Infrastructure services in bringing together and supporting this larger community. Some simple steps that we see being hugely useful are to provide well connected and high capacity / low latency "working storage" for shared access, which we would struggle to do within the conventional corporate IT paradigm.
HPC Midlands Case Study – E.ON
We are working to assemble a portfolio of case studies around HPC Midlands, and in this vein I was delighted to be able to report back on a recent project with E.ON's Plant Modelling Unit. This used ANSYS CFX to simulate a leak in a power station gas turbine enclosure, as shown in the figure below:
The simulation was one that E.ON had previously carried out on their internal HPC cluster, and we were interested to explore whether we could a) achieve significant speed-ups by comparison with their in-house system, and b) run at a significantly higher resolution. These are the sort of real world comparisons that will help us all to understand what practical role e-Infrastructure services like HPC Midlands can play in working with industry.
We ran the gas turbine simulation with a number of different parameter sets ranging from a 4.5m cell mesh on 24 cores, through to an 80m cell mesh on 128 cores, and observed speed-ups of betwen 30 and 145 times the performance of the in-house system based on workstation class PCs. This very graphically demonstrated the benefits of having a high performance interconnect and storage subsystem. I was also gratified that my colleagues from E.ON found that our system was “straightforward to use, secure and fast” :-)
Plumbing Considerations
Our work with E.ON also highlighted for me some of the practical "plumbing" related aspects of working with industry. I will single out one particular example here which may give pause for thought. The images below show the car route between our locations at Loughborough and the E.ON Ratcliffe-on-Soar power station (some ten miles apart), and the Internet route, which is somewhat more circuitous. As you may have imagined, we see this routing diagram because E.ON's corporate Internet gateways are based in Germany.
It goes without saying that UK peering could potentially be introduced, or even a dedicated circuit between our two sites. However, there may also be opportunities for JANET(UK) to facilitate e-Infrastructure collaborations on a sector wide scale - e.g. connecting firms to JANET in order to give them high speed / low latency access to all of their University collaborators' systems and services. Project Moonshot, which applies the underlying principles of eduroam (the JANET Roaming Service) to arbitrary network protocols, may also be of interest.
Conclusions
If you were able to attend my Hartree Centre talk, I hope I have been able to provide some useful additional information beyond what there was time for in my session. If you couldn't make it, I hope that your interest will have been piqued by this post - stay tuned for more on HPC Midlands!
No comments:
Post a Comment