Sandia’s Sun Constellation system, Red Sky, has been placed at number 10 on the latest Top 500 supercomputer list. It’s a monster cluster system – 70TB of memory, 47,232 cores, and built up of Sun x6275 blade systems hooked up in a 3D torus with Infiniband interconnects.
Marc Hamilton has posted up a timelapse video on his blog over at Sun showing the system being installed.
Red Sky has replaced Sandia’s existing Thunderbird system, and is actually built in the same place as ASCI Red used to live. The x6275 blades use Intel Nehalem EP processors running at 2.96Ghz, with no local disk fitted to the blades, allowing a much greater density.
Red Sky also features Sun’s new Cooling Door System which pumps cooled water through the cabinet doors. Sandia’s calculations reckon that this will save over 5 million gallons of water a year, compared to traditional air-cooled systems.
Posted on November 2, 2009 by Tom Kranz in HPC, Technology
Sun have released a technical report on Transactional Memory, based on their experiences with the (now sadly canned) ROCK processor. “Early Experience with a Commercial Hardware Transactional Memory Implementation” is available as a free download from Sun’s research website – you can grab it at http://research.sun.com/techrep/2009/abstract-180.html
From the abstract:
We report on our experience with the hardware transactional memory (HTM) feature of two revisions of a prototype multicore processor. Our experience includes a number of promising results using HTM to improve performance in a variety of contexts, and also identifies some ways in which the feature could be improved to make it even better. We give detailed accounts of our experiences, sharing techniques we used to achieve the results we have, as well as describing challenges we faced in doing so. This technical report expands on our ASPLOS paper [9], providing more detail and reporting on additional work conducted since that paper was written.
Anyone who’s interested in High Performance Computing (HPC) or performance gains from Transactional Memory should have a read through this paper – it’s interesting stuff.
I’d still like to know what to call Sun once the merger is complete. Sucle is my favourite, although I suspect it’ll end up being something boring like “Sun, an Oracle company”. While busy decrying the EU investigation into the takeover costing Oracle $100m a day, Larry has also been addressing a technology forum on Monday, and was shy about laying out his plans for world domination.
Key takeaway points from his speech were:
“I would like us to be the successor to IBM”
“We think with the combination of Sun technology and Oracle technology we can succeed and beat IBM – that’s our goal.”
“We are keeping everything. We’re keeping tape. We’re keeping storage. We’re keeping x86 technology and SPARC technology – and we’re going to increase the investment in it.”
“Sun has fantastic technology. We think it’s got great microprocessor technology – it needs a little more investment, but we think it can be extremely competitive. It’s got the leading tape archival systems. We think the Open Storage on their new disk system is absolutely fantastic. Java speaks for itself. Solaris is overwhelmingly the best open-systems operating system on the planet.”
“I shall buy Sun for *seven* *billion* *dollars* muahahahhaha”
“I shall call [Jonathan Schwartz] Mini Me”
Wait, sorry, no – those last two I made up. Although I’m sure Larry is enjoying a Bond villain style evil laugh as he contemplates finally putting the boot in to IBM, whilst stroking his devil goatee.
Joking aside, he’s right – Sun has a lot of really excellent technology, and historically they’ve failed to follow through with most of it. Oracle’s ruthless sales methodology will be striking some fear into some Sun sales managers, but engineers (and customers) should be taking comfort hearing this sort of fighting talk.
Of course, the deal (still!) isn’t done, and it might be a while before we see some systems hitting the market on the back of this enthusiasm. Still, I think Sun have totally neglected the higher end systems, relying on Fujitsu to do the hard graft – with any luck, we should be seeing some big Niagra boxes coming soon.
There’s been a great need for a while now for a decent, easy to understand introduction to HPC. Those of us who’ve worked in this niche understand the acronyms and weird technology, but for newcomers – even those with a good background in IT – HPC can be an intimidating arena.
Couple this with the blurring between ‘traditional’ HPC systems for research, and new high end business solutions for statistical analysis and database warehouses, and there’s a real need to de-mystify HPC for all.
Douglas Eadline, working with Sun and AMD, has written HPC for Dummies, and the blurb explains:
This special edition eBook from Sun and AMD shares details on real-world uses of HPC, explains the different types of HPC, guides you on how to choose between different suppliers, and provides benchmarks and guidelines you can use to get your system up and running.
What makes this really great is that this is a free ebook, available direct for download from Sun’s site. I can highly recommend that anyone with an interest in HPC (or just large scale systems design) grabs this and has a read through.
The Hotchips conference is over, and with it comes some news from Sun about their SPARC plans. Sun shared details of it’s “Rainbow Falls” processor – the next iteration in the Niagara line.
Boasting 16 cores, each with it’s own cache, it’s an impressive bit of silicon. Each core has 4 Coherency Units (COU). Familiar to anyone who’s played with big Silicon Graphics kit and other cc:NUMA boxes, Coherency Units keep track of memory contents (from L2 cache up to physical RAM).
4 COUs per core, along with 16 seperate L2 caches, is a lot on a single die. To simplify things (and to help improve performance) Sun has added a Core to Cache Crossbar (CCX). Two cores will share a single entry point into the CCX, which is linked to every core on the chip.
It’s essentially a standard switching crossbar – like UPA, Xbow, NUMAlink etc. – except it’s linking cores and L2 caches, not CPUs, RAM, and I/O bays.
All pretty impressive stuff – remember, this is all on the processor die – and it sounds like it would be ideal in a large system. Those M9000 boxes look pretty inviting, especially after the disappointment of the Niagara system board upgrades for the F15k.
And in fact, maybe that’s what Oracle has up it’s sleeve. The Prophets of Larry have said they will be making a major SPARC announcement on October 14th, during Larry’s yearly sermon to the faithful at Oracleworld. (Sorry, guys – I love you really)
With IBM pushing forward development of it’s fearsome Power7 chip, and with Power6 a bit of a monster anyway, Sun clearly needs to keep in the game – especially after (foolishly) canning Rock.
The Oracle announcement will be aimed clearly at blowing the doors off IBM – check their rhetoric and the fuzzy teaser advert on the Sun plus Oracle is Faster page.
Now, there are a number of ways they could bury IBM in the TPC-C benchmarks. Most obvious would be a massive RAC install, probably with Fujitsu’s new 8 way SPARC64 chip. But that wouldn’t really be blowing the Sun SPARC trumpet, would it?
The current top end Niagara box – the T5440 – can have up to four 8 core Niagara CPUs (along with half a terabyte of RAM). It’s pretty good, but again – the only way you can scale is by clustering them. Oracle have an insanely great scalable database solution with Oracle RAC, so it would seem a no-brainer.
However, if Sun are close to releasing Rainbow Falls, they could use the system boards and interconnects from the M9000 chassis to produce a hugely thread-dense NUMA machine. All that coherency hardware makes no sense for a cluster – it’s role is in a big Single System Image (SSI) machine.
