Nice to see your interest in our team’s work and kudos for the effort in finding articles on our website 🙂
I’m a member of the E3SM project. FYI, the finalist papers are out now.
Our paper: https://dl.acm.org/doi/10.1145/3581784.3627044

If you are attending Supercomputing Conference, join us.
Presentation (Nov 15 at 10:30AM): https://sc23.conference-program.com/presentation/?id=gbv102&sess=sess298

… and highly informative fully-captioned plots can be found at: https://e3sm.org/e3sm-is-a-finalist-for-the-gordon-bell-prize-for-climate-modeling/

They include how solution performance scales with number of Frontier nodes, with and without GPU optimization, and relative to Summit. Exciting stuff!

The “SCREAM resonates” boldly on El Reg. as well (09/19, by TNP veteran expert N.H.-P.) where the article pointfully links to the LLNL’s news piece on GBP (under the “team” link, in the 4th paragraph) which has this very nice picture: https://contenthub.llnl.gov/sites/contenthub/files/styles/carousel_823x450/public/2023-09/screamv1BlueMarbleGordonBell2023.jpg?itok=PrGdaT8g

It beautifully compares SCREAM’s whole-earth output to cloud measurements by the Japanese Himawari meteorological satellite, in two quadrangles that look to me like southern Australia, and eastern China. Visually, the correlations are spectacularly good!

I may have to double-check with former UMCP colleague Howard C. Elman (Dr. Preconditioner, not mayonnaise), but I think that well-crafted preconditioners can substantially reduce PCG fill-in (holy grail of the HPC secret sauce!) and that’s the expected advantage of iterative methods over direct solvers, to tackle the most portly of sparse matrix-vector systems, within limited RAM (when they work!).

Not so fast! Wouldn’t PCG generate a lot of fill-in within the matrix’s band (during solution) which may be as large as 40,000 x 6 x 128 = 31e6 values per row (for 6 dependent vars going around earth’s circumference at 1km spacing, with 128 vertical nodes)? That would boost the weight matrix storage requirement to 400 exabytes (during solution) and make Frontier explode, I think.

It will be absolutely crucial and paramount for TNP to attend that SC23 presentation and Q&A to better ascertain the details of the specific numerical procedure employed by SCREAM (as mapped onto Frontier), and so-inform its most worldwide of high-caliber readerships! Is their preconditioner super-good? Are they using some anisotropic form of spherical ADI? Are captain Krylov kryptonite and its approximate matrix exponential sidekick involved?

At (my napkin calcs) SCREAM’s 400 billion simultaneous equations over 65 billions spatial nodes, the corresponding FDM matrix is nominally a 1280 zettabyte affair (for FP64: 400e9 x 400e9 x 8 bytes), but luckily very sparse, with “just” 12.8e12 non-zero items (102 TBs of 27-point FP64 FDM weights). Frontier has 32 PB of RAM so these weights can fit nicely there, along with the 400 billion-value FP64 solution vector (3 TB) at each time-step — well, at least for some 10,000 time-steps; simulating one year of weather takes at least 100x as many (paging doctor NVMe!)!

Assuming that their solution algo (PCG?) requires 5 FLOPs per FDM weight, that their max wind speed is 300 km/h, and that nonlinearity converges in just 5 iterations, means 1.26 years of simulation is accomplished in (with the 14 PF/s perf of Frontier on HPCG):

12.8e12 x 5 x 3 x 5 x 1.26e6 / 14e15 = 86,400 seconds = 1 day

(as they report) which, for the whole earth, is truly awesome! Now, imagine if we could break the memory wall, and get the full 1.2 EF/s on such sparse problems (an 80x speedup)!

Right on! BTW, digging archeologically through the trove of computational history that is TNP, following the narrow links into dimly lit bat-infested caves, slithering snakes, venomous funnel web spiders, the skeletal remains of ghostly buccaneers, and nittany lions, does lead to treasure chests of crispy golden nuggets, and related riddles. Tracking the 2021 Oceanlite through its labyrinthine lair suggests:

For 2021: 1.05 EF/s sustained, 1.2 EF/s single-precision, 4.4 EF/s mixed-precision, 35 MW

… or 30 GF/W. The SC23 update and Q&A should be invaluable to confirm the most recent progress, most accurate figures, and to get one’s mind blown, clean-off! 8^b

George Stokes was barely 3 years old when Navier published them, but rivalry between France and Britain, due in part to Louis the XVIth’s sponsorship of the 1776 American Revolution, just 45 years prior, probably delayed their broader adoption (monarchies hold grudges for a long time — Haiti’s still reeling from its 1804 revolution against France!).

Interestingly (for TNP’s French readership), the ORNL News Desk did recently feature a partnership between GE Aerospace and the French Safran that seems quite related to the Oceanlite GBP effort, but using Frontier’s Exascale oomph, for turbofan flow optimization (https://www.ornl.gov/news/exascale-drives-industry-innovation-better-future).

One can’t help but wonder what a world free of monarchies, dictatorships, and authoritarian governments could do!