Chris Sewell, Jim Ahrens, LANL
Berk Geveci, Patrick O’Leary, Kitware Inc.
From the Houston “Scientific Discovery at Exascale” report: Post-processing is currently the dominant processing paradigm for visualization and analysis on ASCR supercomputers (and other supercomputers): simulations write out files, and applications dedicated to visualization and analysis read these files and calculate results. However, supercomputers that have come on line recently are increasing memory and FLOPs more quickly than I/O bandwidth and capacity. In other words, the I/O capability is decreasing relative to the rest of the computer. It will be slow to write data to disk, there will not be enough space to store data, and it will be very slow to read the data back in. This trend hampers the traditional post-processing paradigm; it will force simulations to reduce the amount of data they write to disk and force visualization and analysis algorithms to reduce the amount of data they read from disk.”
The goal of in situ processing is to tightly couple analysis and visualization codes with simulation codes and run the together on supercomputers. This has the benefit that disk I/O is no longer necessary and data can be transferred in-memory from one code to another. There are some concerns about this approach however:
- Scalability of analysis and visualization codes, which will be essential to continued scalability of simulation codes due to the tight coupling,
- Memory usage of analysis and visualization codes given the limited total memory on compute nodes,
- Stability of analysis and visualization codes at scale
Kitware, Sandia and LANL has developed an in situ library based on DOE’s flagship visualization application ParaView. This library, called ParaView Catalyst, is a general purpose tool that can be embedded in many types of simulation codes for in situ analysis and visualization.
Under SciDAC SDAV, we have started integrating Catalyst with various codes important to ASCR. We have been investigating the issues described above and addressing them as needed. The figure on the slides shows how Catalyst, coupled with VPIC, scales well to 5K processors and has significantly less overhead than writing data to disk. In this case, we used ParaView Catalyst to generate iso-surfaces and render the result all in situ.