During the 10-week program, students will receive training and lectures on modern topics in HPC and software development, including:
- Parallel programming
- Programming models
- Hardware architecture and its impact on code design choices
- High-quality software development in collaborative environments
- Visualization and workflow
Students will collaborate in teams to identify and investigate different computational problems within the scientific focus area, and implement solutions guided by mentors with scientific and computational expertise. By working on cutting-edge HPC hardware, students will gain hands-on experience and learn how to effectively communicate their work through posters and oral presentations.
For the 2024 internship, there will be 6 different projects for students to choose from. In the projects listed below, students will work in pairs and with their mentors who will oversee their project. The mentors are all established scientists at Los Alamos National Laboratory.
Thermonuclear Burn in FLAG on GPUs
Mentors: Edward Norris (email@example.com) and Tyler Markham (firstname.lastname@example.org), XCP-1
Students will investigate how to effectively integrate the GPU capabilities of the Singe thermonuclear burn library into the FLAG multiphysics code. FLAG currently links to Singe for calculation of thermonuclear burn rates but does not utilize the GPU capabilities provided by Singe. Students will explore Singe and other codes to identify how best to leverage the Singe’s GPU capabilities.
Squeezing Every Last Drop of Performance from a Multi-Physics Code
Mentors: Michael Witek (email@example.com), David Culp (firstname.lastname@example.org), and Thomas Henderson (email@example.com), XCP-2
Pagosa is a hydrodynamics computer code built for running simulations in massively parallel environments. Simulations are run in an Eulerian framework that uses a fixed Cartesian mesh, and this computational model naturally leads to a domain decomposition where MPI ranks bound to cores each receive portions of the mesh to work on. However, as processor speeds approach physical limits, simply scaling up the number of ranks may not offer the same increases in performance as in the past, and hardware manufacturers are increasingly turning towards innovations in many-core architectures, memory bandwidth, or heterogeneous computing configurations. In this project, we intend to explore ways in which Pagosa can benefit from new Sapphire Rapids nodes in upcoming LANL systems. Each node has 2 sockets, 56 cores per socket, 2 threads per core (via hyperthreading), and, depending on the system, will be configured to use either DDR5 or High Bandwidth Memory (HBM). The selected candidates will be tasked with investigating the performance effects of hyperthreading as well as ways to modify Pagosa’s MPI environment, possibly through a hybrid MPI and OpenMP parallel decomposition, to better utilize Sapphire Rapids nodes with HBM. The results of this project will lead to a more modern and performant code capable of tackling the toughest computational problems.
PARTISN GPU Profiling and Kernel Sizing for El Capitan
Mentors: Joe Zerr (firstname.lastname@example.org), Nate Hart (email@example.com), and Mario Ortega (firstname.lastname@example.org), CCS-2
The deterministic neutron transport code PARTISN is being ported to the GPU-accelerated system El Capitan, based on the porting work done for Sierra. GPU kernels were launched with sizes (register usage and threads/block) that were optimized for Sierra hardware. This project will focus on reconsidering these optimizations for the El Capitan hardware. This will involve a mix of using profiling tools and doing straightforward performance comparisons to recommend new values for optimized performance.
More powerful tasks for task-based parallelism
Mentors: Davis Herring (email@example.com) and Richard Berger (firstname.lastname@example.org) CCS-7
Task-based parallelism decomposes an application into a set of tasks: distributed function calls with known data dependencies. The need for analysis and parallel, portable execution of these tasks restricts how they are expressed, which can make programming in this model more challenging. We plan to extend our C++ task-based parallelism library FleCSI with additional forms of task invocation that automatically select relevant data types, take advantage of data already located in available host or GPU memory, and facilitate access to data allocated by parts of the application not based on FleCSI. The goal is to improve the separation between the scientific application and the computational infrastructure, allowing experts in each field to better focus their attention.
Ocean Model Optimization on GPUs
Mentors: Mark Petersen (email@example.com), CCS-2 and Brian O’Neill (firstname.lastname@example.org), T-3
Climate research at the U.S. Department of Energy includes the development of ocean, sea-ice, atmosphere, land-vegetation and land-ice models. The ability to run high-resolution global simulations efficiently on the world’s largest computers is a priority for the DOE. The proposed project is to assist in the development of a new ocean model component for the Energy Exascale Earth System Model, which will be designed to run at scale on new exascale computers, and take full advantage of GPU-accelerated hardware. The student should have some background in numerical methods and c++. The student would be involved in code development, testing, verification, and simulations. The new model, OMEGA: The Ocean Model for E3SM Global Applications, is specifically designed for modern exascale computers with both CPUs and GPUs. OMEGA is an unstructured-mesh, variable-resolution ocean model. It is written in c++ with the YAKL library to facilitate efficient and convenient utilization of heterogeneous architectures. The PCSRI project will involve comparisons of quality and performance of model terms between the CPU and GPU versions. The student will be involved in team discussions and planning for code design and optimization, and carry out optimization work on select parts of the model.
Modeling Sediments in Sea Ice
Mentors: Nicole Jeffery (email@example.com) and Mark Petersen (firstname.lastname@example.org), CCS-2 River fluxes and erosion carry terrestrial sediments into coastal ocean waters. In polar regions, these particles are entrained during sea ice formation and transported to shelf and deep waters as winds and ocean currents drive sea ice circulation. The presence of sediments in sea ice alters the optical properties, enhancing the absorption of light energy in interior layers and increasing the albedo when present at the surface. We hypothesize that sediment fluxes from Arctic rivers cause pronounced declines in sea ice extent and volume in some coastal regions and significantly reduce light penetration through the sea ice. To test this hypothesis, the student will compare and contrast three 10-year simulations using the parallel high performance code -- Energy Exascale Earth System Model (E3SM) -- in the following historically forced ocean-sea ice configurations: 1) activate river-sediments in sea ice but without the radiative transfer feedback, 2) activate river-sediments in sea ice and the radiative transfer feedback, 3) run with no additional river- sediment tracers in the sea ice. The first simulation will be used to compare the distribution of sea ice sediments with remotely sensed and in situ observations. The second will be contrasted with the first to assess the impacts of the radiative feedback on sea ice extent, volume, and light penetration. The student will conduct a performance optimization study of MPAS-Sea Ice overall, and the additional cost of the sediment tracers and radiative transfer code. This will include scaling plots as a function of core count and the number of OpenMP threads. The student will experiment with optimization methods within the sediment tracer and radiative transfer modules.
The Parallel Computing Summer Research Internship covers a broad range of topics that you may not have encountered in your studies. To help augment your education, the mentors have compiled a list of recommended reading and reference materials. Some of these will be used as reference material during the summer. We certainly don't expect students to read all of these references prior to the summer, but should be used in the course of your career. Books without links can be found in the LANL library or on one of our bookshelves.
- Familiarity with Unix/Linux environment
- Familiarity with a real editor (emacs, vi)
- Software Development
- Scientific Programming
- Parallel Programming
- Introduction to High Performance Scientific Computing, by Victor Eijkhout (free download via bitbucket & excellent exercises in the Tutorials chapter)
- Introduction to Parallel Computing
- An Introduction to Parallel Programming, by P.Pacheco and M.Malenski, 2nd Edition (2021)
- Message Passing Interface (MPI)
- HPC Systems
- Numerical Methods, Patterns & Algorithms
- "Finite Difference Schemes and Partial Differential Equations," by John C. Strikwerda
- "Computational Methods for Fluid Dynamics," by J.H. Ferziger and M. Peric
- "Numerical Mathematics," by A. Quarteroni, R. Sacco, and R. Saleri (Earlier release available online, Google " Quarteroni Numerical Mathematics"
- Structured Parallel Programming with Deterministic Patterns (PDF)
- Algorithmic Skeletons: Structured Management of Parallel Computation (PDF)
- Parallel Random Numbers: As Easy As 1, 2, 3
- Parallel Algorithms for Monte Carlo Particle Transport Simulation on Exascale Computing Architectures
- Good Reading
- Bytes for thought (controversial)
- Proxy applications and procurement benchmarks:
During your 10-week internship, we hope you have the opportunity to explore and enjoy Los Alamos and the surrounding area. Here are some links we've compiled to help you with your stay:
Student Programs (housing, transportation, living, and more!)
Living in Los Alamos (links to help you navigate the town and find resources)
Connect with other students at LANL
- LANL Wellness Center Health & Fitness facilities (available to LANL hires who have taken the required training)
- YMCA provides special rates for limited duration summer students with a college ID
- 3 mos is $75, 1 month is $30, 1 week is $15; And there is no sign up fee
- Website: The Family YMCA
Summer Entertainment (not generally listed in the Living in Los Alamos link)
The Gordons' Summer Concerts provide great evening entertainment. It's where you'll find much of the town on a Friday night.
The Los Alamos Daily Post lists events
Facebook is a key communication mechanism in the Los Alamos community - a popular one is KEEP IT LOCAL-Los Alamos
NOTE: If you're coming with a spouse or children, let us know so that we can point you to resources (e.g., childcare, activities, etc.) If your questions about living in Los Alamos are not answered in these resources, feel free to send us an email.
This highly-selective program is designed for graduate students and advanced undergraduates from all STEM fields. Recent graduates from an accredited U.S. university may also qualify as a post-baccalaureate or post-masters if they are within a year or two of their degree. As a general guideline, students should have moderate experience with a compile scientific computing language, such as C, C++, or Fortran and with the Linux operating system. Applicants must be U.S. citizens enrolled at an accredited U.S. university. As part of the application process, please provide the following documentation:
- Current resume (Please state citizenship in your application)
- Letter of intent describing your:
- Research interests and experience
- Computational/computing experience
- Interest in the program
- Overall strengths and goals
The application process for the Summer 2023 internship is now closed.
Participants will receive a fellowship stipend, the amount to be determined based on your current academic rank, generally between $9,000 and $17,000. You will be responsible to cover your own travel, food, and housing. Housing is in short supply in Los Alamos during the summer, so we encourage acting early and will be available to provide resources to find housing.