Code Overview

The physics, PDEs, and numerical discretization of with miniWeather will be giving in a separate page. Here, we take a look at how the model's code is setup, what the main model variables are, and the overall flow of the model code.

Overall Model Flow:

The miniWeather code uses a traditional Finite-Volume flow:

• Initialize the data
• For each time step while elapsed time < total simulation time:
  • For direction 1
    • For each Runge-Kutta stage (there are three of them):
      • For each cell edge in the this direction, compute cell-edged flux
      • For each cell, compute tendencies based on flux divergence across the cell
  • For direction 2
    • For each Runge-Kutta stage (there are three of them):
      • For each cell edge in the this direction, compute cell-edged flux
      • For each cell, compute tendencies based on flux divergence across the cell
  • If it's time for output, the output the state
• Finalize the model

Main Data Arrays:

• state: This is the fluid state at the current time step, and it is the only array that persists from one time step to the next. The other four are only used within the calculations to advance the model to the next time step. The fluid state describes the average state over each cell area in the spatial domain. This variable contains four fluid states, which are the traditional mass, momenta, and thermodynamic quantities of most fluid models:
  1. Density (ID_DENS): The 2-D density of the fluid, , in (note this is normally , but this is a 2-D model, not 3-D)
  2. U-momentum (ID_UMOM): The momentum per unit area of the fluid in the x-direction calculated as , where u is the x-direction wind velocity. The units are . Note that to get true momentum, you must integrate over the cell, giving units of
  3. W-momentum (ID_WMOM): The momentum per unit area of the fluid in the z-direction calculated as , where w is the z-direction wind velocity. The units are . Note that to get true momentum, you must integrate over the cell, giving units of
  4. Potential Temperature (ID_RHOT): The product of density and potential temperature, , where , , T is the true temperature, and and are the dry air constant and specific heat at constant pressure for dry air, respectively. The units of this quantity are .
• state_tmp: A copy of state used to keep track of intermediate stages in Runge-Kutta time stepping
• flux: The flux function defined as the 1-D line average at cell edges in the x and z directions (it is reused for each direction). The difference in flux across a cell determines by how much that quantity increases or decreases in a time step.
• tend: This is the time tendency of the fluid state , where is the the state vector, and as the name suggests, it has the same meaning and units as state, except per unit time (appending to the units). In the Finite-Volume method, the time tendency of a cell is equivalent to the divergence of the flux across a cell.

Other Important Variables:

• nx_glob, nz_glob: The total number of cells in the x- and z-directions
• dx, dz: The length in meters of a grid cell in the x- and z-direcitons
• xlen, zlen: The total length in meters of the domain in the x- and z-directions, fixed to 20km and 10km, respectively.
• sten_size: The number of cell averages in a "stencil" of data used to interpolate values of the fluid state variables at cell edges in order compute fluxes at cell edges. It is fixed at four (i.e., fourth-order accuracy).
• hs: The number of cells used in the "halo" regions for boundary conditions and MPI domain decomposition. It is fixed at two.
• NUM_VARS: The number of fluid state variables. Fixed at four: density, x-momentum, z-momentum, and density*potential temperature (see above)
• ID_DENS, ID_UMOM, ID_WMOM, ID_RHOT: The index for each fluid state variable in the state data array.
• sim_time: How many total seconds to simulate
• etime: How many seconds have been simulated so far
• dt: Size of the time step in seconds
• output_freq: How often to output data
• data_spec_int: What kind of experiment to simulation. Valid values include: DATA_SPEC_COLLISION, DATA_SPEC_THERMAL, DATA_SPEC_GRAVITY_WAVES, DATA_SPEC_DENSITY_CURRENT, and DATA_SPEC_INJECTION.
• hy_dens_cell, hy_dens_theta_cell: Background hydrostatic density and density*potential temperature as cell averages (depends only on the z coordinate / varies only in the vertical direction).
• hy_dens_int, hy_dens_theta_int, 'hy_pressure_int': Background hydrostatic density, density*potential temperature, and pressure at cell edges in the z-direction (depends only on the z coordinate / varies only in the vertical direction).

Important variables for MPI-decomposed code:

• nx, nz: Number of cells in the x- and z-directions for this MPI task. nz == nz_glob in all cases, since only the x-direction is decomposed over MPI tasks.
• i_beg, k_beg: The beginning global index in the x- and z-directions for this MPI task. Global indices start at zero for C / C++ code and one for Fortran code.
• nranks: Total number of MPI ranks for this model run.
• myrank: My rank ID (rank IDs start at zero for all languages) among all of the other ranks in this simulation.
• left_rank, right_rank: The rank IDs for neighboring MPI tasks to my left and right in the MPI domain decomposition.
• masterproc: Whether myrank == 0 or not.

Brief description of the model grid and indexing strategies

The miniWeather app uses something called a Finite-Volume method to discretized the PDEs, which, in brief, means that cell averages of the data are evolved using fluxes in and out of the cell. Therefore, in the x-direction, there are nx cells in the domain and nx+1 cell edges. In the z-direction, there are nz cells and nz+1 cell edges.

To facilitate easier boundary conditions and MPI data exchanges in the x-direction, we create two "halo cells" at the left and right of each domain. Halo cells are cells whose data "belongs" to the MPI task to your right and left. Before decomposing over MPI, the two halo cells to your left belong to the right part of your domain, and the two halo cells to your right belong to the left part of your domain, meaning what flows through the right edge of the model comes back into the left edge ("periodic" boundaries).

To facilitate easier boundary conditions in the z-direction, we create two "ghost cells" of data that exists solely to satisfy the solid wall boundary conditions at the top and bottom of the model.

Therefore, you will see a total of nx+2*hs cells in the x-direction and nz+2*hs cells in the z-direction. All model variables live as cell averages in the same location, which is called a "collocated" or "A-grid" approach. It's simpler than keeping up with variable indices in a staggered-grid approach, making the code simpler to work with.

Fortran Indexing

Cell-averaged state: The x-direction state grid is laid out such that indices -1 and 0 belong to right-most domain of the MPI task to the left, indices [1,2,...,nx] belong to the current MPI task, and indices nx+1 and nx+2 belong to the left-most domain of the MPI task to the right. In the z-direction, indices -1 and 0 are used for model bottom boundary conditions, indices [1,2,...,nz] are the physical domain, and indices nz+1 and nz+2 are used for model top boundary conditions. The hydrostatic backrgound state variables hy_*_cells also use ghost cells in the z-direction and are indexed accordingly.

Cell-averaged tend: The tendencies do not need halo or ghost cells. Therefore, they are simply indexed as [1,2,...,nx] and [1,2,...nz] in the x- and z-directions, respectively.

Cell-edge flux: The fluxes do not need halo cells or ghost cells. If there are nx cells, there are nx+1 cell edges. Therefore, in the x- and z-directions, there are nx+1 and nz+1 cell edges, respectively. These are indexed at indices [1,2,...nx+1] and [1,2,...nz+1], respectively.

C / C++ Indexing

Cell-averaged state: The x-direction state grid is laid out such that indices 0 and 1 belong to right-most domain of the MPI task to the left, indices [hs+0,hs+1,...,hs+nx-1] belong to the current MPI task, and indices hs+nx and hs+nx+1 belong to the left-most domain of the MPI task to the right, where hs == 2. In the z-direction, indices 0 and 1 are used for model bottom boundary conditions, indices [hs+0,hs+1,...,hs+nz-1] are the physical domain, and indices hs+nz and hs+nz+1 are used for model top boundary conditions. The hydrostatic backrgound state variables hy_*_cells also use ghost cells in the z-direction and are indexed accordingly.

Cell-averaged tend: The tendencies do not need halo or ghost cells. Therefore, they are simply indexed as [0,1,...,nx-1] and [0,1,,...nz-1] in the x- and z-directions, respectively.

Cell-edge flux: The fluxes do not need halo cells or ghost cells. If there are nx cells, there are nx+1 cell edges. Therefore, in the x- and z-directions, there are nx+1 and nz+1 cell edges, respectively. These are indexed at indices [0,1,...nx] and [0,1,...nz], respectively.