forked from wrf-model/TechNote
-
Notifications
You must be signed in to change notification settings - Fork 0
/
var.tex
505 lines (423 loc) · 28.3 KB
/
var.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
\chapter{Variational Data Assimilation}
\label{var_chap}
An introduction to the basic ideas of variational data assimilation and
the WRF-Var system is given in this chapter, followed by a brief
overview of recent major improvements to WRF-Var. This overview
supplements the original description of the three-dimensional
variational (3D-Var) algorithm found in \citet{barker04}. One of the
most important additions to WRF-Var is a new utility {\it gen$\_$be},
used to calculate background error covariances for a user's own
application; it is discussed in the latter half of this chapter. The
WRF-Var system is evolving rapidly,
and a future technical note will accompany the general release of the
4D-Var component of WRF-Var. That technical note will include an
extensive description of the entire WRF-Var system.
\section{Introduction}
\label{var-intro}
The basic goal of any variational data assimilation system is to produce
an optimal estimate of the true atmospheric state at analysis time
through iterative solution of a prescribed cost-function \citep{ide97}:
\begin{equation}
J({\bf x}) = J_b({\bf x}) + J_o({\bf x}) = \frac{1}{2
} ({\bf x}-{\bf x}^{b})^{T}{\bf B}^{-1}({\bf x}-{\bf x}^{b}) +
\frac{1}{2}
({\bf y}-{\bf y}^{o})^{T}({\bf E+F})^{-1}({\bf y}-{\bf y}^{o}).
\label{cost_function}
\end{equation}
The variational problem can be summarized as the iterative minimization
of (\ref{cost_function}) to find the analysis state ${\bf x}$ that
minimizes $J({\bf x})$. This solution represents the {\it a posteriori}
maximum likelihood (minimum variance) estimate of the true state of the
atmosphere given the two sources of {\it a priori} data: the first guess
(or background) ${\bf x^{b}}$ and observations ${\bf y^{o}}$
\citep{lorenc86}. The fit to individual data points is weighted by
estimates of their errors: ${\bf B}$, ${\bf E}$, and ${\bf F}$ are the
background, observation (instrumental), and representivity error
covariance matrices, respectively. Representivity error is an estimate of
inaccuracies introduced in the observation operator $H$ used to
transform the gridded analysis ${\bf x}$ to observation space ${\bf
y}=H({\bf x})$ for comparison against observations. This error will be
resolution dependent and may also include a contribution from
approximations (e.g., linearizations) in $H$.
As described in \citet{barker04}, the particular variational data
assimilation algorithm adopted in WRF-Var is a model-space, incremental
formulation of the variational problem. In this approach, observations,
previous forecasts, their errors, and physical laws are combined to
produce analysis increments ${\bf x^{a'}}$, which are added to the first
guess ${\bf x^{b}}$ to provide an updated analysis.
Figure \ref{var-sketch} illustrates the relationship between WRF-Var,
the various datasets, and the other components of a typical NWP system
(here ARW). The WRF-Var assimilation proceeds as described in
\citet{barker04}. A number of recent upgrades to the WRF-Var algorithm
will be described in Section \ref{var-upgrade}.
%
% Figure 9.1 var-sketch
%
\begin{figure}
\centering
\includegraphics[width=6.5in]{figures/var-sketch.pdf}
\caption{\label{var-sketch}Sketch showing the relationship between datasets (circles),
and algorithms (rectangles) of the ARW system.}
\end{figure}
The three inputs to WRF-Var are:
\vspace{0.5cm}
a) First guess ${\bf x^{b}}$--- In cold-start mode, this is typically a
forecast/analysis from another model interpolated to the ARW grid (and variables) via the
WRF SI and {\it real} programs. In cycling mode, the first guess
is a short-range (typically 1--6 hour) ARW forecast.
\vspace{0.5cm}
b) Observations ${\bf y^{o}}$--- In the current version of WRF-Var, observations may be
supplied either in PREPBUFR format ({\it ob\_format=1}) or an ASCII "little\_r" format
({\it ob\_format=2}). An observation preprocessor (3DVAR$\_$OBSPROC)
is supplied with the code release to perform basic quality control, assign "total"
observation errors (${\bf R = E+F}$ in Fig. \ref{var-sketch}), and reformat observations from the MM5 {\it little$\_$r} text
format into 3D-Var's own text format. Details can be found in \citet{barker03, barker04}.
\vspace{0.5cm}
c) Background error covariances ${\bf B}$--- used to define the spatial
and multivariate response of the analysis to an observation. In
variational systems, these covariances are typically calculated
off-line, and significant tuning is required to optimize performance
for a particular application (e.g., \citet{ingleby01, wu02}). The
amount of work required to do this satisfactorily is significant, and
should not be underestimated. In order to assist the user, WRF
developers supply the following: i) a default set of statistics used
for the initial set up of a domain; ii) a utility {\it gen$\_$be}
(described in Section
\ref{var-be}) to process ensembles of forecasts into the appropriate control variable
space; and iii) diagnostic routines to assess the accuracy of
observation and background error statistics. These routines include
both innovation vector-based approaches \citep{hollingsworth86} and
variational tuning approaches \citep{desroziers01}.
Following assimilation of all data, an analysis ${\bf x^{a}}$ is produced that must be
merged with the existing lateral boundary conditions ${\bf x^{lbc}}$ in the {\it WRF\_BC}
utility (\citet{barker03}). At this stage, the {\it wrfbdy} lateral boundary condition
files (${\bf x^{lbc}}$) output of WPS/real is updated to make the lateral boundaries consistent with the analysis, and surface fields (e.g. SST) are also updated in the {\it wrfinput} analysis file.
\section{Improvements to the WRF-Var Algorithm}
\label{var-upgrade}
The latest version of WRF-Var (V3.0) contains a number of improvements relative
to that described in the MM5 3DVAR technical note (\citet{barker03}). These are described below.
It should also be noted that the public release of WRF-Var V3.0 contains only a subset of
the capabilities of the full WRF-Var system. In particular, the direct assimilation of radiances,
hybrid variational/ensemble data assimilation technique, and 4D-Var will be released once
funding to support these complex algorithms is available.
\subsection{Improved vertical interpolation}
The original WRF 3D-Var system described in \citet{barker04} used height
interpolation for all observation operators. If an observation is reported as a
function of pressure, then height is approximated using the
hydrostatic relation. This step introduces an unnecessary source of error.
The new WRF-Var system performs vertical interpolation in terms of the
original observed coordinate, i.e., pressure or height.
\subsection{Improved minimization and ``outer loop"}
Prior to WRF-Var V3.0, the default WRF-Var cost function minimization used a modified
version of the limited memory Quasi-Newton Method (QNM). In V3.0, an alternative
Conjugate Gradient Method (CGM) has been implemented. Unlike the QNM technique,
the CGM method restricts WRF-Var's inner loop to be completely linear. This limitation is dealt
with through the inclusion of an outer loop in WRF-Var, the purpose of which is to iterate towards
nonlinear solutions (e.g., observation operators, balance constraints, and the forecast itself in
4D-Var) using the WRF-Var analysis from the previous iteration as new first guess. The
outer loop is also used as a form of variational quality control as follows: observations are
rejected if the magnitude of the observation minus first guess differences are larger than a
specified threshold (typically several times the observation error standard deviation). This {\it errormax} test implicitly assumes the first guess is accurate. However, in cases when this assumption breaks
down (i.e. in areas of large forecast error), there is a danger that good observations might be rejected in areas where they are most valuable. The outer loop alleviates this effect by allowing observations
rejected in previous iterations to be accepted if their updated observation minus analysis differences
pass the errormax QC check in in subsequent outer loops. The assimilation of nearby observations in previous iterations essentially provides a ``buddy check" to the observation in question.
\subsection{Choice of control variables}
\label{var-cvs}
A major change that users of previous versions of WRF-Var will notice, is the simplification
of the background error covariance model used within WRF-Var. As before, the background error covariance matrix ${\bf B}$ is computed not in model space ${\bf x}': u, v, T, q, p_{s}$, but in a
control variable space ${\bf v}$ related to model space via the control variable transform ${\rm U}$,
i.e.,
\begin{equation}
{\bf x}' = {\rm U}{\bf v}= {\rm U}_{p} {\rm U}_{v} {\rm U}_{h}{\bf v}.
\label{var-cv}
\end{equation}
The expansion ${\rm U}={\rm U}_{p}
{\rm U}_{v} {\rm U}_{h}$ represents the various stages of covariance modeling: horizontal correlations ${\rm U}_{h}$, vertical covariances ${\rm U}_{v}$, and multivariate covariances
${\rm U}_{p}$.
The components of ${\bf v}$ are chosen so that their error cross-correlations are negligible,
thus permitting the matrix ${\bf B}$ to be block-diagonalized. The major change in WRF-Var V3.0
is to simplify the control variable transform ${\rm U_p}$ to perform a simple statistical regression as described in subsection
(\ref{var-b}) below. Testing in numerous applications has shown
a general improvement of forecasts scores using this definition of balance, as compared to the dynamical geostrophic//cyclostrophic balance constraint defined in \citet{barker03}.
\subsection{First Guess at Appropriate Time (FGAT)}
A First Guess at Appropriate Time (FGAT) procedure has been implemented in
WRF-Var \citep{lee04}.
The FGAT capability results in a more accurate calculation of innovation vectors
(observation minus first guess difference), and hence a better use of observations when
their valid time differs from that of the analysis.
FGAT is most effective for the analysis of observations from
asynoptic, moving platforms (e.g., aircraft and satellite data).
\subsection{Radar Data Assimilation}
A capability to assimilate Doppler radar radial velocity
and reflectivity observations is available in WRF-Var
\citep{xiao05, xiao07, xiao072, xiao08}.
In order to calculate the vertical velocity increment as a result of
assimilating the vertical velocity component of radial velocity,
the Richarson balance equation, which combines the continuity
equation, adiabatic thermodynamic equation and hydrostatic
relation, and its linear and adjoint codes are introduced.
For reflectivity assimilation, total water is used as a control variable.
This requires a partitioning
between water vapor and hydrometeor increments during the minimization procedure.
A warm-rain parameterization is included to assist the calculation
of hydrometeors, which includes condensation of water vapor
into cloud, accretion of cloud by rain, automatic
conversion of cloud to rain, and evaporation of rain to water vapor.
The observation operators for Doppler radial velocity
and reflectivity are included.
\subsection{Unified Regional/Global 3D-Var Assimilation}
There are many benefits to having a single data assimilation system
for regional and global applications. The majority of the code is
common to both (observation operators, minimization, background error
preconditioning, interpolation, etc.). Technically, the main changes
required to extend the regional application to global are related to
the presence of a) the polar singularity, and b) periodic East-West
boundary conditions. Of course, there are also scientific questions
concerning the optimal mix of observations required for
global/regional models, and the choice of control variables and
balance constraints. A unified global/regional data assimilation system should
therefore be flexible to a variety of thinning/quality-control
algorithms and also to alternative formulations of the background
error covariance matrix. This flexibility has been a key design
requirement throughout the WRF-Var project.
The major difference between regional and global options in WRF-Var is
in the definition of horizontal background error covariances. In
regional mode, recursive filters \citep{purser03} are used to project
observation information away from the observation location. In global
mode, a spectral decomposition is applied. The main benefits of the
spectral technique are a) the isotropic and homogeneous covariances
that are implied neatly solve the problems associated with the pole
(the pole is not a special point in spectral space), and b) horizontal
correlations are defined in terms of a single function--- the power
spectrum (a function of total wavenumber). However, the isotropy of
correlation defined in spectral space is also a weakness---
anisotropies need to be defined in an alternative manner. One solution
to this problem is to replace the spectral correlations with
grid-point correlations \citep{purser03}. An alternative
technique is to supplement the isotropic spectral correlations with an
anisotropic component derived via grid transformations, additional
control variables or 4D-Var. Research using the latter techniques is
underway using the WRF-Var system.
The WRF-Var code has been adapted to perform assimilation on a global, regular
latitude-longitude grid. The major modifications are as follows.
\vspace{0.5cm}
a) Spectral to grid-point transformations (and their adjoints) have been included in 3D-Var
to represent the horizontal component (${\rm U}_h$) of the background error covariance
model.
\vspace{0.5cm}
b) A new global WRF-Var registry was created to accommodate the
output of global forecast models (currently only the Korean Meteorological
Administration's (KMA) global model has been tested).
The final analysis increments are written in binary format and added back to the global
first guess to provide the global analysis in the native model format.
\vspace{0.5cm}
c) Changes have been made to allow for periodic boundary conditions in the East--West
direction.
\vspace{0.5cm}
d) A number of minor changes have been made to treat the polar rows as special points
(e.g., in the grid-point $\psi, \chi$ to $u,v$ wind conversion in the ${\rm U}_p$ transform and the
observation operators for polar winds).
\section{Background Error Covariances}
\label{var-be}
Forecast (``first guess" or ``background") error covariances are a vital input to variational
data assimilation systems. They influence the analysis fit to observations and also
completely define the analysis response away from observations. The latter impact is
particularly important in data-sparse areas of the globe. Unlike ensemble filter data
assimilation techniques (e.g., the Ensemble Adjustment Kalman Filter, the Ensemble
Transform Kalman Filter), 3/4D-Var systems do not explicitly evolve forecast error
covariances in real-time (although both 4D-Var and hybrid variational/ensemble data assimilation techniques currently being developed within WRF-Var implement flow-dependent covariances implicitly). Instead, climatologic statistics are usually estimated offline.
The ``NMC-method", in which forecast error covariances are approximated using
forecast difference (e.g., T+48 minus T+24) statistics, is a commonly used approach
\citep{parrish92}. Experiments at ECMWF \citep{fisher03} indicate superior statistics may
be obtained using a cycling analysis/forecast ensemble prediction
system based on perturbed observations/physics.
Recent advances permit the use of flow-dependent forecast error
covariances in 3D/4D-Var through, for example, grid transformations
\citep{desroziers97}, anisotropic recursive filters
\citep{wu02, purser03},
or observation-space formulations of the variational
problem \citep{daley01}. Flow-dependence may be enhanced in 4D-Var
through the use of the forecast model to provide dynamical consistency to the evolving
forecast state during 4D-Var's time window \citep{rabier98}. Still, the practical effort
required to specify and implement flow-dependent error covariances in 3/4D-Var is
significant.
The development of a unified global/regional WRF-Var system, and its widespread use
in the WRF community has necessitated the development of a new, efficient, portable forecast background error covariance calculation code. Numerous applications have also indicated
that superior results are obtained if one invests effort in calculating domain-specific
error covariances, instead of using the the default statistics supplied with the WRF-Var
release. In this section, the new {\it gen$\_$be} code developed by NCAR/MMM to generate
forecast error statistics for use with the WRF-Var system is described.
The background error covariance matrix is defined as
\begin{equation}
{\bf B}=\overline{\epsilon \epsilon^{T}} \simeq \overline{{\bf x'}{\bf x'}^{T}},
\label{var-b}
\end{equation}
\noindent where the overbar denotes an average over time and/or geographical area. The true
background error $\epsilon$ is not known in reality, but is assumed to be statistically
well-represented by a model state perturbation ${\bf x'}$. In the standard NMC-method
\citep{parrish92}, the perturbation ${\bf x'}$ is given by the difference between
two forecasts (e.g., 24 hour minus 12 hour) verifying at the same time. Climatological
estimates of background error may then be obtained by averaging such forecast
differences over a period of time (e.g., one month). An alternative strategy proposed by
\citep{fisher03} makes use of ensemble forecast output, defining the ${\bf x'}$ vectors as
ensemble perturbations (ensemble minus ensemble mean). In either approach, the end
result is an ensemble of model perturbation vectors from which estimates of
background error may be derived. The new {\it gen$\_$be} utility has been designed to work with
either forecast difference, or ensemble-based, perturbations.
Using the NMC-method, ${\bf x}'={\bf x_{T2}}-{\bf x_{T1}}$ where $T2$ and $T1$
are the forecast difference times (e.g., 48h minus 24h for global, 24h minus 12h for regional).
Alternatively, for an ensemble-based approach, ${\bf x_{k}}'={\bf x_{k}}-\bar{\bf
x}$, where the overbar is an average over ensemble members $k=1,n_{e}$. The total
number of binary files produced by this stage is $n_{f} \times n_e$ where $n_f$ is the
number of forecast times used (e.g., for 30 days with forecast every 12 hours, $n_f=60$).
Using the NMC-method, $n_e=1$ (1 forecast difference per time). For ensemble-based
statistics, $n_e$ is the number of ensemble members.
As described above, the WRF-Var background error covariances are specified not in
model space ${\bf x'}$, but in a control variable space ${\bf v}$, which is related to the model variables
(e.g., wind components, temperature, humidity, and surface pressure) via the control
variable transform defined in (\ref{var-cv}). Both (\ref{var-cv}) and
its adjoint are required in WRF-Var. To enable this, the (offline) background error utility is used
to compute components of the forecast error covariance matrix modeled within the
${\rm U}$ transform. This process is described in the following subsections.
The background error covariance generation code {\it gen$\_$be} is designed to process
data from a variety of regional/global models (e.g., ARW, MM5, KMA global model,
NFS, etc.), and process it in order to provide error
covariance statistics for use in variational data assimilation systems. The initial,
model-dependent ``stage 0" is illustrated in Fig. \ref{var-genbe0}.
%
% Figure 9.6 var-genbe0
%
\begin{figure}
\centering
\includegraphics[width=4.0in]{figures/var-genbe0.pdf}
\caption{\label{var-genbe0}Sketch of the role of Stage 0 converters
in transforming model-specific data (e.g., ARW, KMA global model, etc.) to standard
perturbation fields and relevant metadata (e.g., latitude, height, land/sea, etc.).}
\end{figure}
Alternative models use different grids, variables, data formats, etc., and so initial converters
are required to transform model output into a set of standard perturbation fields (and metadata),
and to output them in a standard binary format for further, model-independent processing. The
standard grid-point fields are as follows.
\begin{itemize}\setlength{\parskip}{-4pt}
\item
Perturbations: Streamfunction $\psi'(i,j,k)$, velocity potential $\chi'(i,j,k)$,
temperature $T'(i,j,k)$, relative humidity $r'(i,j,k)$, surface pressure $p_s'(i,j)$.
\item
Full-fields: height $z(i,j,k)$, latitude $\phi(i,j)$. (These are required for the
production of background error statistics stored in terms of physics variables,
rather than tied to a specific grid. This flexibility is included in {\it gen$\_$be} through a
namelist option to define the bins over which data is averaged in a variety of ways
(e.g., latitude height, grid points). Land-sea and orographic effects may also be
represented in this way.
\end{itemize}
In general, the {\it stage$\_$0} converters are developed and maintained by those supporting
individual models. Only the WRF-to-standard-fields converter {\it gen$\_$be$\_$stage0$\_$wrf}
is maintained and supported by the ARW effort.
\subsection{Removal of time-mean}
In order to calculate covariances between fields, the average value must first be removed.
This is performed in the first stage utility {\it gen$\_$be$\_$stage1}.
\subsection{Multivariate Covariances: Regression coefficients and unbalanced variables}
The second stage
of {\it gen$\_$be (gen$\_$be$\_$stage2)} provides statistics for the
unbalanced fields $\chi_u$, $T_u$, and $P_{su}$ used as control
variables in WRF-Var. The unbalanced control variables are defined as
the difference between full and balanced (or correlated) components of
the field. In this stage of the calculation of background errors, the
balanced component of particular fields is modeled via a regression
analysis of the field using specified predictor fields
(e.g., streamfunction; see
\citet{wu02} for further details). The resulting regression coefficients
are output for use
in WRF-Var's ${\rm U}_p$ transform. Currently, three regression analyses are
performed resulting in three sets of regression coefficients (note:
The perturbation notation has been dropped for the
remainder of this chapter for clarity.):
\begin{itemize}\setlength{\parskip}{-4pt}
\item Velocity potential/streamfunction regression: $\chi_b(k)=c(k)\psi(k)$;
\item Temperature/streamfunction regression: $T_b(k)=\sum_{k1}G(k1,k)\psi(k1)$; and
\item Surface pressure/streamfunction regression: $p_{sb}=\sum_{k1}W(k1)\psi(k1)$.
\end{itemize}
The summation over the vertical index $k1$ relates to the integral (hydrostatic) relationship between
mass fields and the wind field. By default, the regression coefficients $c$, $G$, and $W$ do
not vary horizontally, however options exists to relax this assumption via the {\it bin\_type}
namelist variable in order to allow representation of differences between, for example, polar, mid-latitude, and tropical dynamical and physical processes. The scalar coefficient $c$ used to
estimate velocity potential errors from those of streamfunction is permitted to vary with model
level in order to represent, for example, the impact of boundary-layer physics. Latitudinal/height
smoothing of the resulting coefficients may be optionally performed to avoid artificial
discontinuities at the edges of latitude/height boxes (see the future WRF-Var technical note for
details of these "expert" features).
Having computed regression coefficients, the unbalanced components of the fields are
calculated as $\chi_{u}(k)=\chi(k)-c(k)\psi(k)$, $T_{u}(k)=T(k)-\sum_{k1}G(k1,k)\psi(k1)$,
and $p_{su}=p_s - \sum_{k1} W(k1)\psi(k1)$. These fields are output for the
subsequent calculation of the spatial covariances as described below.
\subsection{Vertical Covariances: Eigenvectors/eigenvalues and
control variable projections}
The third stage ({\it gen$\_$be$\_$stage3}) of {\it gen$\_$be}
calculates the statistics required for the vertical component of the
control variable transform. This calculation involves the projection
of 3D fields on model-levels onto empirical orthogonal functions
(EOFs) of the vertical component of background error covariances
\citet{barker04}. For each 3D control variable ($\psi$, $\chi_u$,
$T_u$, and $r$), the vertical component of ${\bf B}$, is calculated
and an eigenvector decomposition performed. The resulting eigenvectors
${\bf E}$ and eigenvalues $\Lambda$ are saved for use in WRF-Var.
The {\it gen$\_$be} code calculates both domain-averaged and local
values of the vertical component of the background error covariance
matrix. Eigendecomposition of the resulting $K\times K$ ($K$ is the number of
vertical levels) climatological vertical error covariance matrix ${\bf
B}={\bf E}{\Lambda}{\bf E}^{T}$ results in both domain-averaged and
local eigenvectors $\bf E$ and eigenvalues $\Lambda$. Both sets of
statistics are included in the dataset supplied to WRF-Var, allowing
the choice between homogeneous (domain-averaged) or local
(inhomogeneous) background error variances and vertical correlations
to be chosen at run time \citet{barker04}.
Having calculated and stored eigenvectors and eigenvalues, the final
part of {\it gen$\_$be$\_$stage3} is to project the entire sequence of
3D control variable fields into EOF space ${\bf v_v}=U_{v}^{-1}{\bf
v_p}=\Lambda^{-1/2}{\bf E}^{T} {\bf v_p}$.
\subsection{Horizontal Covariances: Recursive filter lengthscale (regional), or power
spectra (global)}
The last aspect of the climatological component of background error
covariance data required for WRF-Var is the horizontal error
correlations, the representation of which forms the largest difference
between running WRF-Var in regional and global mode - the rest of
{\it gen\_be} is essentially the same for both regional and global models.
In a global application ({\it gen\_be\_stage4\_global}), power spectra
are computed for each of the $K$ vertical modes of the 3D control
variables $\psi$, $\chi_u$, $T_u$, and relative humidity $r$, and for the 2D control
variable $p_{su}$ data. In contrast, in regional mode, horizontal
correlations are computed between grid-points of each 2D field, binned
as a function of distance. A Gaussian curve is then fitted to the data
as described in \citet{barker04} to provide correlation lengthscales
for use in the recursive filter algorithm.
\section{WRF-Var V3.0 Software Engineering Improvements}
\label{se}
A major overhall of the WRF-Var software has been performed for V3.0. The following
is a summary:
\subsection{Memory improvements}
The WRF-Var registry had become bloated with WRF and U4D-Var 2d and 3d state variables that were unused in
3D-Var applications. These variables were allocated but uninitialised, and written to the analysis files. The removal
of these dummy variable has resulted in a significant (10-50\% depending on application) reduction in the memory
requirements of WRF-Var.
\subsection{Four-Byte I/O}
The WRF-Var algorithm requires eight-byte precision internally. However, it only needs to read and write 4-byte files.
Switching from 8-byte to 4-byte output in V3.0 has improved I/O performance and halves file sizes.
\subsection{Switch from RSL to RSL\_LITE}
The switch from RSL to RSL\_LITE has been made in V3.0 as the latter possesses
a simpler "lighter" communications layer, and has been shown to be scalable
to arbitrary domain sizes (largest to date: 4500x4500) and numbers of processors
(largest to date: 64K processors on Blue Gene). RSL\_LITE supports all capabilities of WRF,
including Halo and periodic boundary exchanges, Distributed I/O, Nesting and moving nests,
and parallel transposes. RSL\_LITE has also been simplified by dropping irregular decomposition,
load balancing, and ragged edge nesting, and the initialisation techniques improved
to avoid recalculation and give better scaling at higher processor counts.
\subsection{Reorganisation of observation structures}
The F90 derived data types used for observations have been rewritten to permit batches
of observations to be passed to subroutine calls, especially interpolation ones and
subsequently makes better use of cache memory.
\subsection{Radar reflectivity operators redesigned}
The efficiency of the coding of the radar observation operators has been improved.
Previously, routines were called once per observation, which would then recalculate common
factors before performing what was often just a one-line calculation. Re-writing the code
in V3.0 to move the calculations inside loops in the calling routine allows them to work
on batches of observations at a time, vastly improving cache hit rates and eliminating recalculation.