MADIS Meteorological Surface Quality Control

This page provides a detailed description of the quality control (QC) processing and data structures for the meteorological surface dataset. For a general overview of MADIS quality control, click here.

Automated Quality Control

Level 1 QC checks are considered the least sophisticated, level 3 the most sophisticated checks. The following table lists the surface variables* that are QC'ed, and the checks that are used:

MADIS Surface Variables with QC
------------------------------------------------------------------------------------------------------------------------
Code       Name                                Max     | Level 1: |                 Level 2:               |   Level 3:
                                             Possible  |          |                                        |
                                             QC Level  | Validity |   Internal      Temporal   Statistical |   Spatial
                                                       |          | Consistency   Consistency  Spatial     | Consistency
                                                                                               Consistency |
------------------------------------------------------------------------------------------------------------------------
TD        dewpoint temperature                 3            X            3             X            X             X
TD1H      hourly average dewpoint temperature  2            X                          X
TD15M     dewpoint temperature - 15m           3            X                          X            X             X
TD1HCHG   1 hour dewpoint temperature change   1            X
RH        relative humidity                    3            X            3             X            X             X
RH1H      hourly average relative humidity     2            X                          X
RH15M     relative humidity - 15m              3            X                          X            X             X
RH1HCHG   1 hour relative humidity change      1            X
Q         specific humidity                    3            X            3             X            X             X
Q1H       hourly average specific humidity     2            X                          X
Q15M      specific humidity - 15m              3            X                          X            X             X
Q1HCHG    1 hour specific humidity change      1            X
DPD       dewpoint depression                  3            X            3             X            X             X
DPD1H     hourly average dewpoint depression   2            X                          X
DPD15M    dewpoint depression - 15m            3            X                          X            X             X
DPD1HC    1 hour dewpoint depression change    1            X
AH        absolute humidity                    3            X            3             X            X             X
AH1H      hourly average absolute humidity     2            X                          X
AH15M     absolute humidity - 15m              3            X                          X            X             X
AH1HCHG   1 hour absolute humidity change      1            X
WVMR      water vapor mixing ratio             3            X            3             X            X             X
WVMR15M   water vapor mixing ratio - 15m       3            X                          X            X             X
WVMR1HC   1 hour water vapor mixing ratio chng 1            X
ALTSE     altimeter                            3            X                          X            X             X
ALTS15M   altimeter - 15m                      3            X                          X            X             X
AL1HCHG   1 hour altimeter change              1            X
PT3       3 hour pressure change               2            X            2
SLP       sea level pressure                   3            X            1             X            X             X
P         station pressure                     3            X            1             X            X             X
P15M      station pressure - 15m               3            X                          X            X             X
T         air temperature                      3            X            3             X            X             X
T1H       hourly average temperature           2            X                          X
T15M      air temperature - 15m                3            X                          X            X             X
T1HCHG    1 hour temperature change            1            X
ARCHVT    archived temperature                 3            X                          X            X             X
TV        virtual temperature                  3            X            3             X            X             X
TV1H      hourly average virtual temperature   2            X                          X
TV15M     virtual temperature - 15m            3            X                          X            X             X
TV1HCHG   1 hour virtual temperature change    1            X
DD        wind direction                       3            X                          X            X             X
DD1H      hourly average wind direction        1            X
DD15M     average wind direction - 15m         1            X
DDM15M    wind direction mean vector - 15m     1            X
DDU15M    mean unit vector wind dir - 15m      1            X
DD24H     daily average wind direction         1            X
DDSD15M   wind direction std dev - 15m         1            X
FF        wind speed                           3            X                          X            X             X
FF1H      hourly average wind speed            2            X                          X
FF15M     wind speed - 15m                     3            X                          X            X             X
FF24H     daily average wind speed             1            X
FFSD15M   wind speed std dev - 15m             3            X                          X            X             X
U         u wind component                     3            X                          X            X             X
U1H       hourly average u wind component      2            X                          X
U15M      u mean unit vector wind comp - 15m   3            X                          X            X             X
U24H      daily average u wind component       1            X
USD15M    u wind component std dev - 15m       3            X                          X            X             X
V         v wind component                     3            X                          X            X             X
V1H       hourly average v wind component      2            X                          X
V15M      v mean unit vector wind comp - 15m   3            X                          X            X             X
V24H      daily average v wind component       1            X
VSD15M    v wind component std dev - 15m       3            X                          X            X             X
VIS       visibility                           1            X
PCP5M     accumulated precip - 5m              1            X
PCP15M    accumulated precip - 15m             1            X
PCP1H     accumulated precip - 1h              1            X
PCP3H     accumulated precip - 3h              1            X
PCP6H     accumulated precip - 6h              1            X
PCP12H    accumulated precip - 12h             1            X
PCP18H    accumulated precip - 18h             1            X
PCP24H    accumulated precip - 24h             1            X
PCPLM     accum. precip - since local midnight 1            X
PCPUTCM   accum. precip - since UTC midnight   1            X
PCPCDAY   accum. precip - local calendar day   1            X
PCPUDAY   accum. precip - user-defined day     1            X
PCPRATE   precipitation rate                   1            X
ARCHPCP   1h archived precip accumulation      1            X
SOILMP    soil moisture percent                1            X
SOILM2    soil moisture percent - 2 inch       1            X
SOILM4    soil moisture percent - 4 inch       1            X
SOILM8    soil moisture percent - 8 inch       1            X
SOILM20   soil moisture percent - 20 inch      1            X
SOILM40   soil moisture percent - 40 inch      1            X
SOILT     soil temperature                     2            X                          X
SOILT2    soil temperature - 2 inch            2            X                          X
SOILT4    soil temperature - 4 inch            2            X                          X
SOILT8    soil temperature - 8 inch            2            X                          X
SOILT20   soil temperature - 20 inch           2            X                          X
SOILT40   soil temperature - 40 inch           2            X                          X
DDGUST    wind dir at gust                     1            X
FFGUST    wind gust                            2            X
FF24MAX   daily maximum wind speed             1            X
T24MIN    24 hour minimum temperature          1            X
T24MAX    24 hour maximum temperature          1            X
DDMAX1H   wind dir at hourly max wind speed    1            X
FFMAX1H   hourly maximum wind speed            2            X                          X
SNOWC     snow cover                           1            X
SNOW6H    snow fall - 6h                       1            X
SNOW24H   snow fall - 24h                      1            X
SST       sea surface temperature              2            X            5             X


Internal Consistency Checks
---------------------------

  1   sea level pressure vs. station pressure
  2   3h pressure change vs. station pressure
  3   air temperature vs. dewpoint temperature
  5   sea surface temperature vs. air temperature

The level 1 validity checks restrict each observation to falling within a TSP-specified set of tolerance limits, while the level 2 temporal consistency checks restrict the temporal rate of change of each observation to a set of (other) TSP-specified tolerance limits. In both cases, observations not falling within the limits are flagged as failing the respective QC check. The following table lists the tolerance limits:


  -----------------------------------------
  Validity Checks
  -----------------------------------------
  Dewpoint temperature       -90 -   90   F
  Relative humidity            0 -  100  %
  Relative humidity 1hr chng -50 -   50  %
  Altimeter                  568 - 1100  mb
  Altimeter 1hr change       -10 -   10  mb
  Pressure change              0 - 30.5  mb
  Sea level pressure         846 - 1100  mb
  Station pressure           568 - 1100  mb
  Air temperature            -60 -  130   F
  Air temperature 1hr change -35 -   35   F
  Wind Direction               0 -  360  deg
  Wind Speed                   0 -  250  kts
  Visibility                   0 -  100 miles
  Accumulated precip - *h      0 -   44  in
  Precipitation rate           0 -   44  in
  Soil moisture percent        0 -  100  %
  Soil temperature           -40 -  150   F  
  Wind dir at gust             0 -  360  deg
  Wind gust                    0 -  287  mph
  24 hour min temperature    -60 -  130   F
  24 hour max temperature    -60 -  130   F
  Wind dir at hourly max       0 -  360  deg
  Wind speed                   0 -  287  mph
  Hourly maximum wind speed    0 -  287  mph
  Snow cover                   0 -   25  ft
  Snow fall - 6h               0 -   50  in
  Snow fall - 24h              0 -  300  in
  Sea surface temperature   28.4 -  104   F

  ---------------------------------------------
  Temporal Consistency Checks
  ---------------------------------------------
  Dewpoint temperature             35  F/hour
  Sea level pressure               15  mb/hour
  Air temperature                  35  F/hour
  Wind speed                       20 kts/hour
  Soil temperature                  5  F/hour
  Sea surface temperature           9  F/hour

The level 2 internal consistency checks enforce reasonable, meteorological relationships among observations measured at a single station. For example, a dewpoint temperature observation must not exceed the temperature observation made at the same station. If it does, both the dewpoint and temperature observation are flagged as failing their internal consistency check. Pressure internal consistency checks include a comparison of pressure change observations at each station with the difference of the current station pressure and the station pressure three hours previous, and a comparison of the reported sea-level pressure with a sea-level pressure estimated from the station pressure and the 12 hour mean surface temperature. In the former check, if the reported 3h pressure change observation does not match the calculated ob, then only the reported observation is flagged as bad. In the latter check, however, if the reported sea-level pressure does not match the calculated ob, then both the sea-level and station pressure obs are flagged as failing.

The level 2 statistical spatial consistency check uses weekly QC statistics to mark observations as failed if they failed any QC check 75% of the time during the previous 7 days. These observations will continue to be marked as failed by this check until such time as the failure rate falls below 25% in the weekly statistics. This check is only performed on observation types that go through the level 3 spatial consistency check.

The level 3 spatial consistency (or "buddy") check is performed using an Optimal Interpolation (OI) technique developed by Belousov et al. (1968). At each observation location, the difference between the measured value and the value analyzed by OI is computed. If the magnitude of the difference is small, the observation agrees with its neighbors and is considered correct. If, however, the difference is large, either the observation being checked or one of the observations used in the analysis is bad. To determine which is the case, a reanalysis to the observation location is performed by eliminating one neighboring observation at a time. If successively eliminating each neighbor does not produce an analysis that agrees with the target observation (the observation being checked), the observation is flagged as bad. If eliminating one of the neighboring observations produces an analysis that agrees with the target observation, then the target observation is flagged as "good" and the neighbor is flagged as "suspect." Suspect observations are not used in subsequent OI analyses. The following figure illustrates the reanalysis procedure:

To improve the performance of the OI, RSAS analysis fields from the previous hour are used as background grids. The analyses provide an accurate 1-h persistence forecast and allow the incorporation of previous surface observations, thus improving temporal continuity near stations that report less frequently than hourly. The differences between the observations and the background are calculated and then interpolated to each observation point before the OI analysis is performed. In addition, uniform distribution of the neighboring observations used in the spatial consistency check is guaranteed (whenever possible) by a search algorithm which locates the nearest observation in each of eight directional sectors distributed around the target observation.

Temperature observations are converted to potential temperature before application of the spatial consistency check. Potential temperature varies more smoothly over mountainous terrain when the boundary layer is relatively deep and well mixed, a marked advantage during daytime hours. For example, potential temperature gradients associated with fronts tend to be well defined during the day even in mountainous terrain (Sanders and Doswell 1995). Unfortunately, this advantage often disappears at night when cool air pools in valleys. To improve the efficacy of the spatial consistency check in these circumstances, elevation differences are incorporated to help model the horizontal correlation between mountain stations. (Miller and Benjamin 1992). The error threshold (to which the absolute value of the difference between analyzed and observed values is compared) is a function of the forecast error, the observational measurement error, and the expected analysis error (Belousov et al. 1968, pg. 128).

*It should be noted that while the QC checks discussed here are generally applied to the form of the variable stored in the database, the QC results will also be applied to any forms of the variable that are requested by the user and are derived from the primary variable. For example, specific humidity will get the QC results from the checks applied to dewpoint temperature.

Station Monitoring

For surface data only (at this time), the MADIS processing also keeps statistics on the frequency and magnitude of the observational errors encountered for NWS sea-level pressure, potential temperature, dewpoint, and surface wind. At the completion of each hourly analysis, the system provides the total number of observations for each variable, the number of observations that failed the QC check, the station names for the failed observations, and the error and threshold values for each of the failed observations. The error is defined as the difference between the QC analysis value and the observed value, as computed in the spatial consistency check described above.

Statistics are calculated for all stations. Stations from different networks are kept statistically separate. Specifically, the following stratifications are currently maintained: "ASOS," "SAO" (METAR manual), "AUTO" (METAR automated, but not ASOS), "BUOY," and "NPN" (NOAA Profiler Network). Local mesonets are stratified by provider. For example, "CDOT," for the Colorado Department of Transportation.

Current hourly, daily, weekly, and monthly QC messagesare available for the various surface observing networks.

Subjective Intervention

Two text files, a "reject" and an "accept" list provide the capability to subjectively override the results of the automated QC checks. The reject list is a list of stations and associated input observations that will be labeled as bad, regardless of the outcome of the QC checks; the accept list is the corresponding list of stations that will be labeled as good, regardless of the outcome of the QC checks. In both cases, observations associated with the stations in the lists can be individually flagged. For example, wind observations at a particular station may be added to the reject list, but not the temperature observations.

Here are the current subjective intervention lists in use:

QC and station monitoring procedures are not affected by subjective intervention lists, with the sole exception that observations on the reject list will be labeled as "suspect" and not used to check the spatial consistency of neighboring observations. This will allow MADIS personnel to continue to monitor the performance of the stations contained in the lists. For example, a station with wind observations that fail the QC checks a large percentage of the time may be added to the reject list. However, once the observation failure rate at the station falls back to near zero (possibly due to an anemometer that has been repaired), the station will likely be deleted from the list.

QC Data Structures

The MADIS QC information available for each variable includes the following QC structures: a single-character "data descriptor", intended to define an overall opinion of the quality of each observation by combining the information from the various QC checks, and for users desiring detailed information, a "QC applied" bitmap indicating which QC checks were applied to each observation, and a "QC results" bitmap indicating the results of the various QC checks.

The following table provides a complete list of the data descriptors and the bits used in the bitmaps:

  ------------------------------
  MADIS QC Information - Surface
  ------------------------------

  QC Data Descriptor Values
  -------------------------

  No QC available:

   Z - Preliminary, no QC

  Automated QC checks:

   C - Coarse pass, passed level 1
   S - Screened, passed levels 1 and 2
   V - Verified, passed levels 1, 2, and 3
   X - Rejected/erroneous, failed level 1
   Q - Questioned, passed level 1, failed 2 or 3

       where level 1 = validity
             level 2 = internal consistency, temporal consistency, 
                       statistical spatial consistency checks
             level 3 = spatial consistency check

  Subjective intervention:

   G - Subjective good
   B - Subjective bad

  Interpolated/Corrected observations:

   T - Virtual temperature could not be calculated, air temperature passing all QC 
       checks has been returned

  Bitmask for QC Applied and QC Results
  -------------------------------------

   Bit       QC Check                              Decimal Value
   ---       --------                              -------------
    1        Master Check                                 1
    2        Validity Check                               2
    3        Reserved                                     4
    4        Internal Consistency Check                   8
    5        Temporal Consistency Check                  16
    6        Statistical Spatial Consistency Check       32
    7        Spatial Consistency Check                   64
    8        Reserved                                   128
    9        Reserved                                   256
   10        Reserved                                   512

The QC bitmask is used in the QC applied and QC result "words" returned along with the QC data descriptor. By examining the individual bits, the user can determine which checks were actually applied, and the pass/fail status of each check that was applied.

In the QC applied word, a bit value of 1 means the corresponding check was applied, a bit value of 0 indicates the check wasn't applied.

In the QC results word, a bit value of 1 means the corresponding check was applied and failed, a bit value of 0 indicates the check passed (given that the check was applied).

The "Master Check" is used to summarize all of the checks in a single bit. If any check at all was applied, this bit will be set in the QC applied word. If the observation failed any QC check, it will be set in the QC results word.

When read as decimal numbers, the different bits that are set in the bitmask are summed together. For example, a QC applied value of 67 should be interpreted as 1 + 2 + 64, meaning the validity and spatial consistency checks were applied.


References

Belousov, S.L., L.S. Gandin, and S.A. Mashkovich, 1968: Computer Processing of Current Meteorological Data. Ed. V. Bugaev. Meteorological Translation No. 18, 1972, Atmospheric Environment Service, Downsview, Ontario, Canada, 227 pp.

Miller, P.A. and S.G. Benjamin, 1992: A System for the Hourly Assimilation of Surface Observations in Mountainous and Flat Terrain. AMS Monthly Weather Review, 120, 2342-2359.

Sanders, F., C.A. Doswell III, 1995: A case for detailed surface analysis. Bull. Amer. Meteor. Soc., 76, 505-521.

Technique Specification Package 88-21-R2 For AWIPS-90 RFP Appendix G Requirements Numbers: Quality Control Incoming Data, 1994. AWIPS Document Number TSP-032-1992R2, NOAA, National Weather Service, Office of Systems Development.


Last updated 16 February 2007.
Back to MADIS Home Page