DWR_Hydrologic_Modeling_Validation

Hydrologic Modeling Validation
&
DWR’s use of WaterFALL
November 27, 2012
Ecological Flow Science Advisory Board
Tom Fransen
Division of Water Resources
NC Department of Environment and Natural Resources
Modeling and Ecological Flows
Issues/Concerns
Historically the models have focused on
water supply (municipal and industrial)
reliability.
Larger streams and rivers that support or the
potential to support withdrawal and discharges of
100,000 gpd or greater.
Calibration and validation is concentrated on
normal and low flow periods, when the water
supplies are stressed.
•Normal 25th to 75th percentiles
•The 20th Percentile to the Low Flow Minimum of Record
(20th percentile is the beginning of D1 – Moderate Drought conditions.)
Slide - 2
Terminology
Calibration is the estimation and adjustment of model parameters and
constants to improve the agreement between model output and a data set.
Models contain variables and constants that need to be parameterized.
Calibration improves the parameterization of a model. Calibration can occur
as part of either verification or validation.
Validation is a demonstration that a model within its domain of applicability
possesses a satisfactory range of accuracy consistent with the intended
application of the model, refers to its performance. It refers to the processes
and techniques that are used to assure that the model represents the real
system (or proposed real system) to a sufficient level of accuracy. Validation
compares simulated system output with real system observations using
data not used in model development.
Certification encompasses calibration and validation and includes such
things as documentation, quality assurance, and version control.
Certification is intended to put a "seal" of credibility on a model. Certification
means that a certain standard has been satisfied which offers some high
level guidance on verification and validation of the model.
Slide - 3
How good is good enough?
We’re not making watches.
To answer the question “How good is good
enough” depends upon the intended uses of a
model. DWR’s models are planning tools for
water resources planning, NOT a regulatory
tool for water allocation.
Slide - 4
How good is good enough?
Is the model good enough to meet the
requirements of SL 2010-143:
Locations the yield may be inadequate to meet all
needs or meet all essential water uses:
•Determine the yield of reservoirs?
•Determine the yield of run-of-river intakes?
•Determine the yield of wells?
Groundwater resources within the river basin to the extent known by
the Department.
Determine the location ecological flows are
adversely impacted?
Slide - 5
How good is good enough?
Is the model good enough to meet the non-SL
2010-143 uses of the model?
Water resources planning issues (not covered in
SL 2010-143):
•Screening of alternative water supplies?
•Are water shortage response plan triggers adequate &
how to improve the triggers?
•Hydroelectric generation impacts?
•Recreational impacts – river and lake?
Real-time drought management – forecasting?
Slide - 6
DWR’s New Validation Approach
The inflow datasets are created using gage extension
techniques on monthly USGS streamflow data. Since
the planning efforts are focused on the larger streams,
rivers, and reservoir that can support a 100,000 gpd or
larger withdrawal, having a good annual and monthly
validation is adequate. DWR includes the daily
analyzes to check to be sure there is nothing outline
that will impact the model results. Analyzes are done at
streamflow nodes with good long-term USGS records
and existing reservoirs with good operations records.
Slide - 7
Overall Goodness-Of-Fit (GOF)
Standard Model Goodness-Of-Fit Statistics
•Nash-Sutcliffe efficiency (NSE) – satisfactory NSE > 0.50
•Ratio of the root mean square error to the standard
deviation of the measured data (RSR) –
satisfactory ≤ 0.70
•Percent bias (PBIAS) – satisfactory PBIAS ≤ ± 25%
•Coefficient of determination (R2) – satisfactory R
2 > 0.5
Qualitative
•Hydrograph
•Frequency Curve – full range and low-flow ( ≤ 30%)
•Residuals – normalized as a percent difference
•Scatter Plot – gage vs. model data
Slide - 8
S.L. 2010-143 & Water Resources Planning
GOF Measures
Reservoir and Run-Of-River Intake Yield
•Period-Of-Record mass curve analysis
Ground Water Yield
•Baseflow analysis
Ecological Flows (place holder)
•Low-Flow Log-Pearson III 7-Day
•Depending upon the SAB’s recommendation, we
probably will need to rethink the normal – low-flow
approach.
Drought Planning
•Low-Flow Log-Pearson III 30-Day
•Drought Period Analysis based on National Drought
Monitor
Slide - 9
Overall Goodness-Of-Fit (GOF)
Hydrograph
Slide - 10
Overall Goodness-Of-Fit (GOF)
Frequency Curve
Slide - 11
Overall Goodness-Of-Fit (GOF)
Residuals
Slide - 12
Overall Goodness-Of-Fit (GOF)
Scatter Plot
Slide - 13
Water Resources Planning GOF Measures
Period-Of-Record mass curve analysis
Slide - 14
Water Resources Planning GOF Measures
Baseflow analysis
Slide - 15
Water Resources Planning GOF Measures
Low-Flow Log-Pearson III 7-Day & 30-Day
Slide - 16
DWR’s use of WaterFALL
DWR is still working with RTI to
determine if WaterFALL is adequate to
meet the requirements of SL 2010-143.
If WaterFALL is determined to be
adequate, DWR use will be to generate
alterative inflow scenarios for OASIS:
1.Climate Change
2.Land use change
Slide - 17
DWR’s use of WaterFALL
RTI and Hydrologics have
demonstrated that WaterFALL can
provide the output needs to create the
alternative inflows that can be used in
OASIS.
One practical unanswered question is
how much will RTI charge?
Slide - 18
Questions
Contact Information
Tom Fransen, Deputy Director
[e-mail]
919-707-9015
Slide - 19