Loading...
HomeMy WebLinkAboutStatistical Methods for Determining BTVs_01202017 REVISED DRAFT Statistical Methods for Developing Reference Background Concentrations for Groundwater and Soil at Coal Ash Facilities January 2017 Prepared By: HDR Engineering, Inc. 440 S. Church St, Suite 1000 Charlotte, NC 28202 and SynTerra Corporation 148 River Street, Suite 220 Greenville, South Carolina 29601 REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 i | Page CONTENTS INTRODUCTION .......................................................................................................................... 1 PART I – DESCRIPTION OF BACKGROUND DATA SETS ................................................... 4 Groundwater .......................................................................................................................... 4 Soil ......................................................................................................................................... 4 PART II – PRELIMINARY DATA ANALYSIS .......................................................................... 5 1. Descriptive Statistics ............................................................................................... 5 2. Graphical Analysis ................................................................................................... 5 3. Identify Outliers ........................................................................................................ 6 4. Identifying Data Distributions ................................................................................... 6 5. Evaluating Background Groundwater Data ............................................................. 7 6. Autocorrelation ........................................................................................................ 7 7. Seasonality .............................................................................................................. 8 8. Trends ...................................................................................................................... 8 9. Additional Methods for Identifying Trends in Background Groundwater Data ........ 9 10. Determining Baseline Period for Background Wells .............................................. 10 PART III – TESTING FOR SUB-GROUPS IN BACKGROUND GROUNDWATER DATA ... 11 Graphical Analysis ............................................................................................................... 11 Analytical Tests for Comparing Sub-Groups ....................................................................... 12 Tests for Identifying Differences Among Sub-Groups ......................................................... 12 PART IV – DEVELOPMENT OF BTVs FOR CONSTITUENTS IN GROUNDWATER AND SOIL ...................................................................................................................... 13 REFERENCES ........................................................................................................................... 15 REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 ii | Page FIGURES Figure 1. Box-and-Whisker Plot Figure 2. Quantile-Quantile (Q-Q) Plot Figure 3. Scatter Plot of Time versus Concentration Figure 4. Sample Autocorrelation Function Figure 5. Scatter Plots of Time versus Concentration Illustrating Seasonality Figure 6. Scatter Plots of Time versus Concentration Illustrating Trends Figure 7. Piece-Wise Polynomial Regression Output Exhibiting Multiple Trends Figure 8. Empirical Distribution Plot Comparing Constituent Concentrations between Two Seasons TABLES Table 1. Chemical Parameters Analyzed in Groundwater and Soil REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 1 | Page INTRODUCTION In 2015, the North Carolina Coal Ash Management Act (CAMA) required the preparation of a Comprehensive Site Assessment Report (CSA) for each regulated facility. The purpose of the CSA was to identify the source and cause of exceedances of regulatory standards, potential hazards to public health and safety, and identify receptors and exposure pathways. The CSA was conducted in accordance with a conditionally approved Work Plan to meet the requirements of 15A NCAC 02L .0106(g), which includes an assessment of the horizontal and vertical extent of soil and groundwater contamination for all contaminants confirmed to be present in groundwater in exceedance of groundwater quality standards. Regulations regarding North Carolina groundwater quality standards provided in T15A NCAC 02L .0202. Section (b)(3) of the regulation state that: Where naturally occurring substances exceed the established standard, the standard shall be the naturally occurring concentration as determined by the Director. For soil and groundwater assessments under the CAMA, naturally occurring concentrations of constituents need to be determined in order to complete horizontal and vertical delineations required as a basis for development of Corrective Action Plans. The horizontal and vertical extent of constituent migration cannot be determined until naturally occurring background concentrations are known. This document serves as a framework for a consistent technical approach which will be utilized for Duke Energy sites in North Carolina to determine proposed provisional background threshold values (PPBTVs1) for naturally occurring constituents in groundwater and soil. For the purpose of establishing background threshold values (BTVs1) at this time, the value which represents the upper threshold value from the upper tail of the data distribution for a given constituent will be considered the value representative of a naturally occurring concentration, or the PPBTV. The process for evaluating background concentrations over time is iterative; therefore, as additional background data is collected, the approach for developing BTVs may be reviewed and potentially modified with consideration of expanded data sets, changes in data set distribution, and input from the North Carolina Department of Environmental Quality (NCDEQ). For groundwater, non-filtered (total) results will be used to establish BTVs. In general, groundwater data will not be included in the development of BTVs when turbidity of the groundwater sample was reported to be greater than 10 nephelometric turbidity units (NTU) or when pH is greater than 8.5. Professional judgment can be used to retain data that does not meet these criteria. However, the decision to retain data that does not satisfy these criteria must be documented; such as, concurrence with NCDEQ that naturally occurring pH is greater than 8.5 in the unit being evaluated. Background locations for groundwater were identified for each site in the CSA Reports and/or Corrective Action Plans (CAPs). Other wells unaffected by 1 The terms PPBTV and BTV are used interchangeably in this document. The term BTV is used in the EPA ProUCL User guide. REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 2 | Page plant operations may also be used to augment the background data set with agreement from NCDEQ. For soil, only samples collected above the water table and at locations not influenced by Plant operations will be included in the calculation of BTVs. Site-specific soil sampling locations and intervals are described in the CSA Work Plans. The methods for developing PPBTVs described in this document are based on the US Environmental Protection Agency (USEPA) “Unified Guidance” (USEPA 2009), USEPA’S Guidance for Comparing Background and Chemical Concentrations in Soil for CERCLA Sites (USEPA 2002), and the ProUCL Technical Guide (USEPA 2015). In addition, the North Carolina Division of Water Quality (NCDWQ) technical assistance document for Evaluating Metals in Groundwater at DWQ Permitted Facilities (NCDWQ 2012) was also referenced. The methods described in this document are intended to serve as guidelines to develop BTVs. The use of the upper tolerance limit (UTL) to establish BTVs for constituents analyzed during assessment monitoring is consistent with NCDEQ Guidance as well as the USEPA’s Unified Guidance (2009). The UTL will be evaluated as the statistic for development of groundwater and soil BTVs. BTVs will be developed for a select group of constituents derived from the list of parameters investigated as part of CAMA (Table 1). The UTL will be used to represent an upper limit for naturally occurring concentrations such that values exceeding this limit may be indicative of groundwater and soil impacts. Naturally occurring concentrations determined by the process presented in this document will be submitted to the NCDEQ Division of Water Resources for determination of the PPBTVs. Site-specific reports documenting the procedures, evaluations, and calculations will be prepared and submitted to NCDEQ. Following NCDEQ’s approval of the PPBTVs, the PPBTVs will be used as groundwater and soil standards when the values exceed concentrations appearing in T15A NCAC 02L .0202(g) or the Interim Maximum Allowable Concentrations (Appendix #1 to T 15A NCAC 02L) for groundwater or Preliminary Soil Remediation Goals as described in Section 4 of the NCDEQ 2015 Inactive Hazardous Sites Program Guidelines for Assessment and Cleanup (NCDEQ 2015) for soil. This document consists of four parts describing the process for establishing BTVs for constituents in groundwater and soil: Part I – Description of Background Data Sets Part I provides discussion of groundwater and soil sample collection, background data set attributes, and preliminary treatment of background data. Part II – Preliminary Data Analysis Part II includes analyses used to assess and transform data (where necessary) for use in producing appropriate UTLs. This analysis includes screening data sets for outliers, fitting data sets to distribution models, assessing data for temporal variability, and appropriateness of the period of record (sampling period). REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 3 | Page Part III – Testing for Sub-Groups in Background Groundwater Data Part III summarizes the approach for testing data sets for distinct sub-groups. If testing indicates presence of subgroups, the same steps described in Part I can be applied to the partitioned data to better understand the distribution of the samples within a sub-group for each constituent. Part IV – Development of BTVs for Constituents in Groundwater and Soil Part IV documents the steps for producing UTLs for each constituent for groundwater and soil. REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 4 | Page PART I – DESCRIPTION OF BACKGROUND DATA SETS Background data sets vary by size at each of the sites. Background groundwater data has been collected over a period of time at multiple locations per site. Background soil samples were primarily collected as part of the CSA activities. The following sections describe the groundwater and soil samples. Additional details regarding site-specific data sets have been provided in the CSA, CAP 1 and CAP 2, supplemental reports and electronic data submittals for each site. The data sets continue to be refined as additional data are available over time. Sample results with a detection or reporting limit greater than the applicable standard will not be included in the background data sets. Should the detection or reporting limit criteria impact the data set such that sufficient data is not available for producing BTVs for particular constituents, NCDEQ will be consulted to discuss alternative evaluation options for assessment of background, such as groundwater fate and transport modeling. Groundwater Groundwater samples are collected from monitoring wells screened in different flow layers using low-flow sampling techniques in accordance with the USEPA Region 1 Purging and Sampling Procedure for the Collection of Groundwater Samples from Monitoring Wells (revised January 19, 2010) and the Groundwater Monitoring Program, Low Flow Sampling Plan, Duke Energy Facilities, Ash Basin Groundwater Assessment Program, North Carolina, dated June 10, 2015. Groundwater samples have been analyzed for constituents listed in Table 1. Only non-filtered sample results will be utilized for producing BTVs. Sample data associated with turbidity reported to be greater than 10 NTUs, samples without a recorded turbidity, or samples with a pH greater than 8.5 will be excluded from the background data set. Where site conditions require, professional judgment can be used to retain data that does not meet these criteria (such as where the naturally occurring groundwater pH is greater than 8.5). However, the decision to retain data that does not satisfy these criteria must be documented. BTVs will be calculated for each constituent within a flow layer using data pooled from all background wells screened within that flow layer. Soil Discrete soil samples were collected from multiple depth intervals above and below the water table during the CSA or other assessment events. The total number of samples and depths collected vary by site. Soil samples have been analyzed for constituents listed in Table 1. Only constituent concentrations in unsaturated soils above the water table will be utilized for producing BTVs. Duke Energy and NCDEQ have agreed soil samples can be pooled from multiple depth intervals. A statistical evaluation of the soil data sets will be performed to confirm the approach is appropriate. REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 5 | Page PART II – PRELIMINARY DATA ANALYSIS Preliminary data analysis includes eight steps and is summarized in the following sections. 1. Descriptive Statistics Descriptive statistics are useful for characterizing data, increasing data set understanding, and for assessing information quality. For each site, descriptive statistics will be calculated for groundwater and soil data sets. For groundwater, descriptive statistics will be calculated for each constituent within each groundwater flow layer using pooled data from that groundwater flow layer. Soil descriptive statistics will be calculated for each constituent using the pooled background data set. The following statistics will be calculated to describe each data set.  Sample Size  Mean and median  Number of detects and non-detects  Maximum and minimum  Percentage of non-detects  Standard Deviation  Number of distinct observations  Skewness  Number of distinct method detection limits (MDL)  Kurtosis 2. Graphical Analysis Background groundwater data can be graphically portrayed using scatter plots, box-and- whisker and quantile-quantile (Q-Q) plots (Figures 1 and 2), while background soil data can be illustrated using box-and-whisker and Q-Q plots. The construction of scatter plots of concentration versus time (Figure 3) for each constituent within each background monitoring well or using the pooled data across all the background wells can assist in identifying potential trends or seasonality within data. Box-and-whisker and Q-Q plots can be constructed for each constituent within each groundwater flow layer using all data pooled from background wells monitoring that flow layer to identify possible outliers and sub-groups in addition to assessing data set distributions. Instructions for interpreting box-and-whisker plots can be found on Figure 1. Q-Q plots (Figure 2) evaluate if a theoretical distribution can accurately model a sampled distribution. If the sampled population is accurately modeled by the theoretical distribution, then quantiles from the sampled distribution should plot along a straight line when plotted against the quantiles of the theoretical distribution. Sampled values that plot markedly away from the straight line or REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 6 | Page jumps or breaks in the plot may indicate the presence of multiple sample populations, potential outliers, or non-normal sample distributions. The graphical analysis provides information regarding a steady-state baseline period. Multiple method detection limits over time will also be evaluated to determine if such variability affects the quality of the data. This process will be used to determine if all data can be incorporated into the analysis or if older historical data may need to be removed from the data set due to a change in the data reporting protocols for samples over time. 3. Identify Outliers Outliers are values that are not representative of the population from which they were sampled and whose presence can significantly alter statistical results. Data sets will initially be screened for potential outliers visually using box-and-whisker and Q-Q plots (Figures 1 and 2). Following the visual assessment of data for potential outliers, data sets will be screened for outliers quantitatively. While there are several tests available to test for possible outliers, Dixon’s or Rosner’s Outlier tests are specifically identified in the Unified Guidance (USEPA 2009) for providing requisite statistical strength and power necessary to appropriately identify potential outliers. Dixon’s Outlier Test is suitable for data sets containing less than 25 samples, whereas Rosner’s test is applicable for data sets containing greater than 25 samples. Both tests assume data are normally distributed. Extreme outliers are of interest; therefore, outlier tests will be conducted using a significance level of 0.01. Groundwater and soil constituent concentrations determined to be outliers will be provided in the statistics report submitted to NCDEQ. If statistical outliers have been detected, the project scientist will review the values to determine if they should be removed from the data set or are representative of background and should be retained for statistical analysis. Reasons as to why a particular statistical outlier should be included or excluded from either groundwater or soil background data sets will be documented as part of the final reference background concentration value documentation notes. 4. Identifying Data Distributions Many statistical tests, such as UTLs, make an explicit assumption concerning the distribution of sample data. Therefore, data must be fitted to a known distribution model (e.g., normal distribution). Upon completion of screening data sets for outliers, groundwater and soil data will be fitted to known distribution models using Goodness-of-Fit (GOF) tests. GOF tests assess how closely a data set resembles a given distribution model. The distribution models under consideration for the determination of groundwater and soil BTVs are normal, lognormal, and gamma distributions. In order to assess if data are normally or lognormally distributed, the Shapiro-Wilk or Lilliefors GOF test will be used. The Shapiro-Wilk GOF test is applicable for data sets comprised 50 or fewer samples, while the Lilliefors GOF test is appropriate for data sets containing more than 50 samples. To evaluate if data are gamma distributed, the Anderson-Darling or Kolmogorov- Smirnov GOF test will be utilized. GOF tests will be performed using a significance level of 0.05. REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 7 | Page The software package developed by the US EPA, ProUCL, has incorporated these methods to automatically test for either normal, lognormal, or gamma distribution types. If all GOF tests fail, non-parametric estimation methods will be used. The distribution of data will be evaluated as the data sets are established, with the understanding that distributions may change over time. 5. Evaluating Background Groundwater Data The following section applies to data sampled over time, such as groundwater data, and is not applicable to soil data. Constituent concentrations in groundwater sampled over time from multiple background well locations may exhibit patterns which suggest concentrations are increasing or decreasing over time. For background samples to be considered representative of areas unimpacted by human activity and be meaningful in the production of the BTVs, constituent concentrations over time should reflect a steady state, or ‘temporal stationarity’. In other words, a constituent’s population characteristics (mean and variance) do not fluctuate over time (with consideration of normal seasonal fluctuations). Another assumption regarding samples collected across multiple background wells at a site is a constituent’s mean and variance are constant across background wells, or ‘spatial stationarity’. If data collected from the background wells exhibit temporal or spatial non-stationarity, pooling of background well data can result in an inflated population variance and biased estimates of BTVs. A comparison of multiple box-and-whisker plots (Figure 1) can be used to visually assess whether background wells distributions have similar constituent concentration means and variances. Based on visual inspection of box-and-whisker plots, further analysis (such testing for differences in means or medians across background well locations) may be warranted to determine if a background well should be considered representative of background. Statistical tests for trends over time using the pooled data from the background wells should show no statistical significance. However, before proceeding to test for trends in the background samples, another assumption regarding constituent concentrations is the values must be independent from one another. When values are related to each other over varying time intervals, then values at any point in time can be expressed as a function of previous value(s). This type of relationship is termed autocorrelation. When values express a seasonal relationship, this type of autocorrelation is termed seasonality. The presence of autocorrelation, seasonality, or trends indicates data are temporally non- stationary. Assessment of background groundwater data should be performed to address temporal stationarity prior to pooling background data for the production of BTVs. Details for assessing data sets for temporal stationarity is summarized in the following sections. 6. Autocorrelation Autocorrelation occurs when measurements collected at different points in time correlate with one another. Sources of autocorrelation in groundwater data can be due to seasonality, trends, REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 8 | Page or samples being collected too close to one another in time. Data that exhibit autocorrelation can affect sample variance and can lead to biased estimates of BTVs. Constituent concentrations in groundwater at a given background well will be checked for autocorrelation using the sample autocorrelation function (USEPA 2009). The sample autocorrelation function graphs correlation values between successive measurements against the time lag between sampling events and assumes data can be fitted to a known distribution model (Figure 4). Correlation values can be between zero and one, where one indicates a perfect correlation (dependence) and zero represents no correlation (independence). The sample autocorrelation function will be calculated using a significance level of 0.05. Autocorrelated observations can be corrected by 1) reducing sampling frequency and increasing the time between sample collection; 2) altering the statistical test used to analyze the data; or, 3) removing temporal patterns using a technique such as deseasonalization. 7. Seasonality Constituents in groundwater at background well locations may experience predictable recurring increases and decreases in concentrations, termed seasonality (Figure 6). Seasonality within a data set can introduce bias into the calculation of BTVs and result in falsely identifying a seasonal effect as potential impacts. Data should be assessed for seasonality once an adequate number of background groundwater samples have been collected. Useful diagnostic tools for evaluating data sets for seasonality are autocorrelation and scatter plots (Figures 4 and 5). When sufficient observations are available, then a side-by-side comparison of multiple box-and-whisker plots constructed by season are informative. If constituent concentrations within a given background well appears to experience seasonal fluctuations, the seasonal component within the data can be removed for the purpose of testing for trends. If seasonality is not addressed prior to testing for trends, then the statistical tests for trends may be misleading (i.e., fail to detect a trend when one is actually present or may indicate a significant trend when in fact, no trend exits). Strong evidence for the cause of seasonality within a data set should exist prior to removing seasonal components from background data sets. 8. Trends Wells installed at background locations monitor natural groundwater quality unaffected by anthropogenic activities. Therefore, a key assumption regarding background is constituent concentrations in groundwater should demonstrate stationary conditions through time, free of any trends (Figure 6). Background data exhibiting trends (upward or downward) violate the assumption of temporal stationarity. Trending constituent concentrations in background wells may identify potential anthropogenic impacts (resulting in the well no longer being considered background), seasonality, or altering groundwater conditions. Furthermore, presence of trends in background data can lead to overestimation of variances which result in inflated BTVs. REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 9 | Page Prior to the calculation of BTVs, background well data will be evaluated for the presence of trends. Depending on the presence of non-detects (NDs) and seasonality, background data sets can be assessed for trends using one of three tests:  Mann-Kendall trend test  Seasonal Kendall Regression  Maximum likelihood estimation (MLE) regression The Mann-Kendall (MK) trend test is a nonparametric test method that can be used to identify linear trends within data sets that do not adhere to specific distribution models, do not exhibit seasonality, and contain NDs. The MK trend test can only be utilized to evaluate data sets containing only one MDL. Seasonal Kendall regression is similar to the MK test (data sets do not have to adhere to specific distribution models and can contain NDs as long as they are represented by a single MDL), except it accounts for seasonality. MLE Regression is a parametric method that estimates parameters of a statistical model and for fitting a statistical model to data. MLE Regression can be performed on data sets that can be fitted to a specific distribution model, do not demonstrate seasonality, and contain NDs. In cases where trending background constituent concentrations are identified, further analysis is recommended to rule out if the trend is more of an artefact related to the length of time available for the analysis and/or the small sample sizes. For example, if less than 10 samples have collected from a background well over a short duration (less than two years), then an observed trend in the well may not necessarily indicate changes in the natural variability of groundwater quality and may be representative of natural variation. If sufficient data is available for a constituent (> 20 observations), a statistical method called the piece-wise polynomial model can be used to inform the overall trend results. A description of this approach is as follows. 9. Additional Methods for Identifying Trends in Background Groundwater Data The piece-wise polynomial model is a useful tool for assessing constituent concentrations that have experienced multiple trends throughout monitoring. Piece-wise polynomial models attempt to find an appropriate mathematical function that expresses the relationship between the constituent concentrations and the sampling dates by using piece-wise regressions. Two types of piece-wise models can be used to evaluate trends, the linear-linear and linear-linear- linear regression models. The linear-linear regression model assumes and identifies one structural break in a time-series, in which the two portions of the data separated by the break point exhibit two different trends modeled by two different linear equations. Similarly, the linear-linear-linear regression model attempts to identify two structural breaks to assess three different linear trends. Piece-wise polynomial models can be informative, but it have the disadvantage of not being able to account for NDs in within data sets. Therefore, it is recommended implementing piece- REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 10 | Page wise polynomial models in conjunction with MLE regression. Piece-wise models can also serve as a visual guide when selecting the baseline sampling periods for statistical analysis. For example, in Figure 7 the MLE regression suggested that the overall trend in constituent concentrations over time are steadily increasing, whereas the polynomial piece-wise regression with two structural breaks indicates concentrations have experienced upward and downward trends. 10. Determining Baseline Period for Background Wells This step provides information to make a determination of whether the entire period of record from which the background samples were collected is representative of natural background conditions and represents a baseline against which downgradient constituent concentrations in groundwater can be tested. If trend analysis indicates that over time the observations are steadily increasing or decreasing, then review of the data will be performed to determine if a sub-segment of the data set better represents the background period. For values to be considered representative of background, they should demonstrate temporal stationarity. A minimum of eight samples will be collected from background wells screened in each groundwater flow layer prior to statistical analysis. Analytical results from eight or more sampling events will be used for the statistical determination of BTVs for constituents in groundwater. A minimum of eight additional sampling events should be completed before evaluating if new background data should be combined with previous background data to produce revised BTVs. REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 11 | Page PART III – TESTING FOR SUB-GROUPS IN BACKGROUND GROUNDWATER DATA The following sections summarize the methodology for identifying sub-groups in groundwater data resulting from spatial or temporal variability and will not be applicable for the assessment of soil data. Part III summarizes steps to validate if statistically significant differences in background concentrations exist across potential sub-groups. Sub-groups represent distinct populations with statistically significant differences in mean or median concentrations among potential groups within a data set. An example of possible sub-groups is a difference in concentrations among constituents detected in background wells monitoring bedrock groundwater that were installed in different rock types. Each rock type has its own chemical characteristics that can influence groundwater chemistry and result in differing concentrations for constituents across background wells. In order to test for differences across potential sub-groups, sufficient sample size of at least eight to 10 samples is recommended for each potential sub-group (USEPA 2009, 2015). Testing for potential sub-groups within background data will be completed in three steps:  Graphical analysis  Analytical test for comparing sub-group differences  Tests for distinguishing which sub-groups are different Statistical tests utilized to test for potential sub-groups will be performed using a significance level of 0.05. Graphical Analysis Graphical representation of data is an effective tool for depicting patterns and relationships within data. Background groundwater data can be assessed for sub-groups using box-and-whisker and Q- Q plots (Figures 1 and 2). Multiple box-and-whisker and Q-Q plots can be constructed for comparing constituent concentrations and variability across perceived sub-groups. Another useful visual test assessing potential sub-group differences is the Empirical Distribution Function (EDF). EDFs compute summary statistics, generate EDF plots (Figure 9), and compute hypothesis tests appropriate for comparing two or more groups for data containing NDs (provided less than 50 percent of the results are NDs). Figure 8 of an EDF plot demonstrates that the two sub-groups representing samples taken during two different seasons show similar distributions or no differences in constituent concentrations between the two seasons. REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 12 | Page Analytical Tests for Comparing Sub-Groups The following methods can be used to test for differences across sub-groups:  T-test and One-way Analysis of Variance (ANOVA)  Wilcoxon rank-sum and Kruskal-Wallis (KW) tests  Kaplan-Meier (KM) (log-rank) test All three types of tests can be used to test data sets containing NDs. The t-test and One-way ANOVA are parametric statistical analyses that test for differences in means among groups. T-tests are used to test for differences in means among two groups, whereas ANOVA is used to test for differences in means across three or more groups. Both tests assume data are normally distributed with normally (or lognormally) distributed residual values and the variances among groups being compared are roughly the same. The Wilcoxon rank-sum test and KW Test are nonparametic equivalents of the parametric t-test and One-way ANOVA. Both the Wilcoxon rank-sum and KW Test analyze the ranks of the data rather than the actual concentrations and test for difference among average ranks between groups. The Wilcoxon rank-sum test compares the average rank between two groups, while the KW test compares the average rank across three or more groups. The KM (log-rank) test is a nonparametric test that compares the survival distribution between two or more groups. The KM (log-rank) test is useful for data sets that cannot be fitted to a discernible distribution model and contain a large percentage of NDs concentrations. Testing for potential sub-groups within background groundwater data sets will be performed using a significance level of 0.05. Tests for Identifying Differences Among Sub-Groups If results from One-way ANOVA, KW, and KM (log-rank) tests indicate a statistically significant difference during comparison of three or more groups, additional tests need to be performed to compare all possible pairs of sub-group means or average ranks to determine which ones are different from one another. These tests are referred to as ‘post-hoc’ tests because they are performed after the fact. The Tukey-Kramer and Dunn’s test are post-hoc tests that should be used to compare possible pairs of sub-groups means or average ranks. The Tukey-Kramer test is parametric and should be used to evaluate One-Way ANOVA results, whereas Dunn’s test is nonparametric and should be utilized to assess KW and KM (log-rank) test results. Post- hoc analysis will be performed using a significance level of 0.05. REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 13 | Page PART IV – DEVELOPMENT OF BTVs FOR CONSTITUENTS IN GROUNDWATER AND SOIL The USEPA Unified Guidance (2009) recommends using UTLs to estimate BTVs for constituents evaluated during assessment monitoring as opposed to the use of other statistical intervals such as confidence limits (UCL) and upper prediction limits (UPL). UTLs represent fixed values that do not rely on future observations, unlike UPLs, and are constructed using background data as opposed to a fixed health-based standard, unlike UCLs (USEPA 2009). UTLs allow for a “suitably high enough level above current background to allow for reversal of the test hypothesis and are the preferred statistical interval (USEPA 2009). In nearly all cases, the UTL is computed because the concern is generally for exceedances greater than the value. The only parameter that may require both upper and lower tolerance limits is pH. Site-specific BTVs for select constituents from Table 1 in groundwater and soil will be produced using UTLs. Tolerance intervals test the null hypothesis that concentrations in downgradient wells or at impacted soil sampling locations are similar to that of background and are constructed using the mean, standard deviation, and tolerance factor. For the estimation of BTVs for constituents in groundwater and soil, a coverage of 95 percent (p) and a confidence interval of 95 percent (1-α) will be used. This means, there is a 95 percent probability that 95 percent of background sample concentrations will fall within this limit. The formulation of the UTL may vary slightly with the details of the test to be made and the characteristics of the data involved (see chapters 3 and 5 of ProUCL’s Version 5.1.02 Technical Guide for the full specifications of the UTL formula under differing parametric and non-parametric assumptions), but the basic form for the (1-α)*100 percent UTL with coverage coefficient, p, under normal distribution assumptions in general is: ܷܶܮ ൌ ݔ̅൅ܭ∗ݏ Where ݔ̅ = baseline (historical data) sample mean; and, s = baseline (historical data) standard deviation. K represents a special function called the tolerance factor. It depends on the sample size (n), the confidence coefficient (1 – α), and the coverage proportion (p). For selected values of n, p, and (1-α), values of the tolerance factor (K) have been tabulated extensively in the statistical literature. ProUCL will be utilized to produce UTLs for each constituent. The type of UTL produced is a factor of distribution type, the desired confidence interval, coverage, and the percentage of NDs. Following completion of the preliminary data analysis described in Part II and applicable steps in Part III, the steps below will be completed for selection of appropriate UTLs. 1. UTLs will be produced for constituents in groundwater and soil using the statistical software program ProUCL. The first step in constructing UTLs using ProUCL is to REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 14 | Page categorize constituents based on the presence or absence of NDs. ProUCL calculates UTLs differently depending on whether NDs are present within a data set. The algorithms in ProUCL use imputation and modeling techniques to address NDs. ProUCL does not substitute values (e.g., multiplying the MDL by a constant) for NDs, as this method introduces bias into the estimation of UTLs. Some constituent data sets may be represented 50 percent or more NDs. A large percentage of NDs make it difficult to fit data to distribution models. For data sets containing 50 percent or more NDs, UTLs will be constructed utilizing nonparametric techniques. 2. Produce UTLs using a coverage (p) and confidence level (1- α) of 95 percent. 3. Record all UTLs under all parametric and non-parametric distribution models. If data sets used for producing UTLs can be fitted to multiple distribution models, preference is given to one distribution type over another. In cases where data fit normal, log normal, and gamma distribution models, preference should be given to UTLs calculated assuming data were gamma distributed since this distribution has the advantage of not having extremely long tails to the right. This mitigates the chance that UTLs will be extremely high relative to the median of the distribution as could be the case if one used either the lognormal or normal distribution assumptions for producing UTLs. If data are normally and lognormally distributed, but not gamma distributed, preference should be given to UTLs produced assuming data are lognormally distributed provided the logged data have a standard deviation less than or equal to one. If the standard deviation of the logged data is greater than one, then the gamma distribution is preferred. Otherwise, preference should be given to UTLs produced assuming normally distributed data. If data cannot be fitted to a discernible distribution, UTLs are produced using nonparametric techniques. Data set distributions will continue to be evaluated as additional samples are collected, with the understanding that distributions may change over time. 4. It has been demonstrated that if there are insufficient samples sizes, the non-parametric UTL cannot achieve the desired confidence coefficient of 95 percent. Depending on background sample size, a different order statistic is selected to produce UTLs. For constituent data sets containing eight to 11samples, UTLs will be produced using a coverage of 85 percent (i.e., the 85th percentile) and a confidence coefficient of 95 percent. For data sets containing 29 or more samples but less than 59 samples, UTLs will be calculated using coverage of 90 percent and confidence coefficient of 95 percent, and data sets containing more than 59 samples will be calculated using a coverage of 95 percent and a confidence coefficient of 95 percent. 5. A minimum of eight valid background groundwater samples should be obtained prior to producing BTVs for each constituent in each flow layer. If it is deemed necessary to produce BTVs prior to obtaining eight valid samples, NCDEQ will be consulted and the maximum observation may need to be used as a BTV. In addition, a minimum of eight additional samples should be obtained prior to evaluating if new background data should be combined with previous data to produce revised BTVs. REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 15 | Page REFERENCES NCDEQ DWR, 2012. Evaluating Metals in Groundwater at DWQ Permitted Facilities: A Technical Assistance Document for DWQ Staff. http://digital.ncdcr.gov/cdm/ref/collection/p16062coll9/id/251181. USEPA, 1992. Supplemental Guidance to RAGS: Calculating the Concentration Term. Publication 9285.7-081. USEPA, 2002. Guidance for Comparing Background and Chemical Concentrations in Soil for CERCLA Sites. EPA 540-R-01-003. USEPA, 2009. Statistical Analysis of Groundwater Monitoring Data at RCRA Facilities – Unified Guidance, March 2009. EPA 530-R-09-007. USEPA, 2015. Statistical Software ProUCL 5.1.002 for Environmental Applications for Data Sets with and without Nondetect Observations. EPA/600/R07/041. REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 FIGURES FIGURE 1 BOX-AND-WHISKER PLOT 75th Percentile 50th Percentile 25th Percentile 90th Percentile 10th Percentile Possible Outliers FIGURE 2 QUANTILE-QUANTILE (Q-Q) PLOT FIGURE 3 SCATTER PLOTS COMPARING TIME VERSUS CONCENTRATION BETWEEN TWO WELLS 0 50 100 150 200 250 300 350 400 450 1/1/2010 8/24/2011 4/15/2013 12/6/2014 CO N C E N T R A T I O N DATE MW-1 MW-2 FIGURE 4 SAMPLE AUTOCORRELATION OUTPUT FIGURE 5 SCATTER PLOT OF TIME VERSUS CONCENTRATION ILLUSTRATING SEASONALITY 0 10 20 30 40 50 60 3/3/2010 5/11/2012 7/20/2014 9/27/2016 PA R A M E T E R ( C O N C E N T R A T I O N ) DATE FIGURE 6 SCATTER PLOTS OF TIME VERSUS CONCENTRATION ILLUSTRATING TRENDS 0 20 40 60 80 100 120 3/3/2010 5/11/2012 7/20/2014 9/27/2016 PA R A M E T E R (C O N C E N T R A T I O N ) DATE 0 20 40 60 80 100 120 3/3/2010 5/11/2012 7/20/2014 9/27/2016 PA R A M E T E R (C O N C E N T R A T I O N ) DATE FIGURE 7 PIECE-WISE POLYNOMIAL REGRESSION OUTPUT EXHIBITING MULTIPLE TRENDS Upward Trends Downward Trend FIGURE 8 EMPERICAL DISTRIBUTION PLOT COMPARING CONSTITUENT CONCENTRATIONS BETWEEN TWO SEASONS REVISED DRAFT Duke Energy Carolinas, LLC Statistical Methods For Developing Reference Background Concentrations For Groundwater and Soil At Coal Ash Facilities January 2017 TABLE TABLE 1 CHEMICAL PARAMETERS ANALYZED IN GROUNDWATER AND SOIL FIELD PARAMETERS RADIONUCLIDES pH*†Radium 226* Specific Conductance*Radium 228* Temperature*Uranium (233, 234, 236, 238)* Dissolved Oxygen*ANIONS/CATIONS/OTHER Oxidation Reduction Potential*Alkalinity (as CaCO3)* Eh*Bicarbonate* Turbidity*Calcium INORGANICS Carbonate* Aluminum Chloride Antimony Magnesium Arsenic Nitrate (as N)† Barium Nitrate + Nitrite* Beryllium Potassium Boron Percent Moisture† Cadmium Methane* Chromium Sodium Cobalt Sulfate Copper Sulfide* Iron Total Dissolved Solids* Lead Total Organic Carbon Manganese Total Suspended Solids* Mercury Molybdenum Nickel Selenium Strontium Thallium (low level) Vanadium (low level) Zinc NOTES: * = Indicates parameter analyzed in groundwater only. † = Indicates parameters analyzed in soil only. Metals in groundwater were analyzed for total and dissolved concentrations. Soil pH measured at 25 degrees C. Analysis of groundwater and soil samples for Chromium (VI) began after initial samples were collected as part of CSA. Page 1 of 1