Pre-calibration Monte Carlo: Settings
FePEST provides an option for doing a “pre-calibration” Monte Carlo analysis directly from the Problem Settings dialog. The “pre-calibration” entails the use of Monte Carlo method before calibrating of the FEFLOW model. This is particularly useful to have a first idea of the sensitivities of the model output to certain parameter variation based on the prior knowledge of the modeller (e.g. range of conductivities for a specific unit, etc.). The results of the pre-calibration Monte Carlo analysis can then be used in the calibration process, for example, with a primary focus on certain model parameters.
Optimization Control: Activating Pre-calibration Monte Carlo mode.
Criteria for randomly generated parameters
The Monte Carlo Analysis enables the user to randomly generate set of parameters (zonally-constant or pilot points) based on a (user-defined) criteria:
Number of samples: maximum number of parameter realizations for Monte Carlo analysis.
Distribution type: this can either be a normal distribution or a uniform distribution. The defined distribution type is used for the random generation of the parameter. If the parameter definition uses log-transform, the parameter generation also result in a log-transformed values.
Mean method: the random set of parameter is centred on an user-defined mean value, which could either be the current initial value of the parameter or computed as the midpoint of the bounds of the parameter. This information is provided in the section Parameter Definition of the Problem Settings dialog.
Respect ranges: it is possible for some set of parameters, generated by a fully-random process, to fall outside of the expected parameter ranges (for example, defined for the calibration). The user has the option to set the ranges of the parameter (including tied parameters) accordingly.
Uncertainties: the uncertainty associated to the distribution of the parameters (i.e. how much a parameter can vary according to modeller’s knowledge) can be prescribed by three options:
Bounds: The standard deviation for the parameter generation is computed from the parameter bounds automatically. This is calculated as the difference between the log-transformation of the parameter bounds divided by 4. The rough estimation is based on the assumption that the parameters are log-normally distributed, and the intervals between respective parameter bounds correspond roughly to 95% pre-calibration parameter confidence intervals. The differences between these bounds then correspond to approximately 4 parameter standard deviation. Alternatively, the standard deviation can be provided by the user in the input field.
Covariance matrix: the same covariance matrix used for the Tikhonov regularization is used as a measure of uncertainty to compute the random parameters.
File: Experienced users can provide an “uncertainty file (*.unc)” for the standard deviation or covariance matrix of a parameter. The uncertainty files conform to certain PEST definitions. It can also provide measures of uncertainty either for a group of parameters or to specific parameters.
Seed number: By default, the parameters generated using the Monte Carlo Analysis in FePEST are fully random. This implies that the same parameters cannot be obtained between different FePEST problems even when the same distribution type, mean value and uncertainty source is considered. However, identical parameters can be obtained by providing the same seed number across all FePEST problems.
The Monte Carlo option in FePEST (either pre-calibration or post-calibration) automatically generates parameter values to all available and active parameter definitions in the project, i.e. defined in PEST Control file (*.pst).
Optimization of random parameter sets
After the parameters are randomly generated, the user may decide to run a separate PEST problem for each parameter set by activating the option Optimize. In doing so, FePEST automatically creates a new PEST Control file for each set of parameter generated. The PEST run is fully parallelized using the BeoPEST utility, as described in the Parallelization section of the Problem Settings dialog.
Maximum number of iterations and threshold value
A PEST problem is run separately for each parameter set that is randomly generated using the Monte Carlo analysis. By default, the number of PEST iteration is set to 30 cycles. The PEST optimization will terminate if the maximum number of PEST iterations is reached and/or the threshold objective function (Phi parameter) is achieved.
A PEST problem is run separately for each parameter set. No optimization is performed. The aim is to estimate the current objective function for each of the parameter sets without any adjustments. A FEFLOW model is run once per each Monte Carlo realization.
Jacobian and statistics
PEST is run in a special estimation mode to compute the Jacobian matrix and statistics for each parameter set.
PEST is run in a special estimation mode to compute the Jacobian matrix for each parameter set. Note that each computation of the Jacobian matrix requires a FEFLOW model run.
Monte Carlo Analysis: Settings and options.
Running FePEST under Pre-calibration Monte Carlo mode
To run FePEST with the pre-calibration Monte Carlo mode, the same steps are required as for other operation modes (e.g. estimation or prediction). The run dialog is prompted either by clicking the Run button or from the Estimation menu. The dialog prompts the user to create the PEST files for the Monte Carlo analysis and/or to run the problem.
Running pre-calibration Monte Carlo analysis in FePEST.
If the option of creating files is enabled, standard PEST files (such as the PLPROC files), regularization files, and several other files (as described below) are created under the pre-calibration Monte Carlo mode:
Parameter file (*.par): By default the name of this file is composed by mc_rand followed by a number, ranging from 1 to the total number of samples for the Monte Carlo analysis. The file contains four columns, namely the parameter name, value, scale and offset. The values are in fact the randomly generated values (using the normal or uniform distribution) for each parameter. The scale and offset are typically set to 1 and 0 respectively.
Monte Carlo PEST file (mc.pst): This is a PEST control file, which is the same as the project control file (case.pst) except that mc.pst is contains the values of the parameters from each realization. The Optimize option in FePEST expects a PEST control file for each parameter realization. Instead of creating separate files, FePEST updates the files after each PEST run.
Parameter summary table file (*.dat): A file under the name mc_rand_mulpar is created after executing the PEST utility MULPARTAB. This utility reads all the parameter files contained in the FePEST working directory and creates a file containing the information from all the parameter files.
Results of the Pre-calibration Monte Carlo Analysis
For generating random parameters, FePEST uses the PEST utility RANDPAR. After Pre-calibration Monte Carlo Analysis is successfully finished, two additional dialogs appear in FePEST, namely Parameter Uncertainty and Observation Uncertainty. The first one is always visible whereas the second chart appears only if the user activates the option Optimize in Monte Carlo Analysis settings page. The charts plot a histogram for the parameters and observations, respectively.
An example of histogram plot for Parameter Uncertainty is shown in below. Different parameters can be visualized by changing the parameter name directly in the combo box available. The Observation Uncertainty chart follow the same pattern.
The Observation Uncertainty chart enables the user to see the histograms for each observation defined in the FePEST project (e.g. process variables, rate and period budgets, IFM-implemented observations, etc.).
Moreover, the results of the uncertainty plots can be exported from the Properties option from the chart context menu in case of further post-processing.
Monte Carlo Analysis: Parameter uncertainty chart.
All the different model scenarios created by the Pre-calibration Monte Carlo Analysis are available from the menu Results - Show Results. Each scenario can be saved as a different FEFLOW FEM file by selecting the relevant column and followed by clicking the Save button.
Pre-calibration Monte Carlo results.