Grey Box Model

In mathematics, statistics, and computational modeling, a «grey box» model combines a partial theoretical structure with data to complete the model. The theoretical structure can range from information about the smoothness of results to models that only need parameter values from data or existing literature. [ 5 ] Thus, almost all models are «grey box» models, as opposed to «black box» models, where no form of model is assumed, or «white box» models, which are purely theoretical. Some models assume a specific form, such as linear regression or a neural network. Special analysis methods exist for these. In particular, linear regression methods [ 10 ] are far more efficient than most nonlinear methods. A model can be deterministic or stochastic (i.e., containing random components) depending on its intended use.

The general case is a nonlinear model with a partial theoretical structure and some unknown parts derived from data. Models with different theoretical structures need to be evaluated individually, possibly using simulated annealing or genetic algorithms.

Within a specific model structure, it may be necessary to find parameters or relationships of variable parameters. For a specific structure, it is arbitrarily assumed that the data consists of sets of feed vectors f, product vectors p, and operating condition vectors c. Typically, c will contain values extracted from f, as well as other values. In many cases, the model can be converted into a function of the form:

m(f,p,q)

where the vector function m shows the errors between the data p and the model predictions. The vector q contains some variable parameters that are the unknown parts of the model.

The parameters q vary depending on the operating conditions c in a way that has yet to be determined. This relationship can be defined as q = Ac, where A is a matrix of unknown coefficients, and c, as in linear regression, includes a constant term and possibly transformed values of the original operating conditions to obtain nonlinear relationships between the original operating conditions and q. It then remains only to choose which terms in A are nonzero and assign them values. Completing the model becomes an optimization task to determine the nonzero values in A that minimize the error terms m(f,p,Ac) across the data.

After selecting the nonzero values, the remaining coefficients in A can be determined by minimizing m ( f , p , Ac ) over the data with respect to the nonzero values in A, usually using the nonlinear least squares method. The selection of nonzero terms can be carried out using optimization methods such as simulated annealing and evolutionary algorithms. In addition, the nonlinear least squares method can provide accuracy estimates for the elements of A, which can be used to determine whether they differ significantly from zero, thus providing a method for term selection

Sometimes it is possible to calculate the values of q for each data set directly or using the nonlinear least squares method. A more efficient linear regression can then be used to predict q using c, thereby selecting the nonzero values in A and estimating their values. Once the nonzero values are found, the nonlinear least squares method can be used on the original model m(f,p,Ac) to refine these values.

The third method is model inversion, which converts the nonlinear m ( f , p , Ac ) into an approximate linear form with respect to the elements of A, which can be examined using efficient term selection and linear regression estimation. For the simple case of a single value q ( q = aTc ) and an estimate q* for q. Substituting dq = aTc − q *, we obtain

m(f,p,a T c) = m(f,p,q* + d q) ≈ ​​m(f,pq*) + d q m'(f,p,q*) = m(f,pq*) + (a T c − q*) m'(f,p,q*)

so that T is now in a linear position, with all other terms known, and can therefore be analyzed using linear regression methods. For more than one parameter, the method extends in a straightforward manner. After verifying that the model has been improved, this process can be repeated until convergence. This approach has the advantage that it does not need to determine the parameters q from a separate data set, and linear regression is performed on the original error terms

When there is enough data, it is recommended to split it into a separate data set for building the model and one or two data sets for evaluation. This procedure can be repeated using several samples from the model-building data set, and the resulting models can be averaged or used to assess differences in predictions.

A statistical test, such as chi-square, for the residuals is not particularly useful. The chi-square test requires known standard deviations, which are rarely available, and failed tests provide no indication of how to improve the model. There are a number of methods for comparing both nested and non-nested models. These include comparing the model's predictions with repeated data.

Attempting to predict the residuals m(, ) under operating conditions c using linear regression will show whether the residuals can be predicted. Residuals that cannot be predicted offer little chance of improving the model using the current operating conditions. Terms that predict residuals are promising terms to include in the model to improve its performance.

The model inversion method described above can be used to determine whether the model can be improved. In this case, the selection of nonzero terms is less important, and linear prediction can be performed using the significant eigenvectors of the regression matrix. The values in A determined in this way must be substituted into the nonlinear model to assess the improvement in model errors. The absence of significant improvement indicates that the available data does not allow the current form of the model to be improved using the specified parameters. [ 5 ] For a more thorough test, additional parameters can be added to the model.