A protocol for developing and evaluating neural network-based surrogate models and its application to building energy prediction

Because of their low computational costs, surrogate models (SMs), also known as meta-models, have attracted attention as simplified approximations of detailed simulations. Besides conventional statistical approaches, machine-learning techniques, such as neural networks (NNs), have been used to develop surrogate models. However, surrogate models based on NNs are currently not developed in a consistent manner. The development process of the models is not adequately described in most studies. There may be some doubt regarding the abilities of such models due to a lack of documented validation. In order to address these issues, this paper presents a protocol for the systematic development of NN-based surrogate models and how the procedure should be reported and justified. The protocol covers the model development procedure sample generation, data processing, SM training and validation, how to report the implementation, and how to justify the modeling choices. The protocol is used to critically review the quality of NN-based SMs in the prediction of building energy consumption. Sixty-eight papers are reviewed, and details of the developed surrogate models are summarized. The reported developing procedures were evaluated using the criteria proposed in the protocol. The results show that the selection of the number of neurons is the best-implemented step with a justification, followed by the determination of model architecture, mostly justified in a discussion way. While greater focus should be given to sample dataset generation, especially input variables selection, considering independence check and clear report of model validation on training and test data. Also, data preprocessing is strongly recommended.