Using docking/replication to verify and validate computational models

被引:0
|
作者
Olaru, D. [1 ]
Purchase, S. [1 ]
Denize, S.
机构
[1] UWA Business Sch, Perth, WA, Australia
关键词
docking; replication; validation; simulation; SIMULATION;
D O I
暂无
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
Conventional practice in modelling requires checking that a model is correct with respect to its conceptualisation (verification) and that it corresponds to the real world phenomenon modelled (validation). Verification and validation assure the external and operational validity of a model (its quality). In settings where data for estimation is not readily available, the behaviour of the computational model and its results are questionable. An alternative approach that has been recently gaining attention is docking or replication, which is a process where one model is tested against another to see if they produce the same results. This paper reports on the docking experience and validation stages performed when replicating a fuzzy logic (FL) model's findings with an agent-based model (ABM) in the context of innovation in business networks. Using two modelling paradigms and software programs, we modelled in an 18 month-interval a network of three agent categories, which collaborate on adopting and advancing new ideas and technologies. The network links describe relations between agents, which drive processes of innovation. The autonomous agents are organisations of different sizes, characteristics, and roles and they interact/share resources/collaborate for the purpose of adoption and diffusion of innovation that fits with the organisation's goals. Depending on their resources, there is scope for innovation or otherwise. In addition, the environment can foster or hinder the innovation processes. The verification and validation of these two models involved several stages: 1) Expert judgement - the structure of the conceptual model is supported by literature and discussions with colleagues in various forums; 2) Checking the correspondence between what is emerging from the model and what is expected to be seen in the real world (passing the believability test); it is desirable for the model components to adequately represent a real equivalent behavioural effect but as real data was not available at the time of designing the models, the alignment of the model results to expectations acts as an external validation of the model; 3) Internal validity - assessing consistency by changing input data distributions and analysing extreme conditions. 4) Docking (also known as alignment or replication with contrasting alternative theories) - comparing the results of the two different modelling approaches. The models ensured the distributional equivalence, but they were not identical. As both models used the same parameters, we believe that the differences in results arose only from relaxing the restrictive assumptions in the FL or ABM models. The ABM results matched the FL conditions tested. The stochastic ABM generated a distribution of outcomes caused by random encounters among agents, while FL generated an ensemble of crisp values as result of multiple rules of interaction applying simultaneously. The replication experience has been a positive one. Although this does not justify the models' acceptance, the docking results encourage us to pursue collecting data to validate empirically both models in the near future. We conclude with some thoughts from Kleindorfer et al. (1998) in relation to various positions in the philosophy of science with respect to validation: in the simulation literature there is a continuum of opinions ranging from extreme objectivist (model validation can be separated from model builder and its context) to relativist ("model and model builder are inseparable" and "validity is a matter of opinion" - Kleindorfer et al., 1998: 1097). Their debate leads to a perspective that simulation modelling should not follow a prescriptive set of approaches to validation, but rather modellers should "responsibly and professionally argue for the warrant of the model".
引用
收藏
页码:4432 / 4438
页数:7
相关论文
共 50 条
  • [1] Tips to validate, verify software
    不详
    CONTROL ENGINEERING, 2006, 53 (05) : 30 - 30
  • [2] Verify: a toolbox for deterministic verification of computational models
    Palumbo, Giuseppe Alessandro Parasiliti
    Russo, Giulia
    Sgroi, Giuseppe
    Viceconti, Marco
    Pennisi, Marzio
    Pappalardo, Francesco
    Curreli, Cristina
    2020 IEEE INTERNATIONAL CONFERENCE ON BIOINFORMATICS AND BIOMEDICINE, 2020, : 1262 - 1267
  • [3] Validate and Verify: Keys to Scientific Excellence
    Chaguturu, Rathnam
    Lushington, Gerald H.
    COMBINATORIAL CHEMISTRY & HIGH THROUGHPUT SCREENING, 2016, 19 (03) : 178 - 179
  • [4] Theoretical and computational studies of docking dynamics using desolvation models
    Cheung, MS
    Finke, J
    Onuchic, JN
    BIOPHYSICAL JOURNAL, 2002, 82 (01) : 328A - 328A
  • [5] On experiments to validate computational models to describe liquid structure impact
    Göller, B
    STRUCTURES UNDER SHOCK AND IMPACT VI, 2000, 8 : 341 - 350
  • [6] Using ontology to validate conceptual models
    Shanks, G
    Tansley, E
    Weber, R
    COMMUNICATIONS OF THE ACM, 2003, 46 (10) : 85 - 89
  • [7] Computational Docking to Sweet Taste Receptor Models
    Walters, D. Eric
    SWEETNESS AND SWEETENERS: BIOLOGY, CHEMISTRY, AND PSYCHOPHYSICS, 2008, 979 : 162 - 167
  • [8] HOW TO DESIGN, VERIFY AND VALIDATE EMERGENCY SHUTDOWN SYSTEMS
    GREEN, DL
    DOWELL, AM
    ISA TRANSACTIONS, 1995, 34 (03) : 261 - 272
  • [9] On Verify and Validate a Next Generation Automotive Communication Network
    Brunthaler, Sebastian
    Waas, Thomas
    Kucera, Markus
    PECCS: PROCEEDINGS OF THE 9TH INTERNATIONAL CONFERENCE ON PERVASIVE AND EMBEDDED COMPUTING AND COMMUNICATION SYSTEMS, 2019, : 121 - 127
  • [10] Understanding Replication: Trust But Verify
    Suter, W. Newton
    Suter, Paula M.
    HOME HEALTH CARE MANAGEMENT AND PRACTICE, 2019, 31 (04): : 207 - 212