A Quasi-Experiment to Expose Attention-Decision-Learning Cycles in Engineering Project Teams

Engineering project outcomes are driven by a dynamic mix of the social physics of teams, the unique complexities of the engineering challenge at hand, and stakeholder pressures in context. Various related research has demonstrated formal experiments for tightly controlled problems in small teams, including work in organizational psychology, computational organization theory, design thinking, and coordination science. We realize there is room for testing these foundational concepts in quasi-controlled environments with distributed teams challenged by problem, solution, and organization complexity common today. This paper presents a quasi-experiment to study how engineers proceed through attention, decision, and learning cycles in the design of a System of Systems. The experiment utilized an ensemble of an agent-based model, a decision-support interface, and a variety of sensors to record behavior and activity. Four pilots for a maritime industry challenge were conducted with experienced industry experts, followed by a primary experiment for data collection. Though this work is preliminary, the experimental approach detects (for this case) how designers focused on different variables (attention), manipulated variables to accomplish desired outcomes (decisions), and explored the system performance trade space variously over time to reveal false assumptions and uncover better decisions (learning). Lessons learned from this quasi-experiment are guiding this research team to prepare scalable and reproducible engineering teamwork experiments that include sensors of events over time in the problem, solution, and socials spaces of engineering projects.