Identifying “sloppy” users in TMS through operation logs

A transportation management system (TMS) is an integral software system for modern logistics and transportation companies. It is crucial to evaluate the quality of a TMS objectively, a task that currently presents significant challenges to both the IT and logistics sectors. One approach to this evaluation is usage analysis. However, usage analysis is complicated by the presence of both diligent users who utilize the system correctly and 'sloppy' users who enter inaccurate data haphazardly. This inaccuracy hampers the success of the information system and obstructs effective decision-making. Thus, identifying and excluding data from such users is essential for an accurate and objective evaluation of a TMS. Yet, the focus has primarily been on identifying outliers, typically for security reasons, while the identification of 'sloppy' users has been overlooked. Against this context we propose a novel method—Log Evaluation through Operation Sequence Distribution (LEOSD). This method distinguishes between abnormal and normal usage of a TMS by analysing system-generated logs. LEOSD is highly efficient and lightweight, minimizing any disruption to ongoing operations. Our experiment, based on real logs gathered from the industry, supports our hypothesis, and shows that LEOSD is effective in identifying 'sloppy' users. The positive results attest to the efficacy and practicality of our proposed method. © The Author(s) 2023.