Identifying and Detecting Inaccurate Stack Traces in Bug Reports

Bug reports contain a combination of structured and unstructured information, with the stack trace being one of the most important structured pieces of information. However, stack traces that do not contain accurate fault location information can lead to significant delays in software maintenance. This work examines eight projects from two open-source software communities to understand how to identify and detect inaccurate stack traces in bug reports. Using our approach to detecting inaccurate stack traces, we found that accurate fault location information is found roughly 33% of the time in the top frame, 50% in the Top-3 frames, 75% in the Top-5 frames and that the information accuracy does not significantly improve beyond this point. Next, we found that roughly 33% of stack traces can be considered inaccurate using the Top-5 stack trace frames. Third, NPEs commonly occur in inaccurate stack traces, and project-specific exceptions occur too infrequently to likely be useful in identifying inaccurate stack traces. Finally, the best machine learning classifier to identify inaccurate stack traces in a project's bug reports uses a linear regression algorithm trained using the filename, method name and exception name from the top frame of a stack trace. Depending on the project, this classifier has a 78-98% accuracy, 60-94% precision, and 16-97% recall.