Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

GLIB: Towards Automated Test Oracle for Graphically-Rich Applications

Version 1 : Received: 15 July 2021 / Approved: 16 July 2021 / Online: 16 July 2021 (16:17:02 CEST)

How to cite: Chen, K.; Li, Y.; Chen, Y.; Fan, C.; Hu, Z.; Yang, W. GLIB: Towards Automated Test Oracle for Graphically-Rich Applications. Preprints 2021, 2021070386 (doi: 10.20944/preprints202107.0386.v1). Chen, K.; Li, Y.; Chen, Y.; Fan, C.; Hu, Z.; Yang, W. GLIB: Towards Automated Test Oracle for Graphically-Rich Applications. Preprints 2021, 2021070386 (doi: 10.20944/preprints202107.0386.v1).

Abstract

Graphically-rich applications such as games are ubiquitous with attractive visual effects of Graphical User Interface (GUI) that offers a bridge between software applications and end-users. However, various types of graphical glitches may arise from such GUI complexity and have become one of the main component of software compatibility issues. Our study on bug reports from game development teams in NetEase Inc. indicates that graphical glitches frequently occur during the GUI rendering and severely degrade the quality of graphically-rich applications such as video games. Existing automated testing techniques for such applications focus mainly on generating various GUI test sequences and check whether the test sequences can cause crashes. These techniques require constant human attention to captures non-crashing bugs such as bugs causing graphical glitches. In this paper, we present the first step in automating the test oracle for detecting non-crashing bugs in graphically-rich applications. Specifically, we propose GLIB based on a code-based data augmentation technique to detect game GUI glitches. We perform an evaluation of GLIB on 20 real-world game apps (with bug reports available) and the result shows that GLIB can achieve 100\% precision and 99.5\% recall in detecting non-crashing bugs such as game GUI glitches. Practical application of GLIB on another 14 real-world games (without bug reports) further demonstrates that GLIB can effectively uncover GUI glitches, with 48 of 53 bugs reported by GLIB having been confirmed and fixed so far.

Keywords

Automated Test Oracle; Game Testing; GUI Testing; Deep Learning

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our diversity statement.

Leave a public comment
Send a private comment to the author(s)
Views 0
Downloads 0
Comments 0
Metrics 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.