Game Test Automation Library Comparison

Did you know that test automation is already popular in mobile apps and web browsers? There are test automation projects called Selenium and Appium, which are used by many people and are considered standard. So what about games? There is no test automation project that can be considered a standard yet. There are different projects. I would like to compare them.

Glossary of Terms

• Engine: Refers to game engines such as Unity and Unreal.
• Script language: Almost all game test automation libraries are structured in a way that allows the game to be controlled through a separate script language outside of the game. This refers to a programming language that can be used for control. Most of them support Python.
• Inspector: In order to control the game from outside the game, you need to be able to find the object you want to control (GameObject in Unity). At this time, each library provides a path or name as a way to find the target. This is the Editor that helps you figure this out.
• Device Farm: Have you heard of AWS DeviceFarm, BrowserStack, or LambdaTest? This is a service that allows you to access and control mobile devices from a web browser, while also remotely executing app tests such as Appium. Likewise, there are services that support game test automation to run remote tests.

Gamium

• Engine: Unity engine is supported and Unreal and Godot engines will be supported in the future.
• Script language: Python and TypeScript are supported as script languages.
• Inspector: Can be used inside Unity Editor and in conjunction with Dogu device farm.
• Manufacturer: This is an open source test automation tool created by Dogu. Github Link
• Device farm: Can be linked to Dogu device farm. 😊
• Function: UI manipulation, KeyInput, and remote code call are possible.It was created to be easy to use, similar to the interface used by Selenium and Appium Clients.

Poco, Airtest

• Engine: Supports Unity, Unreal, and Cocos2d engines.
• Scripting Language: Supports Python as a scripting language.
• Inspector: Available through Airtest IDE and PocoHierarchyViewer.
• Developer: An open-source test automation tool developed by NetEase. GitHub Link
• Device Farm: Compatible with the cloud device farm called AirLab.
• Note: It's worth mentioning that when combining Airtest, an automation tool, with image recognition, automation can be achieved. However, image recognition has a short lifespan and can be challenging to maintain in existing scripts, so it is not recommended.

GAutomator

• Engine: Supports Unity and Unreal engines.
• Scripting Language: Supports Python as a scripting language.
• Developer: An open-source test automation tool developed by Tencent. GitHub Link
• Inspector: Available through GAutomatorView.
• Device Farm: Compatible with the WeTest UDT device farm.
• Note: While documentation is available, most of it is written in Chinese and there are broken links.

AltTester for Unity

• Engine: Exclusive to the Unity engine.
• Scripting Language: Supports C#, Python, and Java as scripting languages.
• Developer: An open-source test automation tool developed by AltTester. GitLab Link
• Inspector: Supports external editors for Unity through the AltTester Desktop App.
• Device Farm: There is no proprietary device farm, but it provides guides for integration with services like BrowserStack and AWS DeviceFarm.

GameDriver

• Engine: Supports Unity, Unreal, and Godot engines.
• Scripting Language: Supports C# as a scripting language.
• Developer: Developed by the company GameDriver.
• SDK related to this tool needs to be requested for access through the official website or Unity AssetStore.

Below is a summarized table of the information.

• GameDriver was excluded from the table due to the difficulty in accessing information, which would make it challenging to provide clear details.

Gamium is a relatively new project. However, it has well-documented examples. Additionally, if you develop automation scripts, there are cases where you can easily handle Unity's internal C# code by calling it. For this purpose, an RPC feature has been developed.

When comparing Poco and GAutomator, both were developed by Chinese companies, likely because the Chinese gaming industry is quite large. Poco has a high number of GitHub stars and its documentation is better than GAutomator. Just reading the README.md was enough to understand how to use it. If I had to choose between the two, I would prefer Poco.

AltTester for Unity is a Unity SDK that only works in Unity. Some of the features, including the inspector and additional functions, are paid features. However, I found out that there is an active community with webinars and Q&A sessions when I participated in the Discord community.

Most of the tools offer test automation for a single device for free. However, for parallel processing, device farm integration was necessary, and in many cases, there were charges associated with using the device farm.

The choice of which game testing automation tool to use is up to you. This article was written to compare Gamium with other similar libraries before introducing Gamium. In the next article, I will continue to explain how to use Gamium.

Dogu Technologies - Seamless Test Automation Platform

Revolutionize your testing workflow with our game and application test automation platform. Execute automated tests using Dogu and optimize your software quality assurance.

Dogu Technologies - Seamless Test Automation Platform