Pengembangan Non-Player Character (NPC) Menggunakan Unity ML-Agents Pada Karting Microgame

Authors

  • Muhammad Yasir Anshari Haq Brawijaya University http://orcid.org/0000-0001-5859-7274
  • Muhammad Aminul Akbar Brawijaya University
  • Tri Afirianto Brawijaya University

DOI:

https://doi.org/10.21111/fij.v7i1.5487

Keywords:

racing game, non-player character, unity ml-agents, reinforcement learning

Abstract

AbstrakPerkembangan teknologi di bidang gim sekarang sudah sangat pesat terutama pada gim balapan. Gim balapan memiliki tujuan untuk memberikan pemain sebuah pengalaman yang menantang dan menyenangkan dalam sebuah balapan melawan mobil yang dikendalikan oleh gim tersebut atau biasa disebut dengan Non-Player Character (NPC). Pengembangan gim balapan tentunya tidak dapat lepas dari pengembangan NPC sebagai lawan main dari pemain. Pada umumnya NPC dikembangkan menggunakan metode waypoint untuk navigasi dalam melintasi trek balapan. Kekurangan dalam metode waypoint adalah harus diatur secara manual untuk setiap trek dan memakan waktu yang lama untuk mengatur waypoint pada setiap trek. Begitu juga untuk membuat NPC balap yang kompetitif dibutuhkan desain rule base yang kompleks. Peneliti mengusulkan menggunakan metode machine learning untuk mengatasi permasalahan tersebut. Unity3D menyediakan sebuah open-source API bernama Unity ML-Agents yang dapat digunakan untuk melatih NPC. NPC dilatih menggunakan metode reinforcement learning dengan Unity ML-Agents yang bertujuan untuk melatih NPC dengan cara memberikan reward agar mencapai hasil yang optimal. Hasil yang didapatkan dengan memanfaatkan Unity ML-Agents adalah NPC yang dapat melintasi berbagai macam trek dan dapat menghindari tabrakan. NPC yang telah dikembangkan dengan Unity ML-Agents juga mendapatkan waktu total yang lebih sedikit dibandingkan dengan waktu total yang diperlukan pemain untuk menempuh 3 lap putaran pada suatu trek yaitu 55,9 detik dibandingkan dengan 59,4 detik.Kata kunci: gim balapan, non-player character, unity ml-agents, reinforcement learning.Abstract[Non-Player Character (NPC) Development Using Unity ML-Agents in Karting Microgame] Nowadays, the development of game technology is very fast, especially in racing games. The racing game aims to provide players with a challenging and fun experience in a race against cars controlled by the game or commonly known as the Non-Player Character (NPC). Of course, the development of a racing game cannot be separated from the development of NPCs as opponents of the players. In general, NPCs were developed using the waypoint method for navigation across racetrack. The disadvantage of the waypoint method is that it must be set manually for each track, and it takes a long time to set the waypoint for each track. Likewise, making a competitive racing NPC requires a complex rule base design. Researchers suggest using machine learning methods to overcome these problems. Unity3D provides an open-source API called Unity ML-Agents which can be used to train NPCs. NPCs are trained using the reinforcement learning method with Unity ML-Agents which aims to train NPCs by providing rewards to achieve optimal results. The results obtained by utilizing Unity ML-Agents are NPCs that can traverse various kinds of tracks and can avoid collisions. NPCs that have been developed with Unity ML-Agents also get less total time compared to the total time required for a player to take 3 laps on a track, which is 55.9 seconds compared to 59.4 seconds.Keywords: racing game, non-player character, unity ml-agents, reinforcement learning.

Author Biographies

Muhammad Yasir Anshari Haq, Brawijaya University

Department of Informatics Engineering, Student

Muhammad Aminul Akbar, Brawijaya University

Department of Informatics Engineering, Lecturer

Tri Afirianto, Brawijaya University

Department of Informatics Engineering, Lecturer

References

[1] "A Visual History of Racing Games - IGN", IGN, 2015. [Online]. Available: https://www.ign.com/articles/2015/03/27/a-visual-history-of-racing-games. [Accessed: 22- Jan- 2020].[2] M. Chan, C. Chan and C. Gelowitz, "Development of a Car Racing Simulator Game Using Artificial Intelligence Techniques", International Journal of Computer Games Technology, vol. 2015, pp. 1-6, 2015.[3] A. Beech, "Asphalt 9: Legends release date and Livestream! | Gameloft Central", Gameloft Central, 2018. [Online]. Available: http://www.gameloft.com/central/asphalt/asphalt-9-legends/asphalt-9-legends-livestream-and-release-date. [Accessed: 22- Jan- 2020]. [4] "IR Information : Sales Data - Top Selling Title Sales Units", Nintendo Co., Ltd., 2019. [Online]. Available: https://www.nintendo.co.jp/ir/en/finance/software/index.html. [Accessed: 22- Jan- 2020]. [5] A. Kim and J. Bae, “Development of Mobile Game Using Multiplatform (Unity3D) Game Engine”, p. 2, 2014. [Accessed 30 November 2020]. [6] Unity Technologies "Karting Microgame | Templates | Unity Asset Store", Assetstore.unity.com, 2019. [Online]. Available: https://assetstore.unity.com/packages/templates/karting-microgame-150956. [Accessed: 22-Jan- 2020].[7] J. Wang and Y. Lin, "Game AI: Simulating Car Racing Game by Applying Pathfinding Algorithms", International Journal of Machine Learning and Computing, pp. 13-18, 2012. Available: 10.7763/ijmlc.2012.v2.82 [Accessed 22 January 2020].[8] A. El Rhalibi, K. Wong and M. Price, "Artificial Intelligence for Computer Games", International Journal of Computer Games Technology, vol. 2009, pp. 1-3, 2009. Available: 10.1155/2009/251652.[9] A. Juliani et al., "Unity: A General Platform for Intelligent Agents", arXiv.org, 2018. [Online]. Available: http://arxiv.org/abs/1809.02627. [Accessed: 22- Jan- 2020].[10] S. Sehad and C. Touzet, “Reinforcement learning and neural reinforcement learning,” ESANN 94 Ed. M Verleysen DFacto Publ. Bruxelles, pp. 1–6, 1994.[11] H. Teigar, M. Storožev and J. Saks, “2D Racing game using reinforcement learning and supervised learning”, p. 1, 2018.[12] F. G. Glavin and M. G. Madden, “Learning to shoot in first person shooter games by stabilizing actions and clustering rewards for reinforcement learning,” 2015 IEEE Conf. Comput. Intell. Games, CIG 2015 - Proc., pp. 344–351, 2015.[13] S. Wender and I. Watson, "Applying reinforcement learning to small scale combat in the real-time strategy game StarCraft:Broodwar", 2012 IEEE Conference on Computational Intelligence and Games (CIG), 2012. Available: 10.1109/cig.2012.6374183 [Accessed 22 January 2020].[14] M. Claypool and K. Claypool, “Perspectives, frame rates and resolutions: It’s all in the game,” FDG 2009 - 4th Int. Conf. Found. Digit. Games, Proc., pp. 42–49, 2009.[15] S. Stewart, "What Is The Best FPS For Gaming? [2020 Guide] - GamingScan", GamingScan, 2020. [Online]. Available: https://www.gamingscan.com/best-fps-gaming. [Accessed: 22- Sep- 2020].

Downloads

Submitted

2021-01-12

Accepted

2021-10-28

Published

2021-10-28

Issue

Section

Articles