Pengembangan Non-Player Character (NPC) Menggunakan Unity ML-Agents Pada Karting Microgame
DOI:
https://doi.org/10.21111/fij.v7i1.5487Keywords:
racing game, non-player character, unity ml-agents, reinforcement learningAbstract
AbstrakPerkembangan teknologi di bidang gim sekarang sudah sangat pesat terutama pada gim balapan. Gim balapan memiliki tujuan untuk memberikan pemain sebuah pengalaman yang menantang dan menyenangkan dalam sebuah balapan melawan mobil yang dikendalikan oleh gim tersebut atau biasa disebut dengan Non-Player Character (NPC). Pengembangan gim balapan tentunya tidak dapat lepas dari pengembangan NPC sebagai lawan main dari pemain. Pada umumnya NPC dikembangkan menggunakan metode waypoint untuk navigasi dalam melintasi trek balapan. Kekurangan dalam metode waypoint adalah harus diatur secara manual untuk setiap trek dan memakan waktu yang lama untuk mengatur waypoint pada setiap trek. Begitu juga untuk membuat NPC balap yang kompetitif dibutuhkan desain rule base yang kompleks. Peneliti mengusulkan menggunakan metode machine learning untuk mengatasi permasalahan tersebut. Unity3D menyediakan sebuah open-source API bernama Unity ML-Agents yang dapat digunakan untuk melatih NPC. NPC dilatih menggunakan metode reinforcement learning dengan Unity ML-Agents yang bertujuan untuk melatih NPC dengan cara memberikan reward agar mencapai hasil yang optimal. Hasil yang didapatkan dengan memanfaatkan Unity ML-Agents adalah NPC yang dapat melintasi berbagai macam trek dan dapat menghindari tabrakan. NPC yang telah dikembangkan dengan Unity ML-Agents juga mendapatkan waktu total yang lebih sedikit dibandingkan dengan waktu total yang diperlukan pemain untuk menempuh 3 lap putaran pada suatu trek yaitu 55,9 detik dibandingkan dengan 59,4 detik.Kata kunci: gim balapan, non-player character, unity ml-agents, reinforcement learning.Abstract[Non-Player Character (NPC) Development Using Unity ML-Agents in Karting Microgame] Nowadays, the development of game technology is very fast, especially in racing games. The racing game aims to provide players with a challenging and fun experience in a race against cars controlled by the game or commonly known as the Non-Player Character (NPC). Of course, the development of a racing game cannot be separated from the development of NPCs as opponents of the players. In general, NPCs were developed using the waypoint method for navigation across racetrack. The disadvantage of the waypoint method is that it must be set manually for each track, and it takes a long time to set the waypoint for each track. Likewise, making a competitive racing NPC requires a complex rule base design. Researchers suggest using machine learning methods to overcome these problems. Unity3D provides an open-source API called Unity ML-Agents which can be used to train NPCs. NPCs are trained using the reinforcement learning method with Unity ML-Agents which aims to train NPCs by providing rewards to achieve optimal results. The results obtained by utilizing Unity ML-Agents are NPCs that can traverse various kinds of tracks and can avoid collisions. NPCs that have been developed with Unity ML-Agents also get less total time compared to the total time required for a player to take 3 laps on a track, which is 55.9 seconds compared to 59.4 seconds.Keywords: racing game, non-player character, unity ml-agents, reinforcement learning.References
[1] "A Visual History of Racing Games - IGN", IGN, 2015. [Online]. Available: https://www.ign.com/articles/2015/03/27/a-visual-history-of-racing-games. [Accessed: 22- Jan- 2020].[2] M. Chan, C. Chan and C. Gelowitz, "Development of a Car Racing Simulator Game Using Artificial Intelligence Techniques", International Journal of Computer Games Technology, vol. 2015, pp. 1-6, 2015.[3] A. Beech, "Asphalt 9: Legends release date and Livestream! | Gameloft Central", Gameloft Central, 2018. [Online]. Available: http://www.gameloft.com/central/asphalt/asphalt-9-legends/asphalt-9-legends-livestream-and-release-date. [Accessed: 22- Jan- 2020]. [4] "IR Information : Sales Data - Top Selling Title Sales Units", Nintendo Co., Ltd., 2019. [Online]. Available: https://www.nintendo.co.jp/ir/en/finance/software/index.html. [Accessed: 22- Jan- 2020]. [5] A. Kim and J. Bae, “Development of Mobile Game Using Multiplatform (Unity3D) Game Engine”, p. 2, 2014. [Accessed 30 November 2020]. [6] Unity Technologies "Karting Microgame | Templates | Unity Asset Store", Assetstore.unity.com, 2019. [Online]. Available: https://assetstore.unity.com/packages/templates/karting-microgame-150956. [Accessed: 22-Jan- 2020].[7] J. Wang and Y. Lin, "Game AI: Simulating Car Racing Game by Applying Pathfinding Algorithms", International Journal of Machine Learning and Computing, pp. 13-18, 2012. Available: 10.7763/ijmlc.2012.v2.82 [Accessed 22 January 2020].[8] A. El Rhalibi, K. Wong and M. Price, "Artificial Intelligence for Computer Games", International Journal of Computer Games Technology, vol. 2009, pp. 1-3, 2009. Available: 10.1155/2009/251652.[9] A. Juliani et al., "Unity: A General Platform for Intelligent Agents", arXiv.org, 2018. [Online]. Available: http://arxiv.org/abs/1809.02627. [Accessed: 22- Jan- 2020].[10] S. Sehad and C. Touzet, “Reinforcement learning and neural reinforcement learning,” ESANN 94 Ed. M Verleysen DFacto Publ. Bruxelles, pp. 1–6, 1994.[11] H. Teigar, M. Storožev and J. Saks, “2D Racing game using reinforcement learning and supervised learning”, p. 1, 2018.[12] F. G. Glavin and M. G. Madden, “Learning to shoot in first person shooter games by stabilizing actions and clustering rewards for reinforcement learning,” 2015 IEEE Conf. Comput. Intell. Games, CIG 2015 - Proc., pp. 344–351, 2015.[13] S. Wender and I. Watson, "Applying reinforcement learning to small scale combat in the real-time strategy game StarCraft:Broodwar", 2012 IEEE Conference on Computational Intelligence and Games (CIG), 2012. Available: 10.1109/cig.2012.6374183 [Accessed 22 January 2020].[14] M. Claypool and K. Claypool, “Perspectives, frame rates and resolutions: It’s all in the game,” FDG 2009 - 4th Int. Conf. Found. Digit. Games, Proc., pp. 42–49, 2009.[15] S. Stewart, "What Is The Best FPS For Gaming? [2020 Guide] - GamingScan", GamingScan, 2020. [Online]. Available: https://www.gamingscan.com/best-fps-gaming. [Accessed: 22- Sep- 2020].
Downloads
Submitted
Accepted
Published
Issue
Section
License
Please find the rights and licenses in the Fountain of Informatics Journal (FIJ). By submitting the article/manuscript of the article, the author(s) agree with this policy. No specific document sign-off is required.
1. License
The non-commercial use of the article will be governed by the Creative Commons Attribution license as currently displayed on Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
2. Author(s)' Warranties
The author warrants that the article is original, written by the stated author(s), has not been published before, contains no unlawful statements, does not infringe the rights of others, is subject to copyright that is vested exclusively in the author, and free of any third party rights, and that any necessary written permissions to quote from other sources have been obtained by the author(s).
3. User/Public Rights
FIJ's spirit is to disseminate articles published are as free as possible. Under the Creative Commons license, FIJ permits users to copy, distribute, display, and perform the work for non-commercial purposes only. Users will also need to attribute authors and FIJ on distributing works in the journal and other media of publications. Unless otherwise stated, the authors are public entities as soon as their articles got published.
4. Rights of Authors
Authors retain all their rights to the published works, such as (but not limited to) the following rights;
- Copyright and other proprietary rights relating to the article, such as patent rights,
- The right to use the substance of the article in own future works, including lectures and books,
- The right to reproduce the article for own purposes,
- The right to self-archive the article (please read out deposit policy),
- The right to enter into separate, additional contractual arrangements for the non-exclusive distribution of the article's published version (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal (Jurnal Optimasi Sistem Industri).
5. Co-Authorship
If the article was jointly prepared by more than one author, any authors submitting the manuscript warrants that he/she has been authorized by all co-authors to be agreed on this copyright and license notice (agreement) on their behalf, and agrees to inform his/her co-authors of the terms of this policy. FIJ will not be held liable for anything that may arise due to the author(s) internal dispute. FIJ will only communicate with the corresponding author.
6. Royalties
Being an open accessed journal and disseminating articles for free under the Creative Commons license term mentioned, author(s) aware that FIJ entitles the author(s) to no royalties or other fees.
7. Miscellaneous
FIJ will publish the article (or have it published) in the journal if the article’s editorial process is successfully completed. FIJ's editors may modify the article to a style of punctuation, spelling, capitalization, referencing, and usage that deems appropriate. The author acknowledges that the article may be published so that it will be publicly accessible and such access will be free of charge for the readers as mentioned in point 3.
