Original Post
[REL] Return of the machines: Machine learning environment using Toribash
In the past people have tried genetic algorithms multiple separate times to teach computers to play Toribash, including at least one academic publication. They had their limitations, but seemed very promising. Now several years later, shall we take another shot at it with recent advances in such fields?
Enter Toribash Learning Environment (ToriLLE), a platform designed for machine learning algorithms, meaning it is convenient to use from Python (de-facto machine learning language at the moment), it runs on Linux and runs fast enough. With the included OpenAI Gym environments people can try their existing machine learning methods on Toribash in a jiffy. We also wrote a white paper on the matter in the hopes it will lure more academy side people trying this environment out (granted, it is not the most modifiable environment).
Using the environment we trained proximal policy optimization (PPO) on smaller tasks to validate if Toribash is indeed suitable as a learning environment. Like in the past we tried destroying an Uke (damage = reward), but we also tried "run-away" scenario where player has to move away from the centre of the map.
Compared to previous attempts we positively tearing a new one for the uke! Maybe this time around we could also train and try these agents against human players?
If anybody feels like they want to contribute, support for objects and custom levels would be something very cool to have (i.e. getting the information on Python side). I do not know if Toribash supports getting information of such objects, and even after that it needs to be planned out how to share the information with Python. Changing mod is easy enough, but there is no point in that if we can't "see" the level.
Special thanks to hampa and Drax for helping out with some crucial parts, and user "box" for initial, encouraging comments.
Edit: Another video of PPO learning to attack Uke, this time with harder environment (random distance to Uke) and longer training (100k games rather than 10k). Looks like there is still room for improvement:
Enter Toribash Learning Environment (ToriLLE), a platform designed for machine learning algorithms, meaning it is convenient to use from Python (de-facto machine learning language at the moment), it runs on Linux and runs fast enough. With the included OpenAI Gym environments people can try their existing machine learning methods on Toribash in a jiffy. We also wrote a white paper on the matter in the hopes it will lure more academy side people trying this environment out (granted, it is not the most modifiable environment).
Using the environment we trained proximal policy optimization (PPO) on smaller tasks to validate if Toribash is indeed suitable as a learning environment. Like in the past we tried destroying an Uke (damage = reward), but we also tried "run-away" scenario where player has to move away from the centre of the map.
Compared to previous attempts we positively tearing a new one for the uke! Maybe this time around we could also train and try these agents against human players?
If anybody feels like they want to contribute, support for objects and custom levels would be something very cool to have (i.e. getting the information on Python side). I do not know if Toribash supports getting information of such objects, and even after that it needs to be planned out how to share the information with Python. Changing mod is easy enough, but there is no point in that if we can't "see" the level.
Special thanks to hampa and Drax for helping out with some crucial parts, and user "box" for initial, encouraging comments.
Edit: Another video of PPO learning to attack Uke, this time with harder environment (random distance to Uke) and longer training (100k games rather than 10k). Looks like there is still room for improvement:
Last edited by Miffylin; Aug 6, 2018 at 07:54 PM.
Originally Posted by matarika
I was overwhelmed by the amount of information. Lmao Someone explain it in dumb people language, what is this
ToriLLE: A way for people to play Toribash from their own code (mainly Python, but can use other programming languages too)
Video: A recent-ish reinforcement learning method trained to play Toribash. ToriLLE gives points on damaging the Uke, and algorithm learns via trial'n'error what gives most points.
You're doing gods work with this. Very cool
The 20k running away attempt is better than most newish players would manage, very interested in seeing how it performs with more training.
You should collaborate with hampa and the devs if you aren't already, maybe you can make something happen together in Next
The 20k running away attempt is better than most newish players would manage, very interested in seeing how it performs with more training.
You should collaborate with hampa and the devs if you aren't already, maybe you can make something happen together in Next
Originally Posted by jisse
You're doing gods work with this. Very cool
The 20k running away attempt is better than most newish players would manage, very interested in seeing how it performs with more training.
You should collaborate with hampa and the devs if you aren't already, maybe you can make something happen together in Next
I am currently running bit longer experiments with more sophisticated setups (agent sees Uke, engagement distance is randomized). Hopefully those will be more interesting in terms of movement. In Run-Away scenario the reward is given on where head has moved, so shouldn't detaching and throwing the head away be quite good?
hampa did already help with this a bit, and we did chat little about Next. As it will use Unity, it may not be as lightweight to run as this Toribash, but will see.
