How did you do it this year?

Last year someone started a thread with the same topic, and people shared their strategies and methods. I thought it was super interesting! Would love to hear more detail about your guys’ bots and experiences.

From my side, I’ve written a blog post about my bot, if you’re interested: https://japeyjapes.wordpress.com/2019/09/24/entelect-challenge-2019/#a10

2 Likes

@japes, thank you for sharing your method.

My strategy that got me to 4th place:

High score and survive:

  1. Optimized Digging. From the start my worms tried to dig as efficient as possible. I achieved this with an algorithm to inspect all possible move + dig possibilities for 9 moves deep. I used multi-threading, but non-optimized code.
  2. The snowball: If another worm came in range of the technologist with available snowballs, I would throw it. I wanted the score, rather than to attack a frozen worm.
  3. The banana: The agent with available bananas would throw at an opponent in range, but will also throw to mine dirt if there were a chance to get 10 or more dirt.
  4. If an opponent worm got inside the shooting range from my active worm, I used a fight-or-flight algorithm: If the opponent is the active opponent I would flight, otherwise attack. --> This reduced the chance of a shoot-and-miss.
  5. A strategy that worked well for me (got me the win to reach 4th position) was to try to bump the opposition (move to the same spot he is moving to) when my health was higher, but score lower than the opposition. This 20 bump damage worked better than trying to shoot (and mostly miss) the opponent.
  6. Lava: I avoided the lava and moved closer to the center as the lava progressed.

What would have helped me to do better? --> In some games I had 3 worms with little health and the opponent had 1 worm with better health. The 1 worm with better health was in a better position. Maybe I should have protected my commando better.

3 Likes

Think I can get in on this,

I have around 200 Hours of time lapse for the first 2 events. Mainly because I had about 4 complete rebuilds. Hitting some limits with my logic.

I will do a decent post about my strategies soon, and I am open sourcing my bot for anyone to play around with. I do not expect anyone learn much from my code. But as a benchmark it should be nice.

I will also mention I ran out of time for the last event and never implemented the snowball. So I had a major weakness (between 50 and 150 points to be exact). I know I will still implement the Snowball to just make sure my code is not incomplete. But I will keep that separate from my actual competition bot.

Back to my actual strategy, I had conditional strategies which I switched between:

Survive
This was a tree based algorithm that analyzed every possible move 5 levels deep and returned the best surviving move sequence. I drew a heat map for both lava as well as player shot ranges and used that to determine my moves.

I also only used my move choice to dodge incoming shots when I had 2 worms (3 worms greatly weakened the select worm function).

The other major thing I did was to greatly increase heat when an opponent clutters (generally meaning they are aggro and I need to run like a boss)

When they were not cluttered I assumed they were score so I loosened my heat a bit so that my worms do not flee for no reason.

The last gem in my survive strategy was I greatly reduced the enemy active worm’s heat map. The reasoning here was that most worms will not try and guess where you will be and then shoot.

The only exception to this was when I was in a 1V1 setting. In those cases I upped the heat again and avoided enemy territory as much as possible.

Counter
Counter was when I shot at the enemy, This was simple, If an enemy worm as in range, and not the enemy’s current active worm, I shot. If I was in range and the enemy worm was active I moved which mostly resulted in a miss, unless the enemy had the same strategy.

This did not do much for equal matches, but whenever I had less worms than my enemy I out damaged and out dodged them a lot.

Score
My score algorithm was a genetic algorithm that generated the best score for (I used around 0.9s for this loop).

I looked 7 move levels deep. With move being all digs included. So I would move 1 tile and dig everything, then the next, dig all. And so on.

My rule was if I was safe I always dug. and I dug the diagonals first which often gave me some cover when running from enemy worms.

Banana
For my bananabomb I checked the max possible points on the map, and subtracted 3. Whenever I had a clear shot at any tiles that had that I shot. And I also added enemy positioning in as extra score bonuses.

Sacrifice
My final function was a sacrifice one. Whenever one of my worms was below 10 health, I killed it off.
First, killing it myself stopped the enemy from getting the points for killing my worm. and B, it prevented some strategies where the enemy keeps worms alive to not get the worms advantage. And in doing this I got some advantage.

I was extremely paranoid over the kill my own worm tactic, Where one player sacrifices worms to better kill the opponent. Basically, start the game, kill your commando and your agent. Then weaken all the enemy worms with your tactician. Finally when they are weak enough, kill the enemy worms and freeze the final one.

So I used the lava to protect myself from losing advantage.

Snowball
I am still going to implement the snowball. but in the perfect world I feel it will look as follows:
Use the snowball when 2 or 3 enemies are in shooting range from one of your worms. Or when the health drops below 50 (Meaning some damage).

If the enemy worms are never in shooting range. You need to have a round check and if you still have snowballs round 350+ you need to start using them. This will likely never happen, but in 1 V 1 on my bot it happens almost every time.

With all that said that was some of the strategies I followed this year.

I just pushed hard. The Entelect Challenge is the only time I write AI, so I can really sit and enjoy it for hours at a time.

Will post some more info on my Bot’s source in the forums soon.

4 Likes

Hi, let me also jump in on this one

The approach which got me to second place was guided Reinforcement Learning. I recently became an A.I enthusiast and wanted to apply some of the theory I’ve learned. So I chose not to use any tree searching but to try out Machine Learning.

So I designed some features and trained their weights. The bot trained from amateur to above average after a few days. However, the progress of the bot was too slow, as is expected on a personal machine. So after training I tweaked the weights manually until I was somewhat satisfied.

PS: I’ve started releasing videos regarding Machine Learning which will cover the approach I used, Reinforcement Learning, on my YouTube Channel https://www.youtube.com/channel/UCe3Zd8h8h9HYHWJva8DvrlA

2 Likes

Very interesting to hear about the different approaches so far. Definitely didn’t expect anyone in the finals to have used reinforcement learning!

I’m a bit late but I can describe my strategy as well. I came 1st, 2nd, 5th, 5th. My strategy was always to win on points, which worked very well in the first two rounds, but the lava really evened things out - plus I think the way the rounds worked allowed others to catch up in terms of strategy.

I didn’t use any kind of tree search in my bot, not for digging and not for running away. I actually started out with a tree search bot in C++ but I didn’t make any of the modifications @japes made to the standard MCTS and so the rollouts of all 400 rounds were just too long for it to be any good at anything other than running away. My bot was written in Python.

I think my strategy was simpler overall than many that have been described, though there was a bit of detail in exactly how it chose where to run away to and when it chose to shoot.

Running away

It would run away when under threat, mostly just to the cell furthest from the opponent worms, but there was a lot of detail here, though I didn’t use heatmaps or anything like @WillieTheron. It did things like assume the opponent in range would shoot it, and treat other cells as dangerous or not dangerous accordingly. It also knew exactly which worms would be active before each worm’s next turn, and only ran away from those - worms not active before the next turn can be run away from later, saving valuable time for digging!

Shooting

It would engage with a worm instead if it was a free shot (the other worm wasn’t active before my worm’s next turn) or if it had higher health, meaning that a 1-on-1 battle would end in victory. It would also engage at the end (once there was no dirt available inside the lava ring) if it was behind on points. But it would also run away in the inner ring if it was ahead on points, and as in the rest of the game, the player running away had the advantage. It lost the last game in the finals like this.

Digging

I didn’t use any tree search or anything here either. If a worm wasn’t in danger and was adjacent to dirt, it would dig it. Otherwise, I calculated the sum of the inverse square distances to each cell of dirt, from each air cell adjacent to my worm, and moved to the cell with the highest value. This was very simple but very effective in picking which dirt to dig next. Squaring the distance gave stronger priority to close dirt which could be dug soon, but that could be balanced by more dirt further away. Plus it has infinite “depth” compared to tree search approaches with a set depth limit.

Selects

It used the select move to run away too - if a non-active worm was in danger (generally because it couldn’t run away fully in its own turn), and it had somewhere safe to run, it would select that worm and run. The strategy of using the selects to shoot rapid-fire was actually more effective in Round 2 (judging by my opponents’ games against me) and I would have switched to it except for the extra importance in being able to run away well with the lava shrinking the game field.

Bananas

It used the bananas to dig and score points, right at the start (as long as it could hit enough dirt cells) as all the top bots seemed to do in Round 2. By Round 3 I think some had kept their bananas for the end where all the worms were close together, and I probably should have done this too.

Snowballs

I didn’t make much use of the snowballs - I just threw them when an enemy was in range, but I didn’t follow up with attacks or anything. Probably would have been better to keep those for the end of the game in the finals too. Should definitely have done more with this.

Lava

I didn’t do much with the lava except avoid it - I marked cells that the lava was going to expand to next, as hot, and avoided them too. I made minimal changes for Round 3 really.

I’ve uploaded my bot to Github if anyone is interested: https://github.com/mallin/Worms-Bot

2 Likes