Bot time outs

[LouisLotter]

Hi.
I would like to know if there is any room for investigating issues with bot runs, Fixing the problem if it is an issue on the bot runner and re-running them with the same seeds ??

My bot is timing out from 1 to 20 times randomly in most matches.

I’m convinced it’s an issue with the bot runner or the infrastructure as this has been reported by other players as well while testing their bots and I have even seen the coffeeref bot doing it.

I was very careful to limit my bot to 850 ms thinking time (I tried reducing this to 500 MS or less and still saw issues) and to have a backup move ready, I restricted my bot to 300 MS on my laptop and I could never duplicate these issues.

I have to say I dedicated a lot of time to this challenge and it would leave quite a bitter taste if it’s ruined by technical issues on the bot runner.

I’m also very happy to accept the verdict of an investigation if it indicates that my bot is to blame.

Thanks in advance.

[demaniak]

Part of the problem might be that you start timing when you receive the request.
BUT the engine starts timing when it makes the request. If there is any significant lag between making the request, and the bot receiving it…possible problems.
I’m obviously reaching here

[LouisLotter]

Well a delay of more that 150 MS on just receiving a request would be a bug in my book…
That’s a massive penalty.

[demaniak]

Yeah, 150ms lag is like… 1995 dialup…

[floreggian_entelect]

When the tournament has run its course we will verify the matches. If any matches need rerunning, we will rerun them.

[LouisLotter]

Thanks, but If possible I’d like to understand what’s causing the problems. This is mostly a learning experience for me so knowing what the problem is and learning from it is more important to me than the outcomes of the tournament.

[floreggian_entelect]

When the tournament is complete, I will investigate your bot. And see if I can find anything. Right now, I unfortunately have to concentrate on the latest tournament.

[LouisLotter]

Wow I really appreciate that thank you.

[demaniak]

Just btw, don’t know if you are looking, but you are, Mister @LouisLotter, numero de uno currently.

[LouisLotter]

Yeah, that’s awesome. I’m super happy but honestly I care more about the timeouts and why it’s happening… It’s shameful and unpredictable.

[demaniak]

Like a sore tooth, right?
Or that weird stupid bug that keeps popping up intermittently in production, but nowhere else… and you lie awake in the evenings pondering, worrying, ruminating, conjecturing.
Know the feeling guy, know that feeling.

[LouisLotter]

As another data point. I noticed with my game against JP da Conceicao I did not time out at all but he did once at the end. Could be why I beat him…
So this is affecting other players as well.

[LouisLotter]

round:202
player: id:2 position: y:2 x:1484 speed:3 state:HIT_MUD statesThatOccurredThisRound:USED_TWEET, HIT_OIL, HIT_MUD boosting:false boost-counter:0 damage:2 score:473 powerups: OIL:13, LIZARD:9, EMP:1, TWEET:2
opponent: id:1 position: y:2 x:1488 speed:3

[░░░░░░░░░░▓░░░░░░░░░║]
[░░░█░2░░░1░░░░░T░░░░C║]
[░░░░░░░░C░░░░░░░░░░░║]
[░░░░░░░░░░░░░░░░░░#░░║]

16:57:41,435 INFO Player B - MadMinimax: Got command [null]
16:57:41,435 WARN Player B - MadMinimax: No command provided, falling back to default no command
16:57:41,437 INFO Completed round: 202
16:57:41,454 INFO ******************************************************************************************************
16:57:41,454 INFO Starting round: 203
16:57:41,454 INFO Player A - LCubed: Map View
16:57:41,454 INFO ======================================================================================================
round:203
player: id:1 position: y:2 x:1496 speed:8 state:USED_BOOST statesThatOccurredThisRound:USED_BOOST boosting:true boost-counter:5 damage:2 score:374 powerups: OIL:16, LIZARD:2, TWEET:2
opponent: id:2 position: y:2 x:1487 speed:3

[░░░░░░░░░║]
[░░░T░1░░C║]
[░░░*░░░░░║]
[░░░░░░#░░║]

16:57:41,505 INFO Player A - LCubed: Got command [TURN_LEFT]
16:57:41,506 INFO Player B - MadMinimax: Map View
16:57:41,506 INFO ======================================================================================================
round:203
player: id:2 position: y:2 x:1487 speed:3 state:NOTHING statesThatOccurredThisRound:NOTHING boosting:false boost-counter:0 damage:2 score:473 powerups: OIL:13, LIZARD:9, EMP:1, TWEET:2
opponent: id:1 position: y:2 x:1496 speed:8

[░░░░░░░▓░░░░░░░░░║]
[█░▓░░2Φ░░░░░T░1░░C║]
[░░░░░C░░░░░░░░░░░║]
[░░░░░░░░░░░░░░░#░░║]

16:57:42,166 INFO Player B - MadMinimax: Got command [TURN_LEFT]
16:57:42,167 INFO Completed round: 203
16:57:42,182 INFO ******************************************************************************************************
16:57:42,183 INFO Game Complete
16:57:42,184 INFO Checking if match is valid
16:57:42,186 INFO =======================================
16:57:42,186 INFO The winner is: A - LCubed

A - LCubed - score:374 health:0
B - MadMinimax - score:470 health:0

16:57:42,186 INFO =======================================
16:57:42,187 INFO Player A - LCubed (id 6857b020-4a87-4c61-b186-1fed4558aa21): Signaling bot runner to shutdown
16:57:42,191 INFO Player B - MadMinimax (id cb057bb9-8315-4b21-8609-6b3f05e8cfe3): Signaling bot runner to shutdown
16:57:42,200 INFO

[japes]

Eish, this is worrying! Thanks for picking it up, @LouisLotter.
(also wow, what a nailbiter of a race!)

I’m sure entelect will get to the bottom of it . Rematches with the same seeds seems the most fair to me.

I haven’t seen my bot take longer than 300ms to make a decision on my side.

[WillieTheron]

Hey,

No real troubles on my side.

I also noticed some strange behaviour in one of my matches (only checked 2).

Im not too bothered as I saw a few of my losses were last minute changes that stole a matchpoint here and there.
Bothersome but humbling. Something I needed.

Would you be able to run for example a script to check if there was timeouts in a match?
In one of my matches I had an instance where I had a timeout and the next 2 moves was completely different from the moves the same seed would have made.

Could have been a bug but im still suspitions.
As for my own bot timing out, Im not saying its impossible. But my rounds are constantly under 30 MS. So a jump to 1000 MS means I have some endless loop bug (Not unlikely) or an issue with the engine itself.

If any more intel is needed feel free to let me know.

So I would say consider flaging matches that had timeouts and doing a rerun. Maybe once all matches have concluded.

Just thought I would chime in on this thread as I have also seen a strange timeout followed by a very random move and it all happened in the first 5 rounds of a match. Can send more details in DM. If required. Such as VS who and so on. Just incase it would help.

[LouisLotter]

The problem with just rerunning is that these issues could affect the Finals. I’d prefer some investigation into the causes first.

Alternatively I guess we could just rerun important games that determine the top 8 and run the Finals on a desktop machine with no VM’s and Docker containers to cause problems.

[kobus-v-schoor]

Just thought I’d share, if you’re on Linux (or at least have access to grep) you could run this in a folder where you extracted all your matches to find timeouts (since you asked about a script to find timeouts)

grep 1000 */*/*/Player*

EDIT: Previous command seems to return some false positives - this one gives more consistent results:

grep "No Command" */*/*/Player*

[LouisLotter]

I just lost a game where I timed out more than a 100 times…
It seems it gets worse as the tournament runs.

Is the bot runner killing of bots properly ? I noticed that on my machine it would not always do that
properly.

[LouisLotter]

Yeah, I’m starting to lose a lot of games now… with tons of timeouts.

[GawieJooste]

@LouisLotter an interesting thing I noticed while checking a couple of my games is that it seems the only games me or the opponent timed out were against your bot. I did not check all my games though so this could be incorrect and just a coincidence.