Submission to player portal fails with some API error

Good afternoon,

I recently made a submission to test out my bot on the player portal but the match fails with some internal API error (seems to be trying to talk to the tournament server?). Output below:

Downloading game engine
Downloading 2020.1.2.zip
Extracting zip: engine-ac42953a-8d65-4be0-b393-e88be1b79810.zip
Retrieving bot path from directory
Downloading bots
Downloading <omitted because I don't know if somebody can use this to download my bot>
Downloading reference-bots/2020-1-2/reference-bot.zip
Extracting zip: <omitted because I don't know if somebody can use this to download my bot>
Extracting zip: player-ac8c2544-4cb6-49a5-bdae-2bd068ce2cf2.zip
Config for player A : <omitted because I don't know if somebody can use this to download my bot>
Failed call: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Retrying request: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Failed call: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Retrying request: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Failed call: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Retrying request: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Failed call: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Retrying request: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Failed call: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Retrying request: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Failed call: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Retrying request: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Failed call: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Retrying request: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Failed call: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Retrying request: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Failed call: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Retrying request: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Failed call: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Retrying request: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
Exception during match execution
java.io.IOException: Request failed: Request{method=POST, url=http://localhost:55555/instantiate_bot, tags={class retrofit2.Invocation=za.co.entelect.challenge.network.BotServices.instantiateBot() [okhttp3.MultipartBody$Part@be35cd9]}}
	at za.co.entelect.challenge.network.RetryableCall.execute(RetryableCall.java:48) ~[game-runner.jar:?]
	at za.co.entelect.challenge.player.TournamentPlayer.instantiateBot(TournamentPlayer.java:60) ~[game-runner.jar:?]
	at za.co.entelect.challenge.player.TournamentPlayer.<init>(TournamentPlayer.java:47) ~[game-runner.jar:?]
	at za.co.entelect.challenge.player.bootstrapper.PlayerBootstrapper.parsePlayer(PlayerBootstrapper.java:75) ~[game-runner.jar:?]
	at za.co.entelect.challenge.player.bootstrapper.PlayerBootstrapper.loadPlayers(PlayerBootstrapper.java:49) ~[game-runner.jar:?]
	at za.co.entelect.challenge.engine.bootstrapper.GameBootstrapper.run(GameBootstrapper.java:75) [game-runner.jar:?]
	at za.co.entelect.challenge.engine.bootstrapper.GameBootstrapper.main(GameBootstrapper.java:55) [game-runner.jar:?]
Notifying of match failure
Creating zip: 250b7f74-8db7-41b7-b553-2af43c0d5ee0 for directory: ./match-logs

Hi @kobus-v-schoor,

Will pass this error on to the team and get back to you with feedback.

Hi @kobus-v-schoor ,

From the team’s investigation it looks like the bot is not starting up or taking too long to start up. The runner will retry 10 times giving 5 seconds for the bot to respond. How long does your bot take to start up? Perhaps try and download the bot again and submit it without any changes to see if the error still occurs.

Hi @Meerkat,

Thanks for getting back to me - I believe I’ve found the issue, my bot.json had a leading “/” in the botLocation parameter (worked fine locally, seems it only became a problem when I uploaded it) - it now runs correctly. Sorry for the trouble

1 Like

Could we please get a proper stack trace / match log? I’m getting the same error on the Match Status for my submission. It’s definitely launching fast enough. I’m pretty sure I won’t be able to figure out how to fix this before the first deadline

1 Like

Hi @marcin_k - make sure that your bot location starts with a slash and doesn’t end with one e.g. “/src” - mine worked fine locally without this but it failed when uploading

it was in the root directory like the python3 starter bot, so ‘/’. I also tried to put the program in a ‘/src’ subdirectory but it still doesn’t work.
i’ve just accepted that it won’t be working this round :man_shrugging:

Try without the starting slash? Don’t know which language you’re using but a few of the bot runners add ./ to the start of the bot location. You can see what they’re doing here: https://github.com/EntelectChallenge/2020-Overdrive/tree/develop/game-runner/src/main/java/za/co/entelect/challenge/botrunners . Good luck!

I don’t think it’s that, I ended up submitting basically the python3 starter bot and it worked correctly (and ended up playing in the tournament :sweat_smile:). The starter bot also has ‘/’ as the location.

I think it has to do either with the bot starting too slowly, or with not pulling dependencies correctly.

The stacktrace is certainly a great idea, and we can’t promise that as of yet, but it would certainly help everyone out a bunch.

Thank you for the feedback,
Could you please have a look at my bot submission (I can DM my details, not sure how to best do this) which failed to run ~30m after successfully building? I downloaded the bot binaries but nothing stands out to me about why it didn’t work. I thought it might be the requirements.txt dependencies but they seem to be all downloaded correctly

Hi @marcin_k

All the logs provided to us by the build server are in the downloadable packages on the portal submission screen, so we won’t be able to get more info about this than that. If the starter-bot works fine, you could try moving your logic in there and just rename the bot.

Hi @pierre.roux,

Since the python starter bot run correctly on the portal I’ve tried to incrementally add to the starter bot and see at what point the portal runner fails to run the bot. It fails at my first step, adding the PyTorch dependency. You can replicate it yourself like this:

  1. Copy the python3 starter bot
  2. Add torch to the requirements.txt file
  3. Submit the bot

You can see that the bot isn’t even importing Pytorch in the script, never mind using a model to choose an action. It’s simply adding the dependency that causes the starterbot test matches to fail.

Could you please have a look and see why adding the dependency causes the bot to fail (after 20 minutes of retrying)? I understand you guys are open to neural network based submissions but at the moment it seems like neither Tensorflow nor Pytorch based submissions run on the portal.

@marcin_k I can confirm this happens with adding tensorflow to requirements.txt also. I used the latest release (with the cybertruck/lizard additions), fixed the issues that seemed to have been overlooked in the python3 starter bot, confirmed it worked with the repo version of requirements.txt and then added ‘tensorflow’ to the requirements .txt. After successfully building, ‘Match Status’ is marked as failed with a failure to instantiate the bot reported.

@pierre.roux this is related to my issues at Python + TF tournament submission issue . I thought it was exclusively related to the slow import times of tensorflow, but after reading @marcin_k’s post I tried with tensorflow and realised the problem occurs even without trying to import the library.

It’s very strange that just adding the dependency to requirements.txt seems to cause the issue. I think there may be an issue with python bots being built in your pipeline. I am completely stumped.

I think one way to resolve with would be to bake Pytorch & TF (the two core machine learning libraries for python) into the Python3 docker image. That way each bot could just import tensorflow without adding anything into a requirements or setup.py file. This is the way that online machine learning competitions I’ve tried do it

Alternatively there could be a seperate python3-ml bot lang which has these dependencies baked in.

@marcin_k I think your best bet is to do what you originally suggested with something like a Golang runner. This is a non-issue that I’ve spent over two weeks trying to get fixed. Taking into account that we’re pretty much using the gold standard for a ML solution, this is very frustrating.

I suspect this may be related to the issues I have been having. I think the job runner is very sensitive to the size and number of files it copies across. I hope they can increase it’s wait time or something to make it more robust.

Goodluck @LouisLotter I’ve been waiting for 4+ weeks and I haven’t heard anything. At this point I’m treating the competition as a learning exercise rather than actually trying to compete.

Hi everyone,

We are looking at all the possible solutions.