Soccer Simulation Demo 2021 - Issues and questions

Hello, everyone.

This topic was created to concentrate all questions and suggestions you might have about the Soccer Simulation Demo of February 2021.

As we mentioned in the call, the idea of this demo competition was to serve as a trial for the official simulation competition to be held in June 2021. To help us improve, we welcome your suggestions and observations about the whole process.

If you want to report a bug or issue observed during the matches, please try to give as much detail as possible to help us identify the problem: identify your team, the exact match and time stamp you are referring to, and explain what you think the problem is.

The links for results and videos of all games can be found in the following topic: Soccer Simulation Demo 2021 - Results

On behalf of the RCJ Soccer TC and RCJ Soccer Simulation OC, I thank you all in advance!
Felipe.

P.S. 1: Please, note that we will not re-run matches. As stated before, we are having this demo competition to learn what we need to improve in order to have a good competition in June. We understand that this might be frustrating, but running the competition is taking much more time and effort than we anticipated and we simply don’t have the capacity to replay matches at this point.

P.S. 2: If you are looking for the simulator or competition rules, check out the post below: Soccer Simulation Demo Competition 2021

Hello @Felipe,

We are Team Reset from Taiwan (in Group 1 for the group phase) that participated in the demo competition and were wondering how the programs were assigned to the robots. When we looked at all four of the recorded matches, we realized that there seems to be one set of code on two different robots (one set of code seems to be unused), and this only happened when we were facing the blue goal (it was normal when we were trying to score on the yellow goal). We did double-check our code and we didn’t see how this problem could be possible since at no point was the offensive code ever copied into the defensive program. We also tested the sets of code that we submitted, with one set of code assigned to one robot for both sides (blue and yellow) locally on Webots, and we didn’t have the same problem.

In “Screenshot 1” of this folder, you can see that when we were attacking the yellow goal, our defensive robot was normal (stayed near the penalty area). However, in that exact same match, after halftime when the robots switched sides and we were attacking the blue goal, as you can see in “Screenshot 2” of this folder, our defensive robot was operating the offensive program.

So we just wanted to ask if there’s a chance that there’s an assignment error when initializing the competition, and if it is possible to have our robots reassigned code such that each robot gets a unique set of code instead of two robots running the same set of code and one set of code being left unused.

Thank you!

Sincerely, Team Reset

Hello @thomastseng0380,

what we ended up doing is editing everybody’s code to make the imports not collide. Changes I made to your code look like this:

What this does is it moves everything that you imported to team_libraries which gets mounted into the Docker container that runs the simulation as /app/controllers/team_033_libraries, your robot code gets mounted from robot1.py to /app/controllers/rcj_soccer_player_y1/rcj_soccer_player_y1.py or /app/controllers/rcj_soccer_player_b1/rcj_soccer_player_b1.py depending on what side you are playing. If spot anything that could have changed your bots’ behavior let me know, we’ll try and fix it for the future; this demo competition will have remain on the current code base though.

I have just run your code against the demo robots (as Team 999) and whichever side I put them on the robots did all move and there were no import-errors (or errors of any kind) on the Webots-console (other than that the container has no audio and my laptop has low GPU power:

For the future we’re hoping to have an upload format that requires fewer or no changes and maybe a test-upload so that you can see your bots run under “competition conditions”.

Best Regards,
David

1 Like

Hello rcj TC & simulation OC,

I’m writing on behalf of GJH_team, as to the latest matches.

While watching our games today, we couldn’t help but notice, that one of our robots was exhibiting unusual behavior and acting exactly like the default code.

So we decided to test this on our local simulation (whether there is a mistake on our side) by extracting the same .zip file, and when right controllers were selected on both sides, all worked perfectly, and we were unable to recreate the behavior exhibited by the robot in the matches (unless we selected default code for one of the robots, in which case we had 100% the same results). This behaviour was directly responsible for multiple goals we received (our robots rely on each one fulfilling its role), and basically meant we were playing 2v3 for one half.

After further review, this seemed to happen on every match we faced blue, on the Y2 robot exclusively. Again, our code was the same for all robots (with slight include changes), and 100% identical between Y2 & Y3. Our Y3 robot worked fine.

Now we may be wrong and will be doing some more testing, but we deem it that there was a mistake on our side (alas still very possible).
Could you please check if the correct codes were selected?

Thank you for the brilliant competition,

let’s hope we can get to the bottom of this,

GJH_team

Hello,

There is a high possibility that all the Y2 robots are running the demo code.

We have randomly chosen one sample from each group in the Group Phase, one sample from the Round of 16, and one sample from the Quarters Final.

Below is the list of samples:

Group1 - 2 vs 33
Group2 - 17 vs 46
Group3 - 19 vs 36
Group4 - 1 vs 9
Group5 - 11 vs 32
Group6 - 26 vs 5
Group7 - 39 vs 27
Group8 - 8 vs 28
Round of 16 - 3 vs 10
Quarters Final - 14 vs 47

We circled the robot in the following videos that has a similar behaviour to the demo robot.

Videos: https://drive.google.com/drive/folders/1U8ct_VFgJf9OcKtF4b-7nOp2JHMR8Ewm?usp=sharing

Leo and Jerry
TFA 312

2 Likes

Hello, everyone.

It seems that the behavior reported by @thomastseng0380, @Leo and @Zvono points to the same issue. Thanks to all of you for reporting the problem, and special thanks to @Leo and Jerry for their excellent report with the videos!

After investigating the problem, we identified a bug in the script we used to automate the simulation process. Indeed, robot 2 from the Yellow team was pointing to the wrong code. In this particular case, because of the way we changed team sides, all teams played with Yellow in one half of each match, which means that there was some balance at the end.

We are going to invite all teams to a meeting in the next few weeks, in which we plan to explain how we ran the competition and collect feedback and suggestions from all of you.

I must point out that we cannot re-run matches now, as mentioned before. Identifying problems and solving them before the actual competition in June is one of the goals of this Demo Competition, and you are contributing a lot!

Thanks again for your collaboration and understanding!

Felipe.
On behalf of the RCJ Soccer TC and RCJ Soccer Simulation OC.

1 Like

Hi,

We found an interesting moment when the ball apparently left the field, and a few seconds later one of the teams got a goal scored. The screenshot shows the match and the time when this happened.

This is rather kind of funny, and probably earns just a low priority as a very rare issue. Maybe RCJSoccerReferee.check_progress() could check if the ball’s coordinates are out of the field, and if so, just mark it as lack of progress for the ball (before the ball reach check_goal() 's GOAL_X_LIMIT thresholds). To make the referee code absolute fool safe, the same check could run for all robots as well, because if any of the robots would somehow get out of the field, it would not have the chance to get back unless for lack of progress.

Best regards,
Robert

1 Like

Thanks @rlaszlo for reminding us of this, I have created an issue (https://github.com/RoboCupJuniorTC/rcj-soccer-sim/issues/77) on the GitHub repository, most likely solution is to reset the ball as you suggested but maybe also to increase the tick rate so the ball cannot pass through the wall in one tick and thus avoid collision detection.
Best Regards,
David

1 Like

@thomastseng0380 you were affected by the bug mentioned below, my apologies for looking in the wrong place first when investigating.
Best Regards,
David

Hello all,

Firstly, we understand that this is a demo competition, and that there is only so much time the TC/OC has allocated for this demo.

However, that does not nullify the time and effort spent by teams preparing for this competition. Since the rules/game are likely to change substantially for the non-demo, it’d be nice to get to see our robots playing real matches in which all robots are functional.

Considering how differently teams approached strategy, it affected all teams in a different way. A team that statically assigned a goalkeeper role to the Y2 robot would be in a significant disadvantage. Strategies (like the one that we built) that dynamically switch robots are essentially punished, since they rely on the fact that every robot is doing its job, otherwise certain positions are not covered, and all robots are severely handicapped in their decision making by assuming that all robots are functional, while in reality one is not.

While its is our opinion that the finals should be re-matched (the results in the group phase were pretty significant in margin, this would also be a lower workload for the TC/OC). We understand that the TC/OC does not wish to rematch games, and respect that.

Once again, thank you for bringing robocup to us, the students, despite the crisis,

GJH_team

1 Like

@Zvono We will release all of the submissions so you can easily download them and run your code against any team on your machine. Stay tuned! :slight_smile: