My experience is mainly with soccer and other competitions but I had to chime in here as I see the use of AI and removal of the engineering journal requirement as critical issues for RCJ in general. As an advocate for and organizer of RCJ in the US, I’m not sure I could recommend anyone start a rescue team with these two changes together.
This year for Eindhoven, I evaluated around 50 soccer documents and wrote specific feedback for each one in a reasonable amount of time before the competition. I had a simple rubric with essentially yes/no questions and a google form setup so that it would quickly record everything. Given that I haven’t heard any complaints, it seems that teams appreciated the new approach that we took to judging soccer this last season; I know it was significantly easier for the judging team. I’m confident that I could do the same document review again this season even without the accidental 10 page restriction (i.e. if teams submitted raw documentation).
Having sat in many judging rooms at all levels for several types of competitions for the last 15 years, I very much dislike giving point values to anything other than what is objectively measurable based on the performance of the robot(s). Almost always there are debates over a point or two (sometimes fractions) that caused one team to win over the other that made everyone leaving the room slightly annoyed/dissatisfied/concerned with results; supposedly an indicator of a good negotiation is everyone leaving dissatisfied but it isn’t a great feeling when you are giving out awards. Many times it came down to which judge was the better advocate for the team rather than who was the best team.
The main goal of RCJ seems to be for kids to learn how to build robots that perform, so why score them on anything else? If you only require but not score documentation for its intended purposes, to hold teams accountable and share what they’ve done for other teams to learn from, then it will circumvent many if not all the issues you were experiencing; you just need to see if the documentation reaches a threshold of integrity and then impose a penalty on the performance score if it does not (after some investigation to ensure there wasn’t a miscommunication). The idea is to make the competition very similar to F1 where everyone sees who won the race but there may be penalties for cutting corners applied after the results.
Teaching courses that require all of my students to document, I get the struggle. I also interview each of my students and give a practical examination when it comes to giving a final grade. It is a pain and takes a lot of time, but it is way more genuine; students can’t get away with using AI or internet searches to do the work for them (those that try quickly find out it isn’t going to work!). This also gives an opportunity for students to explore genuine applications for AI (although this is relatively a new issue so I haven’t seen it done yet).
For those saying documenting is hard, documenting is easier than it ever has been - apps like Keep Notes allow kids to easily take a picture and type out some notes on their phones while working. Allowing AI while at the same time taking out the requirement to provide evidence of genuine work done by the team is going to allow for a serious amount of cheating, goes against what is practiced in education, may go against what is done in industry, and doesn’t align with what makes RCJ stand out from other competitions (i.e. the sharing of information after the event).
Forgetting the accountability and sharing reasons, documentation is a must for students to have a proper accounting of their achievements. I can’t tell you how much I would appreciate having documentation on all the projects I’ve worked on in the past; I’ve needed to remember things I’ve set up years ago, I wanted to prepare for a job interview, or I just wanted to reminisce. As we discovered in soccer this year, a 10 page TDP is not enough for many teams.
The problems you experienced that inspired these changes are understandable and these solutions are rational given the context of everything must have a score. However, please consider evaluating whether or not you should be taking a new approach and perhaps you’ll find better solutions. If you’d like, hit me up on Slack and I’ll happily arrange a time to discuss further and I’ll hopefully be able to volunteer to help out with evaluating documents for any RCJ league for teams going to Brazil if needed.
Thanks,
Mike Ambrose
RCJ USA Regional Rep/Soccer Chair