Home/submission

How to Submit

Predictions go to Codabench. Code of your final submission goes to the organisers for reproduction.

Open CodabenchBaseline & format checker
  1. Step 1

    Register

    Fill in the form with all members and the corresponding member's Codabench username.

  2. Step 2

    Predict

    Upload a zipped predictions file on Codabench to participate in the leaderboard. Up to 3 submissions per day in the test phase.

  3. Step 3

    Leaderboard

    Make sure your final system is on the leaderboard by the deadline. Otherwise your last successful submission counts.

  4. Step 4

    Submit Code

    Submit a code package that reproduces your final submission end-to-end, plus your system description report.

Predictions file

One JSON object per line, zipped as predictions.zip containing predictions.jsonl:

{"id": "test_0001", "level": "easy", "answer": "B"}
{"id": "test_0002", "level": "easy", "answer": "D"}