Frequently Asked Questions
Can't find your answer? Ask on the forum so every team sees the reply.
Who can participate?
Anyone — students, researchers and practitioners, from any institution or country, including IIT and UCSC students. The only exclusion is people who have had access to the full dataset or helped create the test set. Prizes require in-person presentation in Colombo.
Is there a registration fee?
No fee is charged to participate in the shared task. Any details about conference attendance for presenters will be announced.
Can I be on two teams?
No. Each person may belong to only one team. Duplicate registrations can lead to disqualification of all teams involved.
How is the 8B limit counted?
Add up the parameters of every model used at inference: generators, ensemble members, retrievers, rerankers and embedding models. MoE models count total parameters. Quantised models count their original parameters. Non-neural components such as BM25 count as zero.
Does a LoRA adapter count towards the limit?
Yes — count the base model plus the adapter parameters.
Can I use a model released on 1 October 2026?
No. Only open-weight models released before 1 October 2026 are allowed.
Can I use GPT, Claude or Gemini?
Only to generate training data, and you must disclose it in your manifest and report. They may never be used at inference.
Can my system use retrieval?
Yes, over a local, offline corpus built from permitted data — with the retriever and any embedding or reranking models counted towards the 8B limit. No internet access or live web search at test time.
How many submissions can we make?
Up to 3 per day on Codabench. For the final ranking you select up to 2 runs; otherwise your last successful submission counts.
Why is my public leaderboard score different from my final score?
The public leaderboard uses about 30% of the test set; final rankings use the remaining hidden portion. Scores on a small subset naturally vary, so avoid tuning to the public leaderboard.
What hardware will organisers use to reproduce runs?
The target hardware and time limit are specified in the full rules document. Make sure your pipeline runs within them.
What if our corresponding member becomes unreachable?
Your backup contact, supported by a majority of registered team members, can email the organisers from registered addresses to transfer the role.
Are late submissions accepted?
No. Extensions are only considered for a confirmed platform outage and are announced to all teams.
Can we present remotely?
No. Presentations are in person at UCSC. Teams that cannot attend still appear on the final leaderboard but are not eligible for prizes.
No matching questions. Try another word, or ask on the forum.