Q1: What are the main functions of the submission server?
A: Our submission server mainly serves for the following 4 purposes:
1. Register this challenge;
2. Submit validation results and compete with other registered teams in a non-public leaderboard;
3. Submit test results and peek the scores computed over a confidential subset of the test set;
4. Mark one of the submitted test results for final evaluation. After the submission deadline,
the final evaluation will be conducted over the full test set and the final
scores will be released in a public leaderboard at the challenge's main page.
Q2: When will the submission server open for registration?
A: On 8 March.
Q3: How to register this challenge?
A: To register, each team has to join the main task.
To join a task, team leader shall create an account and create an team for the task.
Team members shall create their accounts as well, and join the team created by team leader
by asking the unique team token from team leader.
Q4: When will the submission server fully open for submission?
A: On 27 April.
Q5: What is the submission flow?
A: There are generally 4 steps from submitting your result to getting your score:
1. Click "Create Submission" after entering the corresponding task in the submission server;
2. Select a valid file and fill in necessary information, and then click "Upload". Upon successful
upload, it will create a submission in the submission table with a status of "To be submitted".
Click "view" to check the details of this submission;
3. Click "Submit" to command the server to record and evaluate this submission, and the status of this submission
will change to "Evaluating";
4. It might take a long time for the server to complete the evaluation when there are many
submissions. After completion, the status of this submission will change to "Succeed" and you can
click "view" to see the score and some other evaluation details.
Q6: Is there any limit on the number of submissions?
A: Yes, for each team, there is a submission limit of 6 per day for validation
submissions and 3 per day for test submissions.
Q6: Why do I see the message: your team has reached the upload limit?
A: There can only allow 1 submission whose status is "To be submitted" in the submission
table. Please proceed to submit or discard that submission before making another sumbmission.
Q7: Why do I get an error in my score?
A: It is very likely that your submitted file does not meet the required format.
You can click "view" to see the detailed error message. Since the server uses the same evaluation
code as the released,
you can debug based on this released code before making further submission.
Q8: When and where can I see my team's final score?
A: You can find it at the challenge's main page after July 1,
subject to that your team has responsibly selected one of your test results in the submission server
for the final evaluation.
Q9: Why is the final score different from the one seen in the submission server?
A: This is because the submission server computes the scores for test submissions
over a confidential subset of the test set, in order to avoid parameter tuning against the test set.
However, the final score is computed over the full test set offline.