At the start of the evaluation period (1 March) the test pairs will be made available. Participants will need to be registered in the submission system. Once the test pairs are downloaded, participants will have 5 days (120 hours) to upload the results of their systems. In other words, participants choose when to download the test pairs, but they have a 5 day window to upload the results. The system will cease to accept results on the 15th of March - 23:59 (UTC-11).
The core test dataset will contain pairs of sentences from the collections specified in the trial data (ca. 750 pairs from each, except the FNWN and OnWN datasets, with ca. 375 pairs each).
The typed-similarity dataset will contain pairs of records as in the trial data (ca. 750 pairs).
The system will accept three different runs per participant and task.
Further instructions will be sent to each registered participant.
