RETECO · SemEval-2027 Task 1
The task is grounded in two published, operational benchmarks. Read the source papers, inspect the accepted proposal, and reuse verified citation records.
SemEval-2027
The proposal defines two complementary tracks—temporally grounded retrieval and reasoning-intensive conversational retrieval—with five sub-tracks, held-out evaluation, operational baselines, and a unified retrieval-ranking protocol.
Track 01 foundation
TEMPO introduces 1,730 complex temporal queries over 1,654,055 documents in 13 domains, 3,976 decomposed retrieval steps, and temporal completeness metrics.
@article{abdallah2026tempo,
title={TEMPO: A Realistic Multi-Domain Benchmark for Temporal Reasoning-Intensive Retrieval},
author={Abdallah, Abdelrahman and Ali, Mohammed and Abdul-Mageed, Muhammad and Jatowt, Adam},
journal={arXiv preprint arXiv:2601.09523},
year={2026},
url={https://arxiv.org/abs/2601.09523}
}Track 02 foundation
RECOR contains 707 conversations and 2,971 turns across 11 domains, combining discourse dependencies with reasoning-dependent passage relevance.
@inproceedings{ali2026recor,
title={RECOR: Reasoning-focused Multi-turn Conversational Retrieval Benchmark},
author={Ali, Mohammed and Abdallah, Abdelrahman and Agarwal, Amit and Patel, Hitesh Laxmichand and Jatowt, Adam},
booktitle={Findings of the Association for Computational Linguistics: ACL 2026},
pages={2688--2723},
year={2026},
publisher={Association for Computational Linguistics},
doi={10.18653/v1/2026.findings-acl.129},
url={https://aclanthology.org/2026.findings-acl.129/}
}Citation status
Until the SemEval-2027 task-description paper is published, cite the TEMPO and/or RECOR paper corresponding to the resources you use. The official RETECO citation will be added here after publication; no provisional task-paper metadata is invented.