RETECO · SemEval-2027 Task 1

Sample data

Real records copied from the official TEMPO and RECOR releases, packaged with every referenced positive document and TREC-style qrels.

One package

Download the curated sample.

RETECO curated sample data

5 TEMPO examples + 4 RECOR conversations, supporting passages, qrels, README, and a provenance manifest.

Download ZIP · 64 KB

Track 01 sample

TEMPO temporal retrieval

The curated subset contains five queries from Bitcoin, Economics, Genealogy, History, and History of Science and Mathematics, plus all 26 distinct gold passages they reference.

Temporal queryid · 122291_22

Why was there a jump in the number of transactions per block—from roughly 2,000 to 3,000—in April–May 2023?

The query requires evidence explaining a time-localized change rather than documents that merely mention Bitcoin transactions or the year 2023.

domain · bitcoingold passages · 5reasoning · event analysis
FileRecordsRoleOpen
examples.jsonl5Complete temporal queriesView
steps.jsonl5Matching step recordsView
documents.jsonl26All referenced gold passagesView
qrels.txt26Positive relevance judgmentsView

Track 02 sample

RECOR conversational retrieval

The subset contains four complete conversations from Law, Drones, Biology, and Earth Science: 13 target turns, 15 distinct supporting passages, and 32 positive turn-document judgments.

Three-turn conversationid · ex_111149
Turn 1

In copyright law, how is extending a term different from bringing a work back under copyright after it entered the public domain?

Turn 2 · continuation

Wait, did any works actually get their copyright back after they had already gone public domain?

Turn 3 · contextual dependency

Did those older movies ever get their rights back, or were they just out for good?

domain · lawturns · 3requires full history
FileRecordsRoleOpen
benchmark.jsonl4 conversations / 13 turnsFull conversationsView
documents.jsonl15All referenced passagesView
qrels.txt32Positive turn-document judgmentsView

Important scope

A format demonstration—not a new split.

  • Every record is copied from an official public release; no labels, negatives, questions, or answers were synthesized for this package.
  • The selection is intentionally diverse and inspectable, not statistically representative or domain-balanced.
  • The included documents are only the positive evidence subset. Baselines must run against the corresponding full corpus.
  • This package is not the hidden SemEval-2027 test set.
Do not benchmark on the sample corpus aloneBecause the package contains only referenced positive evidence, retrieval scores computed over it would be invalid and artificially easy.