数据集卡片："cardiffnlp/relentless"

RelEntLess是一个新的基准测试，其中实体对根据它们满足给定分级关系的程度进行排序。本质上，这是一个排序任务，我们为每个关系提供了五个典型示例。以下是我们基线中对每种关系类型的简要描述，以及典型示例。更多详细信息请参阅我们的论文《 A RelEntLess Benchmark for Modelling Graded Relations between Named Entities 》。

{
    "friend/ally of": "entities that are friends or allies",
    "competitor/rival of": "entities that are competitors or rivals",
    "known for": "examples of what entities are known for",
    "influenced by": "what has influenced different entities",
    "similar to": "examples of entities that are similar"
}

数据集摘要

数据集结构

数据实例

{
    "pairs": [["Le Corbusier", "purism art"], ["Sean Connery", "Finding Forrester"], ...],
    "scores_all": [[4.0, 5.0, 3.0, 4.0, 5.0, 3.0, 5.0], [4.0, 5.0, 2, 5.0, 5.0, 4.0, 2], ...],
    "scores_mean": [4.142857142857143, 3.857142857142857, 4.857142857142857, ...],
    "relation_type": "known for",
    "ranks": [8.5, 11, 5, 14, 15, 5, 20, 13, 1.5, 18, 10, 1.5, 17, ...],
    "prototypical_examples": [ [ "Russell Crowe", "Gladiator" ], [ "Cadbury", "chocolate" ],...]
}

引用信息

@misc{ushio2023relentless,
      title={A RelEntLess Benchmark for Modelling Graded Relations between Named Entities}, 
      author={Asahi Ushio and Jose Camacho Collados and Steven Schockaert},
      year={2023},
      eprint={2305.15002},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}