simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (4326)

simpleqa-1088
uvx harbor run -d simpleqa@1.0 -t simpleqa-1088
b8f0fa9
simpleqa-1089
uvx harbor run -d simpleqa@1.0 -t simpleqa-1089
b8f0fa9
simpleqa-109
uvx harbor run -d simpleqa@1.0 -t simpleqa-109
b8f0fa9
simpleqa-1090
uvx harbor run -d simpleqa@1.0 -t simpleqa-1090
b8f0fa9
simpleqa-1091
uvx harbor run -d simpleqa@1.0 -t simpleqa-1091
b8f0fa9
simpleqa-1092
uvx harbor run -d simpleqa@1.0 -t simpleqa-1092
b8f0fa9
simpleqa-1093
uvx harbor run -d simpleqa@1.0 -t simpleqa-1093
b8f0fa9
simpleqa-1094
uvx harbor run -d simpleqa@1.0 -t simpleqa-1094
b8f0fa9
simpleqa-1095
uvx harbor run -d simpleqa@1.0 -t simpleqa-1095
b8f0fa9
simpleqa-1096
uvx harbor run -d simpleqa@1.0 -t simpleqa-1096
b8f0fa9
simpleqa-1097
uvx harbor run -d simpleqa@1.0 -t simpleqa-1097
b8f0fa9
simpleqa-1098
uvx harbor run -d simpleqa@1.0 -t simpleqa-1098
b8f0fa9
simpleqa-1099
uvx harbor run -d simpleqa@1.0 -t simpleqa-1099
b8f0fa9
simpleqa-11
uvx harbor run -d simpleqa@1.0 -t simpleqa-11
b8f0fa9
simpleqa-110
uvx harbor run -d simpleqa@1.0 -t simpleqa-110
b8f0fa9
simpleqa-1100
uvx harbor run -d simpleqa@1.0 -t simpleqa-1100
b8f0fa9
simpleqa-1101
uvx harbor run -d simpleqa@1.0 -t simpleqa-1101
b8f0fa9
simpleqa-1102
uvx harbor run -d simpleqa@1.0 -t simpleqa-1102
b8f0fa9
simpleqa-1103
uvx harbor run -d simpleqa@1.0 -t simpleqa-1103
b8f0fa9
simpleqa-1104
uvx harbor run -d simpleqa@1.0 -t simpleqa-1104
b8f0fa9
simpleqa-1105
uvx harbor run -d simpleqa@1.0 -t simpleqa-1105
b8f0fa9
simpleqa-1106
uvx harbor run -d simpleqa@1.0 -t simpleqa-1106
b8f0fa9
simpleqa-1107
uvx harbor run -d simpleqa@1.0 -t simpleqa-1107
b8f0fa9
simpleqa-1108
uvx harbor run -d simpleqa@1.0 -t simpleqa-1108
b8f0fa9
simpleqa-1109
uvx harbor run -d simpleqa@1.0 -t simpleqa-1109
b8f0fa9
simpleqa-111
uvx harbor run -d simpleqa@1.0 -t simpleqa-111
b8f0fa9
simpleqa-1110
uvx harbor run -d simpleqa@1.0 -t simpleqa-1110
b8f0fa9
simpleqa-1111
uvx harbor run -d simpleqa@1.0 -t simpleqa-1111
b8f0fa9
simpleqa-1112
uvx harbor run -d simpleqa@1.0 -t simpleqa-1112
b8f0fa9
simpleqa-1113
uvx harbor run -d simpleqa@1.0 -t simpleqa-1113
b8f0fa9
simpleqa-1114
uvx harbor run -d simpleqa@1.0 -t simpleqa-1114
b8f0fa9
simpleqa-1115
uvx harbor run -d simpleqa@1.0 -t simpleqa-1115
b8f0fa9
simpleqa-1116
uvx harbor run -d simpleqa@1.0 -t simpleqa-1116
b8f0fa9
simpleqa-1117
uvx harbor run -d simpleqa@1.0 -t simpleqa-1117
b8f0fa9
simpleqa-1118
uvx harbor run -d simpleqa@1.0 -t simpleqa-1118
b8f0fa9
simpleqa-1119
uvx harbor run -d simpleqa@1.0 -t simpleqa-1119
b8f0fa9
simpleqa-112
uvx harbor run -d simpleqa@1.0 -t simpleqa-112
b8f0fa9
simpleqa-1120
uvx harbor run -d simpleqa@1.0 -t simpleqa-1120
b8f0fa9
simpleqa-1121
uvx harbor run -d simpleqa@1.0 -t simpleqa-1121
b8f0fa9
simpleqa-1122
uvx harbor run -d simpleqa@1.0 -t simpleqa-1122
b8f0fa9
simpleqa-1123
uvx harbor run -d simpleqa@1.0 -t simpleqa-1123
b8f0fa9
simpleqa-1124
uvx harbor run -d simpleqa@1.0 -t simpleqa-1124
b8f0fa9
simpleqa-1125
uvx harbor run -d simpleqa@1.0 -t simpleqa-1125
b8f0fa9
simpleqa-1126
uvx harbor run -d simpleqa@1.0 -t simpleqa-1126
b8f0fa9
simpleqa-1127
uvx harbor run -d simpleqa@1.0 -t simpleqa-1127
b8f0fa9
simpleqa-1128
uvx harbor run -d simpleqa@1.0 -t simpleqa-1128
b8f0fa9
simpleqa-1129
uvx harbor run -d simpleqa@1.0 -t simpleqa-1129
b8f0fa9
simpleqa-113
uvx harbor run -d simpleqa@1.0 -t simpleqa-113
b8f0fa9
simpleqa-1130
uvx harbor run -d simpleqa@1.0 -t simpleqa-1130
b8f0fa9
simpleqa-1131
uvx harbor run -d simpleqa@1.0 -t simpleqa-1131
b8f0fa9
simpleqa-1132
uvx harbor run -d simpleqa@1.0 -t simpleqa-1132
b8f0fa9
simpleqa-1133
uvx harbor run -d simpleqa@1.0 -t simpleqa-1133
b8f0fa9
simpleqa-1134
uvx harbor run -d simpleqa@1.0 -t simpleqa-1134
b8f0fa9
simpleqa-1135
uvx harbor run -d simpleqa@1.0 -t simpleqa-1135
b8f0fa9
simpleqa-1136
uvx harbor run -d simpleqa@1.0 -t simpleqa-1136
b8f0fa9
simpleqa-1137
uvx harbor run -d simpleqa@1.0 -t simpleqa-1137
b8f0fa9
simpleqa-1138
uvx harbor run -d simpleqa@1.0 -t simpleqa-1138
b8f0fa9
simpleqa-1139
uvx harbor run -d simpleqa@1.0 -t simpleqa-1139
b8f0fa9
simpleqa-114
uvx harbor run -d simpleqa@1.0 -t simpleqa-114
b8f0fa9
simpleqa-1140
uvx harbor run -d simpleqa@1.0 -t simpleqa-1140
b8f0fa9
simpleqa-1141
uvx harbor run -d simpleqa@1.0 -t simpleqa-1141
b8f0fa9
simpleqa-1142
uvx harbor run -d simpleqa@1.0 -t simpleqa-1142
b8f0fa9
simpleqa-1143
uvx harbor run -d simpleqa@1.0 -t simpleqa-1143
b8f0fa9
simpleqa-1144
uvx harbor run -d simpleqa@1.0 -t simpleqa-1144
b8f0fa9
simpleqa-1145
uvx harbor run -d simpleqa@1.0 -t simpleqa-1145
b8f0fa9
simpleqa-1146
uvx harbor run -d simpleqa@1.0 -t simpleqa-1146
b8f0fa9
simpleqa-1147
uvx harbor run -d simpleqa@1.0 -t simpleqa-1147
b8f0fa9
simpleqa-1148
uvx harbor run -d simpleqa@1.0 -t simpleqa-1148
b8f0fa9
simpleqa-1149
uvx harbor run -d simpleqa@1.0 -t simpleqa-1149
b8f0fa9
simpleqa-115
uvx harbor run -d simpleqa@1.0 -t simpleqa-115
b8f0fa9
simpleqa-1150
uvx harbor run -d simpleqa@1.0 -t simpleqa-1150
b8f0fa9
simpleqa-1151
uvx harbor run -d simpleqa@1.0 -t simpleqa-1151
b8f0fa9
simpleqa-1152
uvx harbor run -d simpleqa@1.0 -t simpleqa-1152
b8f0fa9
simpleqa-1153
uvx harbor run -d simpleqa@1.0 -t simpleqa-1153
b8f0fa9
simpleqa-1154
uvx harbor run -d simpleqa@1.0 -t simpleqa-1154
b8f0fa9
simpleqa-1155
uvx harbor run -d simpleqa@1.0 -t simpleqa-1155
b8f0fa9
simpleqa-1156
uvx harbor run -d simpleqa@1.0 -t simpleqa-1156
b8f0fa9
simpleqa-1157
uvx harbor run -d simpleqa@1.0 -t simpleqa-1157
b8f0fa9
simpleqa-1158
uvx harbor run -d simpleqa@1.0 -t simpleqa-1158
b8f0fa9
simpleqa-1159
uvx harbor run -d simpleqa@1.0 -t simpleqa-1159
b8f0fa9
simpleqa-116
uvx harbor run -d simpleqa@1.0 -t simpleqa-116
b8f0fa9
simpleqa-1160
uvx harbor run -d simpleqa@1.0 -t simpleqa-1160
b8f0fa9
simpleqa-1161
uvx harbor run -d simpleqa@1.0 -t simpleqa-1161
b8f0fa9
simpleqa-1162
uvx harbor run -d simpleqa@1.0 -t simpleqa-1162
b8f0fa9
simpleqa-1163
uvx harbor run -d simpleqa@1.0 -t simpleqa-1163
b8f0fa9
simpleqa-1164
uvx harbor run -d simpleqa@1.0 -t simpleqa-1164
b8f0fa9
simpleqa-1165
uvx harbor run -d simpleqa@1.0 -t simpleqa-1165
b8f0fa9
simpleqa-1166
uvx harbor run -d simpleqa@1.0 -t simpleqa-1166
b8f0fa9
simpleqa-1167
uvx harbor run -d simpleqa@1.0 -t simpleqa-1167
b8f0fa9
simpleqa-1168
uvx harbor run -d simpleqa@1.0 -t simpleqa-1168
b8f0fa9
simpleqa-1169
uvx harbor run -d simpleqa@1.0 -t simpleqa-1169
b8f0fa9
simpleqa-117
uvx harbor run -d simpleqa@1.0 -t simpleqa-117
b8f0fa9
simpleqa-1170
uvx harbor run -d simpleqa@1.0 -t simpleqa-1170
b8f0fa9
simpleqa-1171
uvx harbor run -d simpleqa@1.0 -t simpleqa-1171
b8f0fa9
simpleqa-1172
uvx harbor run -d simpleqa@1.0 -t simpleqa-1172
b8f0fa9
simpleqa-1173
uvx harbor run -d simpleqa@1.0 -t simpleqa-1173
b8f0fa9
simpleqa-1174
uvx harbor run -d simpleqa@1.0 -t simpleqa-1174
b8f0fa9
simpleqa-1175
uvx harbor run -d simpleqa@1.0 -t simpleqa-1175
b8f0fa9
simpleqa-1176
uvx harbor run -d simpleqa@1.0 -t simpleqa-1176
b8f0fa9
simpleqa-1177
uvx harbor run -d simpleqa@1.0 -t simpleqa-1177
b8f0fa9