simpleqa

v1.0

SimpleQA: 4,326 short, fact-seeking questions from OpenAI for evaluating language model factuality. Uses LLM-as-a-judge grading. Source: https://openai.com/index/introducing-simpleqa/

uvx harbor run -d simpleqa@1.0

Tasks (7652)

simpleqa-1150
uvx harbor run -d simpleqa@1.0 -t simpleqa-1150
b8f0fa9
simpleqa-1150
uvx harbor run -d simpleqa@1.0 -t simpleqa-1150
b8f0fa9
simpleqa-1151
uvx harbor run -d simpleqa@1.0 -t simpleqa-1151
b8f0fa9
simpleqa-1152
uvx harbor run -d simpleqa@1.0 -t simpleqa-1152
b8f0fa9
simpleqa-1152
uvx harbor run -d simpleqa@1.0 -t simpleqa-1152
b8f0fa9
simpleqa-1153
uvx harbor run -d simpleqa@1.0 -t simpleqa-1153
b8f0fa9
simpleqa-1153
uvx harbor run -d simpleqa@1.0 -t simpleqa-1153
b8f0fa9
simpleqa-1154
uvx harbor run -d simpleqa@1.0 -t simpleqa-1154
b8f0fa9
simpleqa-1154
uvx harbor run -d simpleqa@1.0 -t simpleqa-1154
b8f0fa9
simpleqa-1155
uvx harbor run -d simpleqa@1.0 -t simpleqa-1155
b8f0fa9
simpleqa-1155
uvx harbor run -d simpleqa@1.0 -t simpleqa-1155
b8f0fa9
simpleqa-1156
uvx harbor run -d simpleqa@1.0 -t simpleqa-1156
b8f0fa9
simpleqa-1156
uvx harbor run -d simpleqa@1.0 -t simpleqa-1156
b8f0fa9
simpleqa-1157
uvx harbor run -d simpleqa@1.0 -t simpleqa-1157
b8f0fa9
simpleqa-1157
uvx harbor run -d simpleqa@1.0 -t simpleqa-1157
b8f0fa9
simpleqa-1158
uvx harbor run -d simpleqa@1.0 -t simpleqa-1158
b8f0fa9
simpleqa-1158
uvx harbor run -d simpleqa@1.0 -t simpleqa-1158
b8f0fa9
simpleqa-1159
uvx harbor run -d simpleqa@1.0 -t simpleqa-1159
b8f0fa9
simpleqa-1159
uvx harbor run -d simpleqa@1.0 -t simpleqa-1159
b8f0fa9
simpleqa-116
uvx harbor run -d simpleqa@1.0 -t simpleqa-116
b8f0fa9
simpleqa-1160
uvx harbor run -d simpleqa@1.0 -t simpleqa-1160
b8f0fa9
simpleqa-1160
uvx harbor run -d simpleqa@1.0 -t simpleqa-1160
b8f0fa9
simpleqa-1161
uvx harbor run -d simpleqa@1.0 -t simpleqa-1161
b8f0fa9
simpleqa-1161
uvx harbor run -d simpleqa@1.0 -t simpleqa-1161
b8f0fa9
simpleqa-1162
uvx harbor run -d simpleqa@1.0 -t simpleqa-1162
b8f0fa9
simpleqa-1163
uvx harbor run -d simpleqa@1.0 -t simpleqa-1163
b8f0fa9
simpleqa-1163
uvx harbor run -d simpleqa@1.0 -t simpleqa-1163
b8f0fa9
simpleqa-1164
uvx harbor run -d simpleqa@1.0 -t simpleqa-1164
b8f0fa9
simpleqa-1164
uvx harbor run -d simpleqa@1.0 -t simpleqa-1164
b8f0fa9
simpleqa-1165
uvx harbor run -d simpleqa@1.0 -t simpleqa-1165
b8f0fa9
simpleqa-1165
uvx harbor run -d simpleqa@1.0 -t simpleqa-1165
b8f0fa9
simpleqa-1166
uvx harbor run -d simpleqa@1.0 -t simpleqa-1166
b8f0fa9
simpleqa-1166
uvx harbor run -d simpleqa@1.0 -t simpleqa-1166
b8f0fa9
simpleqa-1167
uvx harbor run -d simpleqa@1.0 -t simpleqa-1167
b8f0fa9
simpleqa-1168
uvx harbor run -d simpleqa@1.0 -t simpleqa-1168
b8f0fa9
simpleqa-1169
uvx harbor run -d simpleqa@1.0 -t simpleqa-1169
b8f0fa9
simpleqa-117
uvx harbor run -d simpleqa@1.0 -t simpleqa-117
b8f0fa9
simpleqa-117
uvx harbor run -d simpleqa@1.0 -t simpleqa-117
b8f0fa9
simpleqa-1170
uvx harbor run -d simpleqa@1.0 -t simpleqa-1170
b8f0fa9
simpleqa-1170
uvx harbor run -d simpleqa@1.0 -t simpleqa-1170
b8f0fa9
simpleqa-1171
uvx harbor run -d simpleqa@1.0 -t simpleqa-1171
b8f0fa9
simpleqa-1172
uvx harbor run -d simpleqa@1.0 -t simpleqa-1172
b8f0fa9
simpleqa-1172
uvx harbor run -d simpleqa@1.0 -t simpleqa-1172
b8f0fa9
simpleqa-1173
uvx harbor run -d simpleqa@1.0 -t simpleqa-1173
b8f0fa9
simpleqa-1173
uvx harbor run -d simpleqa@1.0 -t simpleqa-1173
b8f0fa9
simpleqa-1174
uvx harbor run -d simpleqa@1.0 -t simpleqa-1174
b8f0fa9
simpleqa-1174
uvx harbor run -d simpleqa@1.0 -t simpleqa-1174
b8f0fa9
simpleqa-1175
uvx harbor run -d simpleqa@1.0 -t simpleqa-1175
b8f0fa9
simpleqa-1175
uvx harbor run -d simpleqa@1.0 -t simpleqa-1175
b8f0fa9
simpleqa-1176
uvx harbor run -d simpleqa@1.0 -t simpleqa-1176
b8f0fa9
simpleqa-1176
uvx harbor run -d simpleqa@1.0 -t simpleqa-1176
b8f0fa9
simpleqa-1177
uvx harbor run -d simpleqa@1.0 -t simpleqa-1177
b8f0fa9
simpleqa-1177
uvx harbor run -d simpleqa@1.0 -t simpleqa-1177
b8f0fa9
simpleqa-1178
uvx harbor run -d simpleqa@1.0 -t simpleqa-1178
b8f0fa9
simpleqa-1178
uvx harbor run -d simpleqa@1.0 -t simpleqa-1178
b8f0fa9
simpleqa-1179
uvx harbor run -d simpleqa@1.0 -t simpleqa-1179
b8f0fa9
simpleqa-1179
uvx harbor run -d simpleqa@1.0 -t simpleqa-1179
b8f0fa9
simpleqa-118
uvx harbor run -d simpleqa@1.0 -t simpleqa-118
b8f0fa9
simpleqa-118
uvx harbor run -d simpleqa@1.0 -t simpleqa-118
b8f0fa9
simpleqa-1180
uvx harbor run -d simpleqa@1.0 -t simpleqa-1180
b8f0fa9
simpleqa-1180
uvx harbor run -d simpleqa@1.0 -t simpleqa-1180
b8f0fa9
simpleqa-1181
uvx harbor run -d simpleqa@1.0 -t simpleqa-1181
b8f0fa9
simpleqa-1181
uvx harbor run -d simpleqa@1.0 -t simpleqa-1181
b8f0fa9
simpleqa-1182
uvx harbor run -d simpleqa@1.0 -t simpleqa-1182
b8f0fa9
simpleqa-1182
uvx harbor run -d simpleqa@1.0 -t simpleqa-1182
b8f0fa9
simpleqa-1183
uvx harbor run -d simpleqa@1.0 -t simpleqa-1183
b8f0fa9
simpleqa-1184
uvx harbor run -d simpleqa@1.0 -t simpleqa-1184
b8f0fa9
simpleqa-1185
uvx harbor run -d simpleqa@1.0 -t simpleqa-1185
b8f0fa9
simpleqa-1185
uvx harbor run -d simpleqa@1.0 -t simpleqa-1185
b8f0fa9
simpleqa-1186
uvx harbor run -d simpleqa@1.0 -t simpleqa-1186
b8f0fa9
simpleqa-1186
uvx harbor run -d simpleqa@1.0 -t simpleqa-1186
b8f0fa9
simpleqa-1187
uvx harbor run -d simpleqa@1.0 -t simpleqa-1187
b8f0fa9
simpleqa-1187
uvx harbor run -d simpleqa@1.0 -t simpleqa-1187
b8f0fa9
simpleqa-1188
uvx harbor run -d simpleqa@1.0 -t simpleqa-1188
b8f0fa9
simpleqa-1189
uvx harbor run -d simpleqa@1.0 -t simpleqa-1189
b8f0fa9
simpleqa-119
uvx harbor run -d simpleqa@1.0 -t simpleqa-119
b8f0fa9
simpleqa-119
uvx harbor run -d simpleqa@1.0 -t simpleqa-119
b8f0fa9
simpleqa-1190
uvx harbor run -d simpleqa@1.0 -t simpleqa-1190
b8f0fa9
simpleqa-1190
uvx harbor run -d simpleqa@1.0 -t simpleqa-1190
b8f0fa9
simpleqa-1191
uvx harbor run -d simpleqa@1.0 -t simpleqa-1191
b8f0fa9
simpleqa-1192
uvx harbor run -d simpleqa@1.0 -t simpleqa-1192
b8f0fa9
simpleqa-1192
uvx harbor run -d simpleqa@1.0 -t simpleqa-1192
b8f0fa9
simpleqa-1193
uvx harbor run -d simpleqa@1.0 -t simpleqa-1193
b8f0fa9
simpleqa-1194
uvx harbor run -d simpleqa@1.0 -t simpleqa-1194
b8f0fa9
simpleqa-1194
uvx harbor run -d simpleqa@1.0 -t simpleqa-1194
b8f0fa9
simpleqa-1195
uvx harbor run -d simpleqa@1.0 -t simpleqa-1195
b8f0fa9
simpleqa-1195
uvx harbor run -d simpleqa@1.0 -t simpleqa-1195
b8f0fa9
simpleqa-1196
uvx harbor run -d simpleqa@1.0 -t simpleqa-1196
b8f0fa9
simpleqa-1196
uvx harbor run -d simpleqa@1.0 -t simpleqa-1196
b8f0fa9
simpleqa-1197
uvx harbor run -d simpleqa@1.0 -t simpleqa-1197
b8f0fa9
simpleqa-1197
uvx harbor run -d simpleqa@1.0 -t simpleqa-1197
b8f0fa9
simpleqa-1198
uvx harbor run -d simpleqa@1.0 -t simpleqa-1198
b8f0fa9
simpleqa-1198
uvx harbor run -d simpleqa@1.0 -t simpleqa-1198
b8f0fa9
simpleqa-1199
uvx harbor run -d simpleqa@1.0 -t simpleqa-1199
b8f0fa9
simpleqa-1199
uvx harbor run -d simpleqa@1.0 -t simpleqa-1199
b8f0fa9
simpleqa-12
uvx harbor run -d simpleqa@1.0 -t simpleqa-12
b8f0fa9
simpleqa-12
uvx harbor run -d simpleqa@1.0 -t simpleqa-12
b8f0fa9
simpleqa-120
uvx harbor run -d simpleqa@1.0 -t simpleqa-120
b8f0fa9
simpleqa-120
uvx harbor run -d simpleqa@1.0 -t simpleqa-120
b8f0fa9
simpleqa-1200
uvx harbor run -d simpleqa@1.0 -t simpleqa-1200
b8f0fa9