Skip to content

Commit 8bb489c

Browse files
JonnodTVictorSanh
andauthored
add templates for riddle_sense dataset (#723)
* add templates for riddle_sense dataset Use almost identical templates as commonsense_qa because the two datasets are very similar. * moving `{% if answerKey != "" %}` to the target Co-authored-by: Victor Sanh <victorsanh@gmail.com>
1 parent 3bb12cf commit 8bb489c

File tree

1 file changed

+116
-0
lines changed

1 file changed

+116
-0
lines changed
Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
dataset: riddle_sense
2+
templates:
3+
3df2405b-4a0e-4fcf-8600-b7f4843b945a: !Template
4+
answer_choices: null
5+
id: 3df2405b-4a0e-4fcf-8600-b7f4843b945a
6+
jinja: 'Use the following options to predict a possible question for them:
7+
8+
9+
{% for i in range(choices[''text'']|length) %}
10+
11+
- {{choices[''text''][i]}}
12+
13+
{% endfor %} |||
14+
15+
{{question}}'
16+
metadata: !TemplateMetadata
17+
choices_in_prompt: false
18+
metrics:
19+
- BLEU
20+
- ROUGE
21+
original_task: false
22+
name: answer_to_question
23+
reference: ''
24+
8b44338c-f635-47fc-86fb-3135ef2a76ae: !Template
25+
answer_choices: '{{choices.text | join("|||")}}'
26+
id: 8b44338c-f635-47fc-86fb-3135ef2a76ae
27+
jinja: 'Given the options below, select the most suitable answer for the following
28+
question:
29+
30+
{{question}}
31+
32+
Options:
33+
34+
- {{answer_choices | join("\n- ")}}|||
35+
36+
{% if answerKey != "" %}
37+
38+
{{answer_choices[choices["label"].index(answerKey)]}}
39+
40+
{% endif %}'
41+
metadata: !TemplateMetadata
42+
choices_in_prompt: true
43+
metrics:
44+
- Accuracy
45+
original_task: true
46+
name: most_suitable_answer
47+
reference: ''
48+
ac002734-5b1f-4478-9ffc-f16d9ca2f70e: !Template
49+
answer_choices: '{{choices.text | join("|||")}}'
50+
id: ac002734-5b1f-4478-9ffc-f16d9ca2f70e
51+
jinja: 'Answer the following question:
52+
53+
{{question}} |||
54+
55+
{% if answerKey != "" %}
56+
57+
{{ answer_choices[choices[''label''].index(answerKey)] }}
58+
59+
{% endif %}'
60+
metadata: !TemplateMetadata
61+
choices_in_prompt: false
62+
metrics:
63+
- Accuracy
64+
original_task: true
65+
name: answer_given_question_without_options
66+
reference: ''
67+
bf3f7bd2-91c6-455c-b9f7-42ce265fa2db: !Template
68+
answer_choices: A ||| B ||| C ||| D ||| E
69+
id: bf3f7bd2-91c6-455c-b9f7-42ce265fa2db
70+
jinja: 'Given the following options, what do you think is the correct answer to
71+
the question below:
72+
73+
74+
{{question}}
75+
76+
77+
Options:
78+
79+
{% for letter, t in zip(answer_choices, choices.text) %}
80+
81+
- {{letter}}: {{t}}
82+
83+
{% endfor %} |||
84+
85+
{% if answerKey != "" %}
86+
87+
{{answerKey}}
88+
89+
{% endif %}'
90+
metadata: !TemplateMetadata
91+
choices_in_prompt: true
92+
metrics:
93+
- Accuracy
94+
original_task: true
95+
name: question_to_answer_index
96+
reference: ''
97+
bf7d7cbc-aa05-4aca-97ff-29eb34502019: !Template
98+
answer_choices: '{{choices.text | join("|||")}}'
99+
id: bf7d7cbc-aa05-4aca-97ff-29eb34502019
100+
jinja: '{{question}}
101+
102+
103+
- {{answer_choices | join("\n- ")}} |||
104+
105+
{% if answerKey != "" %}
106+
107+
{{answer_choices[choices[''label''].index(answerKey)] }}
108+
109+
{% endif %}'
110+
metadata: !TemplateMetadata
111+
choices_in_prompt: true
112+
metrics:
113+
- Accuracy
114+
original_task: true
115+
name: question_answering
116+
reference: ''

0 commit comments

Comments
 (0)