File tree Expand file tree Collapse file tree 1 file changed +1
-1
lines changed Expand file tree Collapse file tree 1 file changed +1
-1
lines changed Original file line number Diff line number Diff line change 3
3
We ran OpenAI's recently released [codex](https://arxiv.org/abs/2107.03374) API (davinci-codex model) on the problems
4
4
for 1,024 attempts per problem. Within each module below, we sort the problem by difficulty from hardest (unsolved)
5
5
to easiest for codex. We used a 5-shot prompt based on the Tutorial (see our [paper](https://arxiv.org/abs/2106.05784)).
6
- We report the success rate (fraction of the 10,000 generated solutions that satisfy the puzzle) and the shortest
6
+ We report the success rate (fraction of the 1,024 generated solutions that satisfy the puzzle) and the shortest
7
7
and longest valid solutions below.
8
8
This document summarizes the dataset stored in the `puzzles.json` file in this directory.
9
9
These files are generated from the `generators/*.py` files.
You can’t perform that action at this time.
0 commit comments