You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
solving many of these puzzles. We hope this dataset will
7
7
**grow rapidly**, and it is already diverse in terms of problem difficulty, domain,
8
8
and algorithmic tools needed to solve the problems. Please
9
9
[propose a new puzzle](../../issues/new?assignees=akalai&labels=New-puzzle&template=new-puzzle.md&title=New+puzzle)
10
10
or [browse newly proposed puzzles](../../issues?q=is%3Aopen+is%3Aissue+label%3ANew-puzzle)
11
11
or [contribute through pull requests](../../wiki/How-to-add-a-puzzle).
12
12
13
-
To learn more about how well AI systems such as GPT-3 can solve these problems, read our paper:
13
+
To learn more about how well AI systems such as GPT-3 can solve these problems, read our two papers:
14
14
15
15
[Programming Puzzles](https://arxiv.org/abs/2106.05784). Tal Schuster, Ashwin Kalyan, Oleksandr Polozov,
16
16
Adam Tauman Kalai. In *Proceedings of the Thirty-fifth Conference on Neural Information Processing Systems Datasets
@@ -22,11 +22,27 @@ title={Programming Puzzles},
22
22
author={Tal Schuster and Ashwin Kalyan and Alex Polozov and Adam Tauman Kalai},
23
23
booktitle={Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
24
24
year={2021},
25
-
url={https://openreview.net/forum?id=fe_hCc4RBrg}
25
+
url={https://arxiv.org/abs/2106.05784}
26
26
}
27
27
```
28
28
29
-
To reproduce the results in the paper, see the [solvers](/solvers) folder.
29
+
To reproduce the results in that paper, see the [solvers](/solvers) folder.
30
+
31
+
**NEW self-teaching:** In our second paper, we have Language Models (LMs) **generate their own puzzles** and, together with the Python interpreter, improve their own puzzle solving capability. Following our paper (arXiv, 2022), there have been several papers where an LM improves itself by checking its own solutions. However, our approach is potentially more powerful because we have the LM generate its own problems, not only its own solutions.
32
+
33
+
[Language Models Can Teach Themselves to Program Better](https://arxiv.org/abs/2207.14502). Patrick Haluptzok, Matthew Bowers, Adam Tauman Kalai. In *Proceedings of the Eleventh International Conference on Learning Representations* (ICLR), 2023.
34
+
```
35
+
@inproceedings{
36
+
haluptzok2022selfteach,
37
+
title={Language Models Can Teach Themselves to Program Better},
38
+
author={Patrick Haluptzok, Matthew Bowers, Adam Tauman Kalai},
39
+
booktitle={Eleventh International Conference on Learning Representations (ICLR)},
40
+
year={2023},
41
+
url=https://arxiv.org/abs/2207.14502}
42
+
}
43
+
```
44
+
45
+
To reproduce the results in that paper, see the [ICLR2023](/ICLR2023) folder.
30
46
31
47
If you just want to dive right into solving a few puzzles, try the [intro notebook at Binder](https://aka.ms/python_puzzles)
32
48
that shows which puzzles the AI baselines solved and which they did not, so you can see how
0 commit comments