Skip to content

Commit 0f1a633

Browse files
authored
Update README.md
1 parent e36d607 commit 0f1a633

File tree

1 file changed

+21
-5
lines changed

1 file changed

+21
-5
lines changed

README.md

Lines changed: 21 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
11
# Python Programming Puzzles (P3)
22

33
This repo contains a dataset of Python programming puzzles which can be used to teach and evaluate
4-
an AI's programming proficiency. We present code generated by OpenAI's recently released
5-
[codex](https://arxiv.org/abs/2107.03374) 12-billion parameter neural network
4+
an AI's programming proficiency. We present code generated by OpenAI's
5+
[codex](https://arxiv.org/abs/2107.03374) neural network
66
solving many of these puzzles. We hope this dataset will
77
**grow rapidly**, and it is already diverse in terms of problem difficulty, domain,
88
and algorithmic tools needed to solve the problems. Please
99
[propose a new puzzle](../../issues/new?assignees=akalai&labels=New-puzzle&template=new-puzzle.md&title=New+puzzle)
1010
or [browse newly proposed puzzles](../../issues?q=is%3Aopen+is%3Aissue+label%3ANew-puzzle)
1111
or [contribute through pull requests](../../wiki/How-to-add-a-puzzle).
1212

13-
To learn more about how well AI systems such as GPT-3 can solve these problems, read our paper:
13+
To learn more about how well AI systems such as GPT-3 can solve these problems, read our two papers:
1414

1515
[Programming Puzzles](https://arxiv.org/abs/2106.05784). Tal Schuster, Ashwin Kalyan, Oleksandr Polozov,
1616
Adam Tauman Kalai. In *Proceedings of the Thirty-fifth Conference on Neural Information Processing Systems Datasets
@@ -22,11 +22,27 @@ title={Programming Puzzles},
2222
author={Tal Schuster and Ashwin Kalyan and Alex Polozov and Adam Tauman Kalai},
2323
booktitle={Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
2424
year={2021},
25-
url={https://openreview.net/forum?id=fe_hCc4RBrg}
25+
url={https://arxiv.org/abs/2106.05784}
2626
}
2727
```
2828

29-
To reproduce the results in the paper, see the [solvers](/solvers) folder.
29+
To reproduce the results in that paper, see the [solvers](/solvers) folder.
30+
31+
**NEW self-teaching:** In our second paper, we have Language Models (LMs) **generate their own puzzles** and, together with the Python interpreter, improve their own puzzle solving capability. Following our paper (arXiv, 2022), there have been several papers where an LM improves itself by checking its own solutions. However, our approach is potentially more powerful because we have the LM generate its own problems, not only its own solutions.
32+
33+
[Language Models Can Teach Themselves to Program Better](https://arxiv.org/abs/2207.14502). Patrick Haluptzok, Matthew Bowers, Adam Tauman Kalai. In *Proceedings of the Eleventh International Conference on Learning Representations* (ICLR), 2023.
34+
```
35+
@inproceedings{
36+
haluptzok2022selfteach,
37+
title={Language Models Can Teach Themselves to Program Better},
38+
author={Patrick Haluptzok, Matthew Bowers, Adam Tauman Kalai},
39+
booktitle={Eleventh International Conference on Learning Representations (ICLR)},
40+
year={2023},
41+
url=https://arxiv.org/abs/2207.14502}
42+
}
43+
```
44+
45+
To reproduce the results in that paper, see the [ICLR2023](/ICLR2023) folder.
3046

3147
If you just want to dive right into solving a few puzzles, try the [intro notebook at Binder](https://aka.ms/python_puzzles)
3248
that shows which puzzles the AI baselines solved and which they did not, so you can see how

0 commit comments

Comments
 (0)