A Dataset of Python Challenges for AI Research
Python Programming Puzzles (P3)
This repo contains a dataset of python programming puzzles which can be used to teach and evaluate an AI’s programming proficiency. We hope this dataset with grow rapidly, and it is already diverse in terms of problem difficult, domain, and algorithmic tools needed to solve the problems. Please propose a new puzzle or browse newly proposed puzzles or contribute through pull requests.
To learn more about how well AI systems such as GPT-3 can solve these problems, read our paper:
Programming Puzzles. Tal Schuster, Ashwin Kalyan, Oleksandr Polozov, Adam Tauman Kalai.
Adam Tauman Kalai.
@misc{schuster2021programming,
title={Programming Puzzles},
author={Tal Schuster and Ashwin Kalyan and Oleksandr Polozov and Adam Tauman Kalai},
year={2021},
eprint={2106.05784},
archivePrefix={arXiv},
}
To reproduce the results in the paper, see the solvers