Skip to content

Latest commit

 

History

History
63 lines (63 loc) · 2.33 KB

2021-07-01-alet21a.md

File metadata and controls

63 lines (63 loc) · 2.33 KB
title abstract layout series publisher issn id month tex_title firstpage lastpage page order cycles bibtex_author author date address container-title volume genre issued pdf extras
A large-scale benchmark for few-shot program induction and synthesis
A landmark challenge for AI is to learn flexible, powerful representations from small numbers of examples. On an important class of tasks, hypotheses in the form of programs provide extreme generalization capabilities from surprisingly few examples. However, whereas large natural few-shot learning image benchmarks have spurred progress in meta-learning for deep networks, there is no comparably big, natural program-synthesis dataset that can play a similar role. This is because, whereas images are relatively easy to label from internet meta-data or annotated by non-experts, generating meaningful input-output examples for program induction has proven hard to scale. In this work, we propose a new way of leveraging unit tests and natural inputs for small programs as meaningful input-output examples for each sub-program of the overall program. This allows us to create a large-scale naturalistic few-shot program-induction benchmark and propose new challenges in this domain. The evaluation of multiple program induction and synthesis algorithms points to shortcomings of current methods and suggests multiple avenues for future work.
inproceedings
Proceedings of Machine Learning Research
PMLR
2640-3498
alet21a
0
A large-scale benchmark for few-shot program induction and synthesis
175
186
175-186
175
false
Alet, Ferran and Lopez-Contreras, Javier and Koppel, James and Nye, Maxwell and Solar-Lezama, Armando and Lozano-Perez, Tomas and Kaelbling, Leslie and Tenenbaum, Joshua
given family
Ferran
Alet
given family
Javier
Lopez-Contreras
given family
James
Koppel
given family
Maxwell
Nye
given family
Armando
Solar-Lezama
given family
Tomas
Lozano-Perez
given family
Leslie
Kaelbling
given family
Joshua
Tenenbaum
2021-07-01
Proceedings of the 38th International Conference on Machine Learning
139
inproceedings
date-parts
2021
7
1