EliteTable paper reports self-improving program-search algorithm for control policies

EliteTable introduces a self-improving program-search algorithm that uses a fixed-capacity table of high-scoring solutions to guide new proposals, edits of existing solutions, and remixes of multiple solutions, all orchestrated by large language models.

The authors evaluated EliteTable across six Gymnasium control tasks, running ten generations per task with fifty candidate slots per generation and ten independently initialized searches per task.

Across all six tasks, candidate-score medians increased from the first to the final generation. Selected-winner search-score medians rose in five of the six tasks, with the sixth task showing a decline.

In the CartPole-v1 task, the selected winner reached the search-score ceiling in the first generation for all ten searches, indicating rapid convergence to optimal performance under the search protocol.

All ten paired held-out endpoints improved in four of the six tasks, while BipedalWalker-v3 and CarRacing-v3 improved without reaching their reporting targets. Six of ten Acrobot-v1 searches declined on held-out episodes, with a negative median paired change.

The paper documents improvement during program search and partial transfer of selected-winner gains to new episodes, but emphasizes that comparisons and ablations are required to determine EliteTable's relative performance and the contributions of its components.