Possible extension

Have travis check whether a commit has more versions than master, and if so, try to build every one of them. It may take a while but it will prove that new additions work.

On the testing point, you can test all new or changed versions compared to master with

set -e
set -x

docker build -t test-container .

git diff --name-only master \
  | grep '^plugins/python-build/share/python-build/' \
  | awk -F '/' '{print $NF}' \
  | xargs -I _ docker run test-container pyenv install _

Would be great to get a review on this! @yyuu ? I found it invaluable for testing new miniconda builds when I added a bunch a while back. The above bash script could potentially go it into CI for new build scripts.

I'd love to get this reviewed - @joshfriend @chrahunt , maybe? Not really sure how to move this forward.

When I was working on bulk-contributing new versions previously, nobody was able to answer how to test that they work "en masse". I still don't know how people are testing newly submitted versions. This represents a drastic improvement whatever that testing workflow is, and doesn't seem to add much maintenance overhead.