Web-based External Binary Helper

This would effectively fix Bug:1453

Currently, when you make a change in external or internal systems, you need to log in to the builder machines with builder account, and manually build newer versions of the binaries, then upload them. Only after that step can you do changes to chandler/Makefile, which will affect the builds most people use. This is obviously bad because of manual steps and direct access to builder machines and accounts.

1. I propose that we augment the full build tinderclients so that they will automatically upload all their (changed?) binaries into the builds server into a staging area.
2. Create a new web-based tool where you can login and say you want to copy binary tarballs to the directory from which the make install builds can find these binaries.

This simplifies the process so that anyone with checkin privileges can have the full builds automatically generate new binaries tarballs and upload them to the staging area. They can do make install testing by changing their chandler/Makefile to download from the staging area (we could also make the makefile check an env variable). After they have done make install testing, they can login to the web-tool and make it copy the files for them.

Design considerations

• make upload should also upload an indicator file when all is done.
• Tinderbox fullbuild clients should do 'make upload UPLOAD=stagingurl'
  • Dir layout as follows:

         staging/
           windows/
             timestamp1/
             timestamp2/
             ...
           linux/
             timestamp1/
             timestamp2/
             ...
           macosx/
             timestamp1/
             timestamp2/
             ...

• The webtool should only allow copying to real download area, not overwrite, so to protect old builds (so it isn't a disaster if some hacker gets access to a developers login to the webtool). If overwrite was necessary, that would require manually doing the overwrites.
• The webtool should give the user choices of binaries to copy:
  • Under each platform, find the latest completed timestamp directory
  • List only files that appear for all platforms in both debug and release format, but filter out files that already appear in the real downloads area
• The webtool should automatically prune directories that are no longer relevant.
• The webtool should log all actions: user, timestamp, files copied, directories deleted

Webtool UI

The following packages are ready to copy: | Sorry, there are no packages to copy because ...

m2crypto-*-0.12-5.tar.gz (Timestamps: windows 20041206120000, macosx 20041206120005, linux 20041206114500)
wxPython-*-2.6-1.tar.gz (Timestamps: windows 20041205120300, macosx 20041206120023, linux 20041206114501)

You need to specify username and password to copy.

Username: [    ]
Password: [    ]

[Copy selected packages] [Reset]

-- HeikkiToivonen - 08 Nov 2004