GitLab

These will be now versioned at
https://heptapod.octobus.net/octobus-public/mercurial-mirror

--HG--
branch : beta

--HG--
branch : beta

Since 0.2.0 has a breaking change (name of refresh script),
we felt compelled to add proper changelogs

The primary goal is to avoid overstraining upstream servers.

This introduces a new '--jobs' argument, and hence argparse.
Took the occasion to stop hardcoding the repos list.

In case we decide to change an upstream URL, it is
surprising and inappropriate that the mirror continues
to use the original one recorded at the time of
original cloning.

Instead, mirror_repos.txt should provide a way to
switch easily upstream URLs

"refresh" is more indicative than "check" of what it does, e.g,
it is not a health check

This allows to ease the load on the real ustreams,
especially Heptapod.

Keeping bitbucket, so that we still test with ssh://
and some other strong upstreams or small repos in order
to get some diversity in upstream hg versions.

--HG--
branch : beta

--HG--
branch : beta

--HG--
branch : beta

--HG--
branch : beta

This will be useful for new versions of Mercurial, or to test new
options.

Eventually, we'll have to remove the subdirs for the public images.
For now, we can work on them from there

--HG--
branch : beta

If we fully clone a repository (without --noupdate) it will create lots of
files and directories, that hgweb will scan continuously, looking for
repositories to serve.

The openjdk people use an old server and an old format, it would be useful to
have a modern server in display.

Then I hope after bringing all self-hosted heptapod repos in
sync, we can declare the bitbucket heptapod repos obsolete
and gain a much needed clarity

We can now provide links to the other image on Docker Hub

Some configuration files have changed locations

Note about refreshing repos periodically.

Now that we rely on an image from Docker Hub, we need to ensure that
it gets refreshed in the process

This means that we don't need to disable StrictHostKeyChecking,
a new IP address gives just a warning in default SSH config if the
DNS entry already exists.

This means also that we have full protection against MITM attacks even
on a a fresh image

The previous chmod was to wide, as it included the mercurial user
itself.

Somehow manually building with the `latest` tag confuses Docker

This is more explicit. Change already taken into account in Dockerfile for
mirror.octobus.net

For consistency, this one should also be renamed private.hgrc
but we can do that later on

When using docker-compose, this has the consequence of having
the repos persist upon updates even if not tied to a volume by
the docker-compose.yml. Only an explicit docker rm would restart
the repos afresh.

For the same reason, and because we'd like for now the conf to be
amendable in a derived image, while using docker-compose,
there's no VOLUME directive for the conf/ subdirectory.

Side note: the ordering with the chown really matters.

It seems worthwile to me to make this easy to customize either in
dependent images or on the fly

The biggest ones stay commented, so that we can perform
somewhat quick testing

Includes consistent tags for upload to Docker Hub

This is done by removing binary build tools and headers
before first layer commit.

The tradeoffs are

- rebuild after change of requirements will be much longer,
  since the whole first step will then have to be redone.
- dependent images will be able to pip install pure python packages
  only, unless they themselves build dependencies.

But we get roughly from 480 MB down to 245 MB for mercurial-http-server,
it's worth it in my mind, especially considering we might add some
more build dependencies later (e.g, adding a Rust toolchain
makes it almost 890 MB if we don't remove in the same command).

- removed stuff to work on a local hg sources
- wait on all subprocesses to finish (can be important in Docker context
  that the parent process does not finish before its children).
- do full clones right away: it's simpler and probably nicer to upstream repos

(all this done while testing previous change on a repolist.txt with only
one repo)

Supporting comments was necessary to produce a sample file.
In the same move, we support blank lines, and strip whitespace
around the '=' sign.

Switching to INI (ConfigParser) or json would probably make sense
though but this is good enough for now.