[Development] Solutions for ensuring that changes in upstream modules are tested with downstream modules before merging

[Edward Welbourne]

Leaving aside the question of what to test when integrating changes to
each given module, I pause to consider how we could sensibly implement
the necessary information for telling automated systems to do that
testing.  That may place constraints on what we can do, that might bound
the discussion of what we want to do / what it makes sense to do.

Presumably we could add a new stanza to dependencies.yaml, a top level
peer of the dependencies stanza (which I'd rather call prerequisites,
but I guess it's too late for that), named dependants, with suitable
meta-data to say what version of it to test against and what to test -
build, test, example - in the dependant, as a prerequisite of accepting
a change in the upstream module in Coin.

PreCheck shall, naturally, test everything that integration shall test.
An option to consider is to have PreCheck perform a broader set of tests
of dependant modules - e.g. the new stanza could have a required:
option, as in the dependencies stanza, whose false means Coin doesn't
reject based on the dependant failing, where PreCheck maybe would.
Whether Coin runs those tests (reporting, but not failing on, any
failures) is a choice we can make based on practicalities like load.

The committer of the new version of the upstream module would then be
able to get changes to the downstream through integration, that prepare
it for the upstream change, and update the upstream commit that would
have broken the downstream's tests otherwise, to show the new sha1 of
the downstream module as the basis for the dependant-testing.

Relevant module owners can then haggle over how far upstream each module
gets protected by being identified as dependant.  That'll be a balancing
act between the upstream not wanting to be slowed too much in Coin by
the building and testing of dependants, vs the dependants wanting that
protection.  That haggling would surely happen in Gerrit changes to the
upstream's dependencies.yaml file, which seems like a suitable enough
forum in which to, at least, open such a discussion.

Which is all just a sketch, but hopefully gives some shape to what's
practically implementable.  Can anyone spot any obvious holes in this
approach ?  Or other constraints it'd impose on what we can do ?

	Eddy.