[Development] dev CI integration stuck for 3½ days

[Oswald Buddenhagen]

On Wed, May 18, 2016 at 08:08:46AM +0000, Simon Hausmann wrote:
> How it is possible that a new staging branch was created while another
> change remained in 'INTEGRATING' state is a mystery to me. Perhaps
> something happened in Gerrit at that moment? Perhaps there's a bug in
> the CI that allows for creating this "impossible" state?
> 
well, something weird definitely *did* happen with gerrit around that
time, as it's also when it started dropping notification mails.
whether/how that is related is unknown to me.

generally, i wouldn't trust that the staging-related commands do all
necessary sanity checks to prevent an amok-running CI from creating a
bogus state. this should be thoroughly re-examined next time we
forward-port the feature.