[Releasing] Proposal to adjust release candidate process

[Giuseppe D'Angelo]

Il 21/12/2016 08:34, Maurice Kalinowski ha scritto:
>>>
>>> 1.       Process to make a RC that is as flawless as final causes
>>> inefficiency as we only get full test coverage with the RC itself
>>
>> Well, "Release Candidate" by definition means "(hopefully) suitable
>> *as-is* for final release"; as such, we must enter that phase only when we
>> have already reached a state as flawless as possible. It's sad that we get "test
>> coverage" with the RC. Do you mind to elaborate more, however? What
>> does "test coverage" mean here?
> 
>  [Kalinowski Maurice] 
> Test coverage implies active user manual testing outside the Qt Company. Whenever new packages are ready, R&D is asked to test all available packages. Those are actually more packages than the ones we send out to everyone asking to test.
> 
> However, the feedback we get from the community on package testing before the RC is almost zero, but then raises significantly when a package is claimed to be a Release Candidate and sometimes a Release Candidate Candidate. But as mentioned before, for many reported items, this is too late in the process.

Then, please, let's try to change _this_; I just don't think that
calling something a "release candidate" (when we know it isn't) with the
sole purpose of gathering more feedback is a good idea. As you say,
"release candidate" is an established name in the industry and we
shouldn't go against it.

> 
> [...]
>>
>>
>> What I really feel here is that these are symptoms of a more serious issue,
>> which is our commitment to time-based releases.
>>
>> Instead of entering the RC phase when the product is ready for it, we forcibly
>> push for it because of the approaching deadlines.
>>
>> This results in lack of proper testing before the RC (since there's no real
>> stabilization phase). Also, because in the RC phase only critical bugfixes are
>> accepted, we end up releasing a product with known serious bugs (causing .0
>> releases to be frowned upon because full of bugs).
> 
>  [Kalinowski Maurice] 
> I would object to that. The reason we are having so many delays for minor releases is that we are not taking the feature freeze seriously, push a huge amount of stuff still into it (with at least questionable significance) and pray that it'll work. As you mention, it never does, but neither does everyone accept the fact that each and every individual contributing raises the chance of causing regressions and further bugs at that stage.
> 
> The idea of having patch level releases more frequent is certainly welcomed, but then again the problems are the same, way too many changes for those causing a lot of effort for testing and verification.

Well, then again we have a problem elsewhere in our process: we're not
being strict enough on the kind of changes landing in stable branches.
Which is a fair critique, and one that should actually make us rivisit
our branching model or our approval/maintainership model if we think
we're failing it.

My 2 cents,
-- 
Giuseppe D'Angelo | giuseppe.dangelo at kdab.com | Senior Software Engineer
KDAB (UK) Ltd., a KDAB Group company | Tel: UK +44-1625-809908
KDAB - Qt, C++ and OpenGL Experts

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4007 bytes
Desc: Firma crittografica S/MIME
URL: <http://lists.qt-project.org/pipermail/releasing/attachments/20161221/abad8025/attachment.bin>