[Development] Introducing: test batching

Morten Sørvig Morten.Sorvig at qt.io
Tue Aug 30 09:57:04 CEST 2022



> On 29 Aug 2022, at 13:48, Thiago Macieira <thiago.macieira at intel.com> wrote:
> 
> On Monday, 29 August 2022 05:34:48 -03 Mikołaj Boć wrote:
>> We plan to enable the feature on CI for WASM. Since that has not yet been
>> done, breaking the tests unintentionally is possible. With CI on, it will
>> become everyone’s responsibility to keep them running (at least on WASM,
>> which will be checked by the CI). Following the guideline on symbol name
>> duplication will save us a considerable amount of work.
>> 
>> Batching tests for other platforms is an opt-in, please do check whether
>> that would simplify the testing pipeline on the platform you are proficient
>> with.
> 
> I'd recommend you consider enabling that for at least one of the fast 
> platforms that are in the minimal build set of the CI. WASM doesn't appear to 
> be a fast compile, so waiting 2 hours to find out that you've made a mistake is 
> sub-optimal.

This was true until recently, but is now (6.4) only partially true.

Compiling individual translation units runs at the normal speed of clang/llvm, also for
the wasm target. Linking can still be slow, in some cases.

llvm links webassembly reasonably fast, on par with static builds on other platforms.
Emscripten has an extra wasm rewrite/optimization step which runs any one of these 
are true:

   - the WASM_BIGINT option is not set (Qt 6.4 sets it by default)
   - optimizations are enabled (I’m unsure of at which level it kicks in)
   - asyncify is used

This final step is what is causing the slowdown.

The plan for the auto tests is to run them with asyncify enabled, since the Qt test suite
makes frequent use of synchronous waitFor() and similar calls. This means linking will
be slow, which is why the batched test feature was introduced to incur the linking overhead
only a few times for the test suite.

So in the end we hope that wasm CI performance will similar to the other platforms,
at least as far as the building step goes. If not, then we can always enable batching on
another platform.

- Morten








More information about the Development mailing list