[Interest] Distro-provided xcb packaging; xcb advnatages over xlib

[Rick Stockton]

On 04/11/2012 01:25 PM, Thiago Macieira wrote:

<< SNIP lots of good stuff, but this is a mostly-new post>>
> Qt 5's XCB plugin will require a somewhat newer XCB (i.e., more recent than
> the dinosaur age). I don't foresee a problem for that, as we've discussed that
> Qt 5 is new, so we don't feel obliged to support very old distributions.
>
> This might be a problem for non-Linux Unixes. I don't know how easy it is to
> build libxcb, but my first guess would be that it isn't: it's just binding to
> the protocol, which has been a standard for decades.
The protocol has been fairly stable, but the schema has occasional fixes 
and enhancements. (Some fixes, IIRC, resolving "GUI environment 
locked-up" bugs. Enhancements to utilities result in corresponding 
enhancements for the utilities schemas, and correspondiging changes in 
generated xcb Code.)

I'm only familiar with building on a fairly modern Linux environment. 
x11 now uses a well-written, and aggressively maintained, set of 
'Modular Build' scripts. They are absolutely wonderful, IMO. If I didn't 
already have latest-and-greatest test versions of x11 and the xcb add-on 
libraries directly from my Distro, I'd build them locally. (I used to do 
that.) BTW, I'm running Fedora-17, pre-beta, with "testing" rpm Repos 
enabled. If I had the brains and willingness to commit to being an xcb 
"maintainer", I would have done that - but I have no packaging 
experience, and my future availability is extremely uncertain. Older 
Ubuntu, like Mageia's current Beta, may be suffering side-effects of the 
split (done last year) between "utilities" and "window manager 
utitities" -- in some packages, header files and source files for the 
Window Manager utilities don't seem to find all of the symbols which 
NEED to be defined for Qt (at runtime).

Here are relevant bugID's:
https://bugs.mageia.org/show_bug.cgi?id=5030
https://bugreports.qt-project.org/browse/QTBUG-24835
https://bugreports.qt-project.org/browse/QTBUG-24913

< no SPAM, please, on QTBUG-24913: this is only a BugID where a person 
may volunteer to be the maintainer of the xlib plugin.> If you want to 
tell us that "we can't build xcb", please open a different bug, and 
describe how X11's modular build script is unworkable on your platform. 
Don't bother with an Ubuntu bug here, except as a reference for tracking 
the corresponding Ubuntu bug (in the same way that QTBUG-24835 shows the 
Qt User's symptoms of Mageia bugid 5030.) If you're on a supported LTR, 
then (IMO) *THEY* need to Backport from current Git, rather than leaving 
you with a Borked "3.8.1-pre". (My opinion only, and I don't speak for Qt!).

Meanwhile: It works, sort of, while staying back at 3.6. (You don't get 
all of the performance advantages which occur with 3.8.1 and newer, but 
you do avoid known xlib bugs.) HOWEVER, at 3.8.1, your packager needs to 
put it together correctly. Don't do as Mageia did- mixing old 
"3.8.1-pre" header files with EITHER old "3.8.1" Versions, or older 
"3.8.1" versions with selected cherry picks. The smartest way to avoid 
the problem, IMO, is to pull both fresh and re-build x11 with this 
"optional" package included. (And anyway, I seem to remember that it's 
not actually "optional" anymore. You HAVE TO build xcb to link the x11 
libraries successfully). My own system works beautifully.

> Still, there is an experimental xlib plugin in the Qt 5 source tree.
> Unfortunately, it is very much behind the xcb one in terms of functionality,
> and it's not guaranteed to compile.

Here's the other "fresh meat" of my post: A person who downloads the 
Alpha Release will not see an 'xlib' subdirectory within 
qtbase/src/plugins/platforms anymore. A Git commit from a few weeks ago 
removed it, along with all the other code which made reference to 
symbols from that platform plugin. It is possible to revert that 
specific commit - if you're comfortable with the consequences (your 
resulting tree will have a disconnected HEAD after doing this, making it 
basically unmaintainable). As of last night, it still compiles. But the 
xlib plugin has several painting problems, some other bugs, and it 
performs much worse than the xcb platform plugin. *I strongly recommend 
against doing this.* The xlib plugin has bugs; it's unsupported; and 
Qt<->xcb has a BIG performance advantage over Qt<->xlib anyway.


--- Gory Details of the performance advantage follow. If you're not 
interested in Gory Details of the Performance difference, you have no 
need to keep reading ---

Last night, I did that job -- for the purpose of creating a 
"Qt-Applications with xcb versus xlib" performance report to the xcb 
Developers Mailing List. (They have made a few lower-level tests before, 
but haven't used a program which is based on a Toolkit such as ours.) 
For my first test, I modified our "qtbase/examples/moveblocks" program 
(Using a larger window size and generating larger cubes, to create more 
screen damage; and also reducing the delays for the 3 "follower" squares 
to begin chasing the preceding square after only 25 ms). This is also a 
very artificial test, but it is using a popular Toolkit API :) , rather 
than native calls to xcb or xlib.

On my machine, (Phenom-X4 with NVidia 9600-GTS graphics, running 
Nouveau, Kwin compositing effects disabled), Kwin and Plasma were using 
less than 1% each, and every other task was using less than 0.1% CPU. 
The CPU power to run the "moveblocks" program task fell from roughly 8% 
(of a single Processor) using the xlib plugin, to roughly 7.2% using 
xcb. (A 10% improvement, internal to the Qt program.) Now don't forget: 
every thing to run the Statemachine, all the calls and events to/from 
the QPA plugins, are constant: That roughly 10% improvement happened 
within the Toolkit code, based only on changing the choice of Platform 
Plugin. Larger benefits accrued in the x11 task, of course: With the 
xlib plugin, x11 was consuming  8-11% of an individual CPU. With xcb, 
this fell to just 2.4 - 2.6%. I need to make further changes to the 
program,. to assure that xlib version of the program was "moving blocks" 
at the nearly same rate as the xcb version. (xcb might also be 
supporting more throughput per second, in spite of the stateMachine's 
timer limits.)

We are pretty sure of the cause of these advantages: xcb uses a 
buffered, asynchronous Request/Reply protocol. It can send "a bunch" of 
requests to x11 at once, and it x11 can return more than reply (or 
Event) to xcb as well.
- - - - -
In short, this graphics-intensive program was almost TWICE AS EFFICIENT 
when migrated from Qt/xlib to Qt/xcb. And the low-level graphics portion 
was actually more than twice as efficient; the other work in the Qt 
application, roughly constant in both implementations, becomes dominant 
in the Qt/xcb implementation. Back with the Qt/xlib/x11 implementation, 
xlib/x11 CPU utilization was was higher than EVERTHING the "moveblocks" 
program task was doing. Less bugs, better performance, not hard to build 
(but maybe hard to package lots of packages). Distros with borked, stale 
"3.8.1-pre" utilsxxx packages need to rebuild them. The Dev headers are 
busted; symptoms shown in 
https://bugreports.qt-project.org/browse/QTBUG-24835.