[Qt-interest] Multiple page rendering using webkit

[Jason H]

Yes, I'd probably use renderWidget() to get a pixmap and find the fastest way to 
get that to the master app. (local socket?)
usability becomes an issue for anything more than a thumb nail. You'd need a way 
to map the mouse/keyboard events back through the socket and make the 
corresponding adjustments in the corresponding webkit process.






----- Original Message ----
From: Arnold Krille <arnold at arnoldarts.de>
To: qt-interest at trolltech.com
Sent: Fri, April 15, 2011 4:46:41 PM
Subject: Re: [Qt-interest] Multiple page rendering using webkit

Hi,

On Friday 15 April 2011 21:33:07 Tarandeep Singh wrote:
> I need to render a large number of pages and obviously need do this as
> efficiently as possible.
> Since GUI rendering needs to be done on the main thread, so I can't create
> multiple QWebPage threads.
> 
> So I am thinking of these 2 solutions-
> 
> 1) Create one process and have N QWebPage instances so that N pages can be
> rendered simultaneously. Here assumption is QT+Webkit behind the scenes
> create threads to do parallel IO etc.
> 
> 2) Create N processes where each process handles 1 page request at a given
> time.
> 
> Can someone please advice which solution should work better or is there is
> another way to achieve this.
> I have 16 cores and 32G of RAM.

I assume you have the to-be-rendered webpages on disk already as html and 
"only" want the rendered output in pdf/svg/png or whatever. (If not I think 
you can easily do with three to four webpages which in turns load the files 
from network and render and save to disk, net-io will be the slowest point.)

As all gui-stuff has to run in the thread that instantiated QApplication, you 
can really only use one thread for rendering. But who says that you can't run 
multiple processes in parallel?
One process, maybe even the master of your app, loads the html from disk into 
a queue which is in shared memory and then starts as many render-processes as 
you like. These render processes each take a pages html from the queue and 
render it and either put the rendered data back onto another shared queue or 
save directly to files on disk.
If you want to leave room for other processes to run (like the desktop), you 
should run like min(1, idealThreadCount()/2) render processes.

Hope that helps,

Arnold