[Qt-interest] load url using QWebPage in a thread
Sylvain Pointeau
sylvain.pointeau at gmail.com
Wed Apr 21 14:43:04 CEST 2010
I would also say that the multi-threading is not bringing any value in this
case.
On Wed, Apr 21, 2010 at 4:36 AM, Jason H <scorp1us at yahoo.com> wrote:
> Since Qt uses non-blocking I/O, there's not a huge amount of performance
> gain, but if you are using a lot of JS and performance *IS* a problem, then
> I am afraid you are stuck. The only way to work around that in Qt is to
> spawn multiple local rendering processes and have a separate proxy process
> connect to one of the many processes. Each render can still service multiple
> requests, but you would want to start N renderers where N=number of CPUs.
> This way, you can distribute load across all cores, while having a central
> socket.
>
> You didn't listen... you can't pass QWebPage to a thread and have it work
> [safely].
> You need to emit the HTML to the main thread for rendering. To do that,
> define a signal, and connect a main GUI thread handler to the thread event
> that is emitted.
>
>
>
>
> ------------------------------
> *From:* Tarandeep Singh <tarandeep at gmail.com>
> *To:* Jason H <scorp1us at yahoo.com>
> *Cc:* qt-interest <qt-interest at trolltech.com>
> *Sent:* Tue, April 20, 2010 8:43:40 PM
>
> *Subject:* Re: [Qt-interest] load url using QWebPage in a thread
>
> On Tue, Apr 20, 2010 at 1:31 PM, Jason H <scorp1us at yahoo.com> wrote:
>
>> What you need to do, simply, is just use QWebFrame::load() in your regular
>> (GUI) thread, just like in the examples.
>> page.mainFrame()->load(url);
>>
>>
> Thanks Jason. I made the changes & got it working, but occasionally the
> application crashes (segmentation fault).
>
> The reason I want to use threads is- I want to have a muti-threaded server
> that will receive request (url), load the url into QWebPage and then get its
> html (after QWebPage runs javascripts, css etc) and send the html back to
> the client.
>
> I cut my application to do the bare minimum and I am pasting the code
> snippet here, Please see if you can find some obvious flaw.
>
> /////////////// main.cpp //////////////
> #include <QApplication>
> #include "server.h"
>
> int main(int argc, char *argv[])
> {
> QApplication app(argc, argv);
> Server server;
> server.listen( QHostAddress( ipaddress), port);
> return app.exec();
> }
>
> ////////////////// server.cpp ///////////////
> #include "server.h"
> #include "thread.h"
>
> Server::Server(QObject *parent) : QTcpServer(parent) { }
>
> void Server::incomingConnection(int socketDescriptor)
> {
> QUrl url ( "some url to load");
> QWebPage *page = new QWebPage( );
> page->mainFrame()->load( url);
> Thread *thread = new Thread(page, socketDescriptor, this);
> connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
> thread->start();
> }
>
>
> //////////////////////// thread.cpp ///////////////////
> #include "thread.h"
>
> Thread::Thread(QWebPage *p, int sd, QObject *parent) : QThread(parent)
> {
> page = p;
> socketDescriptor = sd;
> }
>
> Thread::~Thread( )
> {
> delete page;
> }
>
> void static waitForSignal(QObject* obj, const char* signal)
> {
> QEventLoop loop;
> QObject::connect( obj, signal, &loop, SLOT( quit( )));
> loop.exec( );
> }
>
> void Thread::run()
> {
> waitForSignal( page, SIGNAL( loadFinished(bool)));
> QString html = page->mainFrame( )->toHtml( );
> // send html back to client
> }
>
> /////////////////////////////////////////////////////////////////////////////////
>
>
> Thanks,
> Tarandeep
>
>
> I am not sure why you need to use threads, but lets indulge and say you do
>> (maybe you're trying to spread SSL load across cores or something?) In that
>> case you should just use QNetworkAccessManager in a thread to get the
>> document, then use QWebFrane::setHtml(), passing the HTML retrieved by
>> QN.A.M. The thread would emit a signal in response to replyFinished, , for
>> example, loadedHmtl(QString html) and you'd connect it to a function that
>> would the load it in the page as:
>> page.mainFrame()->setHtml(html);
>>
>> Rememeber the implicitly shared types (like QString) can be used in
>> signals/slots across thread boundaries. Widgets aren't implicitly shared, so
>> you cannot, in a QThread subclass do page->mainFrame()->load().
>>
>>
>>
>>
>> ------------------------------
>> *From:* Tarandeep Singh <tarandeep at gmail.com>
>> *To:* Jason H <scorp1us at yahoo.com>
>> *Cc:* qt-interest <qt-interest at trolltech.com>
>> *Sent:* Tue, April 20, 2010 3:54:06 PM
>> *Subject:* Re: [Qt-interest] load url using QWebPage in a thread
>>
>> On Tue, Apr 20, 2010 at 12:40 PM, Jason H <scorp1us at yahoo.com> wrote:
>>
>>> Non-GUI threads (there is only one) cannot use GUI things.
>>> page->mainFrame( )->load( url); isn't right
>>> You need to collect the page and return it to the GUI thread. You can use
>>> QNetworkAccessManager to get the page in a thread. Also, you can use
>>> QNetworkAccessManager for multiple requets in one (the GUI) thread, since
>>> it is non-blocking. The performance differences should be minimal.
>>>
>>>
>> Thanks Jason for the reply.
>>
>> I didn't understand this clearly- "collect the page and return it to the
>> GUI thread". I am aware this is a method- moveToThread( ), so I should call
>> page->moveToThread( guiThread). But which is the GUI thread here? the main
>> program control flow? How do I get its thread?
>>
>> Also, I checked QNetworkAccessManager api, I don't see any method that
>> will give me Page object.
>>
>> Can you please point me to some code snippet so that I can understand
>> this.
>>
>> Appreciate your help,
>> Tarandeep
>>
>>
>>>
>>> ------------------------------
>>> *From:* Tarandeep Singh <tarandeep at gmail.com>
>>> *To:* qt-interest at trolltech.com
>>> *Sent:* Tue, April 20, 2010 1:15:49 PM
>>> *Subject:* [Qt-interest] load url using QWebPage in a thread
>>>
>>> Hi,
>>>
>>> I am trying to build a multi threaded server that loads a url in QWebPage
>>> in a separate thread. But I am not able to get it working. I am creating the
>>> QWebPage in the run method of the thread. Is this right way? How do I
>>> connect QWebPage's loadFinished signal to find out the page has been loaded
>>> (so I can do further processing, in my case, get the HTML of the loaded
>>> page)
>>>
>>> Here is the code snippet.
>>>
>>> /////////// MyServer extends QTcpServer /////////////////
>>> MyServer::MyServer(QObject *parent) : QTctServer( parent) { }
>>> : QTcpServer(parent)
>>> {
>>> }
>>>
>>> void MyServer::incomingConnection(
>>> int socketDescriptor)
>>> {
>>> QString url( "http://www.dummyurl.com/");
>>> MyThread *thread = new MyThread( socketDescriptor, url, this);
>>> connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
>>> thread->start();
>>> }
>>>
>>>
>>> ///////////// MyThread ////////////////////
>>> MyThread::MyThread( int socketDescriptor, const QString &urlstr, QObject
>>> *parent)
>>> : QThread(parent), socketDescriptor(socketDescriptor),
>>> urlstring(urlstr)
>>> {
>>> }
>>>
>>> void MyThread::run()
>>> {
>>> QTcpSocket tcpSocket;
>>>
>>> if (!tcpSocket.setSocketDescriptor(socketDescriptor)) {
>>> emit error(tcpSocket.error());
>>> return;
>>> }
>>>
>>> QUrl url( urlstring);
>>> QWebPage *page = new QWebPage( );
>>> page->mainFrame( )->load( url);
>>>
>>> //// HOW TO WAIT TILL LOADING FINISHED
>>> //// GET THE html of the page
>>> QString html = page->mainFrame( )->toHtml( );
>>>
>>> //// OTHER STUFF
>>>
>>> tcpSocket.write( returnValue);
>>> tcpSocket.disconnectFromHost();
>>> tcpSocket.waitForDisconnected();
>>> }
>>>
>>> //////////// main.cpp //////////////////
>>> int main(int argc, char *argv[])
>>> {
>>> QApplication app(argc, argv, false);
>>> QApplication::setGraphicsSystem( "raster");
>>> app.setStyle( new QCleanlooksStyle( ));
>>> MyServer server;
>>> server.listen( QHostAddress( ip), portNo))
>>> return app.exec();
>>> }
>>>
>>>
>>> Thanks,
>>> Tarandeep
>>>
>>>
>>
>>
>
>
> _______________________________________________
> Qt-interest mailing list
> Qt-interest at trolltech.com
> http://lists.trolltech.com/mailman/listinfo/qt-interest
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.qt-project.org/pipermail/qt-interest-old/attachments/20100421/c6eeb73d/attachment.html
More information about the Qt-interest-old
mailing list