[Qt-interest] load url using QWebPage in a thread

[Tarandeep Singh]

On Tue, Apr 20, 2010 at 1:31 PM, Jason H <scorp1us at yahoo.com> wrote:

> What you need to do, simply, is just use QWebFrame::load() in your regular
> (GUI) thread, just like in the examples.
> page.mainFrame()->load(url);
>
>
Thanks Jason.  I made the changes & got it working, but occasionally  the
application crashes (segmentation fault).

The reason I want to use threads is- I want to have a muti-threaded server
that will receive request (url), load the url into QWebPage and then get its
html (after QWebPage runs javascripts, css etc) and send the html back to
the client.

I cut my application to do the bare minimum and I am pasting the code
snippet here, Please see if you can find some obvious flaw.

/////////////// main.cpp //////////////
#include <QApplication>
#include "server.h"

int main(int argc, char *argv[])
{
  QApplication app(argc, argv);
  Server server;
  server.listen( QHostAddress( ipaddress), port);
  return app.exec();
}

////////////////// server.cpp ///////////////
#include "server.h"
#include "thread.h"

Server::Server(QObject *parent) : QTcpServer(parent) { }

void Server::incomingConnection(int socketDescriptor)
{
  QUrl url ( "some url to load");
  QWebPage *page = new QWebPage( );
  page->mainFrame()->load( url);
  Thread *thread = new Thread(page, socketDescriptor, this);
  connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
  thread->start();
}


//////////////////////// thread.cpp ///////////////////
#include "thread.h"

Thread::Thread(QWebPage *p, int sd, QObject *parent) : QThread(parent)
{
  page = p;
  socketDescriptor = sd;
}

Thread::~Thread( )
{
  delete page;
}

void static waitForSignal(QObject* obj, const char* signal)
{
  QEventLoop loop;
  QObject::connect( obj, signal, &loop, SLOT( quit( )));
  loop.exec( );
}

void Thread::run()
{
  waitForSignal( page, SIGNAL( loadFinished(bool)));
  QString html = page->mainFrame( )->toHtml( );
  // send html back to client
}
/////////////////////////////////////////////////////////////////////////////////


Thanks,
Tarandeep


I am not sure why you need to use threads, but lets indulge and say you do
> (maybe you're trying to spread SSL load across cores or something?)  In that
> case you should just use QNetworkAccessManager  in a thread to get the
> document, then use QWebFrane::setHtml(),  passing the HTML retrieved by
> QN.A.M. The thread would emit a signal in response to replyFinished, , for
> example, loadedHmtl(QString html) and you'd connect it to a function that
> would the load it in the page as:
>  page.mainFrame()->setHtml(html);
>
> Rememeber the implicitly shared types (like QString) can be used in
> signals/slots across thread boundaries. Widgets aren't implicitly shared, so
> you cannot, in a QThread subclass do page->mainFrame()->load().
>
>
>
>
> ------------------------------
> *From:* Tarandeep Singh <tarandeep at gmail.com>
> *To:* Jason H <scorp1us at yahoo.com>
> *Cc:* qt-interest <qt-interest at trolltech.com>
> *Sent:* Tue, April 20, 2010 3:54:06 PM
> *Subject:* Re: [Qt-interest] load url using QWebPage in a thread
>
> On Tue, Apr 20, 2010 at 12:40 PM, Jason H <scorp1us at yahoo.com> wrote:
>
>> Non-GUI threads (there is only one) cannot use GUI things.
>> page->mainFrame( )->load( url);  isn't right
>> You need to collect the page and return it to the GUI thread. You can use
>> QNetworkAccessManager to get the page in a thread. Also, you can use
>> QNetworkAccessManager for multiple requets in one (the GUI)  thread, since
>> it is non-blocking. The performance differences should be minimal.
>>
>>
> Thanks Jason for the reply.
>
> I didn't understand this clearly- "collect the page and return it to the
> GUI thread". I am aware this is a method- moveToThread( ), so I should call
> page->moveToThread( guiThread). But which is the GUI thread here? the main
> program control flow? How do I get its thread?
>
> Also, I checked QNetworkAccessManager api, I don't see any method that will
> give me Page object.
>
> Can you please point me to some code snippet so that I can understand this.
>
> Appreciate your help,
> Tarandeep
>
>
>>
>> ------------------------------
>> *From:* Tarandeep Singh <tarandeep at gmail.com>
>> *To:* qt-interest at trolltech.com
>> *Sent:* Tue, April 20, 2010 1:15:49 PM
>> *Subject:* [Qt-interest] load url using QWebPage in a thread
>>
>> Hi,
>>
>> I am trying to build a multi threaded server that loads a url in QWebPage
>> in a separate thread. But I am not able to get it working. I am creating the
>> QWebPage in the run method of the thread. Is this right way? How do I
>> connect QWebPage's loadFinished signal to find out the page has been loaded
>> (so I can do further processing, in my case, get the HTML of the loaded
>> page)
>>
>> Here is the code snippet.
>>
>> /////////// MyServer extends QTcpServer /////////////////
>> MyServer::MyServer(QObject *parent) : QTctServer( parent) { }
>>     : QTcpServer(parent)
>> {
>> }
>>
>> void MyServer::incomingConnection(
>> int socketDescriptor)
>> {
>>     QString url( "http://www.dummyurl.com/");
>>   MyThread *thread = new MyThread( socketDescriptor, url, this);
>>   connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
>>   thread->start();
>> }
>>
>>
>> ///////////// MyThread ////////////////////
>> MyThread::MyThread( int socketDescriptor, const QString &urlstr, QObject
>> *parent)
>>     : QThread(parent), socketDescriptor(socketDescriptor),
>> urlstring(urlstr)
>> {
>> }
>>
>> void MyThread::run()
>> {
>>     QTcpSocket tcpSocket;
>>
>>     if (!tcpSocket.setSocketDescriptor(socketDescriptor)) {
>>         emit error(tcpSocket.error());
>>         return;
>>     }
>>
>>         QUrl url( urlstring);
>>         QWebPage *page = new QWebPage( );
>>         page->mainFrame( )->load( url);
>>
>>         //// HOW TO WAIT TILL LOADING FINISHED
>>         //// GET THE html of the page
>>         QString html = page->mainFrame( )->toHtml( );
>>
>>         //// OTHER STUFF
>>
>>         tcpSocket.write( returnValue);
>>         tcpSocket.disconnectFromHost();
>>         tcpSocket.waitForDisconnected();
>> }
>>
>> //////////// main.cpp //////////////////
>> int main(int argc, char *argv[])
>> {
>>   QApplication app(argc, argv, false);
>>   QApplication::setGraphicsSystem( "raster");
>>   app.setStyle( new QCleanlooksStyle( ));
>>   MyServer server;
>>   server.listen( QHostAddress( ip), portNo))
>>   return app.exec();
>> }
>>
>>
>> Thanks,
>> Tarandeep
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.qt-project.org/pipermail/qt-interest-old/attachments/20100420/46961d01/attachment.html