[Development] QTextBoundaryFinder behavior change in Qt-5.0

[David Faure]

Hi Konstantin,

I'm seeing a unittest failure in KDE Frameworks (sonnet framework) due to 
changes in QTextBoundaryFinder. This isn't my domain of expertise, so can I 
ask you to take a look at the information below, to find out if it's an 
intentional change or a bug, and/or if maybe the sonnet code is buggy?

static bool
finderNextWord(QTextBoundaryFinder &finder, QString &word, int &bufferStart)
{
    QTextBoundaryFinder::BoundaryReasons boundary = finder.boundaryReasons();
    int start = finder.position(), end = finder.position();
    bool inWord = (boundary & QTextBoundaryFinder::StartWord) != 0;
    while (finder.toNextBoundary() > 0) {
        boundary = finder.boundaryReasons();
        if ((boundary & QTextBoundaryFinder::EndWord) && inWord) {
            end = finder.position();
            QString str = finder.string().mid(start, end - start);
            if (isValidWord(str)) {
                word = str;
                bufferStart = start;
                qDebug().nospace() << "Word at " << start << " word="
                         <<  str << ", len=" << str.length();
                return true;
            }
            inWord = false;
        }
        if ((boundary & QTextBoundaryFinder::StartWord)) {
            start = finder.position();
            inWord = true;
        }
    }
    return false;
}

The unittest starts from the string
   QString buffer( "This is     a sample buffer.Please test me .")
and calls the above method repeatedly.

The result with Qt5 is

QDEBUG : SonnetFilterTest::testFilter() Word at 0 word="This", len=4
QDEBUG : SonnetFilterTest::testFilter() Word at 5 word="is", len=2
QDEBUG : SonnetFilterTest::testFilter() Word at 12 word="a", len=1
QDEBUG : SonnetFilterTest::testFilter() Word at 14 word="sample", len=6
QDEBUG : SonnetFilterTest::testFilter() Word at 21 word="buffer.Please", 
len=13
FAIL!  : SonnetFilterTest::testFilter() Compared values are not the same
   Actual   (w.word): buffer.Please
   Expected (hits[hitNumber].word): buffer

The result with Qt4 is

QDEBUG : SonnetFilterTest::testFilter() Word at 0 word="This", len=4
QDEBUG : SonnetFilterTest::testFilter() Word at 5 word="is", len=2
QDEBUG : SonnetFilterTest::testFilter() Word at 12 word="a", len=1
QDEBUG : SonnetFilterTest::testFilter() Word at 14 word="sample", len=6
QDEBUG : SonnetFilterTest::testFilter() Word at 21 word="buffer", len=6
QDEBUG : SonnetFilterTest::testFilter() Word at 28 word="Please", len=6
QDEBUG : SonnetFilterTest::testFilter() Word at 35 word="test", len=4
QDEBUG : SonnetFilterTest::testFilter() Word at 40 word="me", len=2

So the dot character is no longer a word separator?

-- 
David Faure, faure at kde.org, http://www.davidfaure.fr
Sponsored by Nokia to work on KDE, incl. KDE Frameworks 5