[Interest] Help parsing output of "mksquashfs" command

[Juan Navarro]

Hello,

I'm building a GUI tool under Kubuntu Linux 14.04, which among other
things, creates SquashFS (https://en.wikipedia.org/wiki/SquashFS) images
with the command-line command "mksquashfs". Due to the size of the images,
this process takes a good amount of time, so I'd like to show its progress
with a QProgressBar.

Problem here is that the default output from this command is not really
intended to be parsed. It uses a dynamic progress bar + a progress
percentage which repaint themselves on the terminal, I guess that using the
technique of printing "\r" (carriage returns) without "\n" (line feeds), so
the progress bar is always drawn on the same row of the console output:

    $ mksquashfs /home /tmp/test.sfs -noappend -processors 1
    Parallel mksquashfs: Using 1 processor
    Creating 4.0 filesystem on /tmp/test.sfs, block size 131072.
    [======================================/                ]  699/1250  75%


However I've found the way of printing the progress bar + percentage value
in a serial way, ie. each iteration gets printed on its own line, with the
following command line:

    $ script --return --flush --quiet \
    --command "mksquashfs /home \
    /tmp/test.sfs -info -progress -noappend \
    -processors 1" > test.txt

With this, the output shown in "file.txt" is somewhat similar to this:

    file ..., uncompressed size 2755336 bytes
    [=========\                                             ]   481/12359
3%
    file ..., uncompressed size 726904 bytes
    [==========\                                            ]   528/12359
4%
    file ..., uncompressed size 577 bytes
    [==============\                                        ]   719/12359
5%

This should be really easy to parse, extract the "x%" from the end of the
line, and send that to the QProgressBar.

So I've ported this command line to my Qt code. This is the relevant part
of the code:

    class MyClass
    {
        QProcess p;

        void start() {
            p.setProcessChannelMode(QProcess::SeparateChannels);
            p.setReadChannel(QProcess::StandardOutput);
            connect(&p, SIGNAL(readyReadStandardOutput()),
                this, SLOT(onProcessReadyReadStdout()));

            QString c = "script";
            QStringList a;
            a << "--return" << "--flush" << "--quiet"
            << "--command"
            << "mksquashfs /media/data/KSHOW_320/RO/home "
               "/tmp/test.sfs -info -progress "
               "-noappend -no-recovery -processors 1";
            p.start(c, a, QIODevice::ReadOnly | QIODevice::Unbuffered |
QIODevice::Text);
        }

    private slots:
        void onProcessReadyReadStdout() {
            qDebug() << "New data:" << p.readAllStandardOutput();
        }
    }


The problem is that given this code, the output doesn't contain the
progress bar...

With Qt 4.8.4:
    New data: "
    file ..., uncompressed size 2755336 bytes
    "
    New data: "
    file ..., uncompressed size 726904 bytes
    "
    New data: "
    file ..., uncompressed size 577 bytes
    "

With Qt 5.6.0:
    New data: "\nfile ..., uncompressed size 2755336 bytes \n"
    New data: "\nfile ..., uncompressed size 726904 bytes \n"
    New data: "\nfile ..., uncompressed size 577 bytes \n"

(Note how in Qt 5.6.0 the same command shows a different output, by
explicitly showing '\n' characters; also the flag "QIODevice::Text" doesn't
make a difference here).

I'm quite lost here, because at this point I assume that QProcess is doing
the right thing, but I don't understand at what point in the command chain
the output is being generated differently between those two different ways
of running the same command.

It's probably the way the carriage returns are managed in the QProcess vs.
shell redirection to a file, but I'd like to see if anyone here has some
comment.

Regards,
Juan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/interest/attachments/20160602/2853e466/attachment.html>