[Interest] Help parsing output of "mksquashfs" command

[Elvis Stansvik]

2016-06-02 13:47 GMT+02:00 Juan Navarro <oneorjuan at gmail.com>:
> Hello,
>
> I'm building a GUI tool under Kubuntu Linux 14.04, which among other things,
> creates SquashFS (https://en.wikipedia.org/wiki/SquashFS) images with the
> command-line command "mksquashfs". Due to the size of the images, this
> process takes a good amount of time, so I'd like to show its progress with a
> QProgressBar.
>
> Problem here is that the default output from this command is not really
> intended to be parsed. It uses a dynamic progress bar + a progress
> percentage which repaint themselves on the terminal, I guess that using the
> technique of printing "\r" (carriage returns) without "\n" (line feeds), so
> the progress bar is always drawn on the same row of the console output:
>
>     $ mksquashfs /home /tmp/test.sfs -noappend -processors 1
>     Parallel mksquashfs: Using 1 processor
>     Creating 4.0 filesystem on /tmp/test.sfs, block size 131072.
>     [======================================/                ]  699/1250  75%
>
>
> However I've found the way of printing the progress bar + percentage value
> in a serial way, ie. each iteration gets printed on its own line, with the
> following command line:
>
>     $ script --return --flush --quiet \
>     --command "mksquashfs /home \
>     /tmp/test.sfs -info -progress -noappend \
>     -processors 1" > test.txt
>
> With this, the output shown in "file.txt" is somewhat similar to this:
>
>     file ..., uncompressed size 2755336 bytes
>     [=========\                                             ]   481/12359
> 3%
>     file ..., uncompressed size 726904 bytes
>     [==========\                                            ]   528/12359
> 4%
>     file ..., uncompressed size 577 bytes
>     [==============\                                        ]   719/12359
> 5%
>
> This should be really easy to parse, extract the "x%" from the end of the
> line, and send that to the QProgressBar.
>
> So I've ported this command line to my Qt code. This is the relevant part of
> the code:
>
>     class MyClass
>     {
>         QProcess p;
>
>         void start() {
>             p.setProcessChannelMode(QProcess::SeparateChannels);
>             p.setReadChannel(QProcess::StandardOutput);
>             connect(&p, SIGNAL(readyReadStandardOutput()),
>                 this, SLOT(onProcessReadyReadStdout()));
>
>             QString c = "script";
>             QStringList a;
>             a << "--return" << "--flush" << "--quiet"
>             << "--command"
>             << "mksquashfs /media/data/KSHOW_320/RO/home "
>                "/tmp/test.sfs -info -progress "
>                "-noappend -no-recovery -processors 1";
>             p.start(c, a, QIODevice::ReadOnly | QIODevice::Unbuffered |
> QIODevice::Text);
>         }
>
>     private slots:
>         void onProcessReadyReadStdout() {
>             qDebug() << "New data:" << p.readAllStandardOutput();
>         }
>     }
>
>
> The problem is that given this code, the output doesn't contain the progress
> bar...
>
> With Qt 4.8.4:
>     New data: "
>     file ..., uncompressed size 2755336 bytes
>     "
>     New data: "
>     file ..., uncompressed size 726904 bytes
>     "
>     New data: "
>     file ..., uncompressed size 577 bytes
>     "
>
> With Qt 5.6.0:
>     New data: "\nfile ..., uncompressed size 2755336 bytes \n"
>     New data: "\nfile ..., uncompressed size 726904 bytes \n"
>     New data: "\nfile ..., uncompressed size 577 bytes \n"
>
> (Note how in Qt 5.6.0 the same command shows a different output, by
> explicitly showing '\n' characters; also the flag "QIODevice::Text" doesn't
> make a difference here).
>
> I'm quite lost here, because at this point I assume that QProcess is doing
> the right thing, but I don't understand at what point in the command chain
> the output is being generated differently between those two different ways
> of running the same command.
>
> It's probably the way the carriage returns are managed in the QProcess vs.
> shell redirection to a file, but I'd like to see if anyone here has some
> comment.

Maybe it needs a TTY to output the progress bar? That's out of scope
for QProcess, but if you're fine with a dependency on kcoreaddons +
kpty, it looks like the latter has a KPtyProcess you could use [1].

Elvis

[1] https://api.kde.org/frameworks/kpty/html/classKPtyProcess.html

>
> Regards,
> Juan
>
> _______________________________________________
> Interest mailing list
> Interest at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/interest
>