[Development] [QtMultimedia] How to make to use a HW codecs by QML media player on Linux?

Wed May 25 16:12:17 CEST 2016

Now, I have added this output:

{code}
qDebug() << "*** TEST Best sink name" << elementName << ", m_videoSink:" <<
m_videoSink;
{code}

to the ctor of QGStreamerVideoOverlay in file
/src/gsttools/qgstreamervideooverlay.cpp,
and now I see this output:

{quote}
root at apalis-t30:~/Downloads# ./video-player welcome.avi
QIconvCodec::convertToUnicode: using Latin-1 for conversion, iconv_open
failed
QIconvCodec::convertFromUnicode: using Latin-1 for conversion, iconv_open
failed
Unable to query physical screen size, defaulting to 100 dpi.
To override, set QT_QPA_EGLFS_PHYSICAL_WIDTH and
QT_QPA_EGLFS_PHYSICAL_HEIGHT (in millimeters).
Setting videosink to  "nv_omx_hdmi_videosink"
NvxLiteH264DecoderInit : Opening TVMR H264 block
NvxLiteH264DecoderInit : Opening TVMR H264 block
*** TEST Best sink name "nv_omx_hdmi_videosink" , m_videoSink: 0x15e1e8
*** TEST Best sink name "nv_omx_hdmi_videosink" , m_videoSink: 0x15e5c0
NvMMLiteOpen : Block : BlockType = 260
++++++ NvAvpOpen +++++++
NvMMLiteBlockCreate : Block : BlockType = 260
++++++++++++ TVMRFrameDelivery +++++++++++++++
BeginSequence 1920x720
pnvsi->nDecodeBuffers = 3
Display Resolution : (1920x720)
Display Aspect Ratio : (1920x720)
cbBeginSequence at 433: SurfaceLayout = 2
pStreamInfo->NumOfSurfaces = 7, MaxDPB = 24, InteraceStream = 0,
InterlaceEnabled = 0
Allocating new output: 1920x720 (x 9)
Warning: "A lot of buffers are being dropped."
Warning: "A lot of buffers are being dropped."
Warning: "A lot of buffers are being dropped."
Warning: "A lot of buffers are being dropped."
^Croot at apalis-t30:~/Downloads#
{quote}

where are:

{quote}
*** TEST Best sink name "nv_omx_hdmi_videosink" , m_videoSink: 0x15e1e8
*** TEST Best sink name "nv_omx_hdmi_videosink" , m_videoSink: 0x15e5c0
{quote}

BUT: Nothing changes, the CPU loading ~300% as before... So, seems, the
problem is *NOT IN VIDEO-SINK* selection...

So, now, I am confused.. WTF? WTF? WTF? :(

BR,
Denis

2016-05-25 16:53 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:

> The solution from:
>
> https://devtalk.nvidia.com/default/topic/894891/jetson-tk1/gstreamer-1-0-and-qt5-video-playback/
>
> with adding the:
>
> {code}
>         qDebug() << "Setting videosink to " << videoSinkString;
>         qputenv("QT_GSTREAMER_WINDOW_VIDEOSINK", videoSinkString.toUtf8());
> {code}
>
> to the main.cpp, where videoSinkString is nv_omx_hdmi_videosink, does not
> work too...
>
> WTF?
>
>
>
> 2016-05-25 15:07 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:
>
>> Now, I have recompiled the QtMM plugin (with default gstreamer 0.10), but
>> with adding of debug macro DEBUG_PLAYBIN.
>> Then I got following log: https://paste.kde.org/p6ztesnsq
>>
>> Where I do not see any mentions about the selected gst sinks.. I see only
>> this:
>>
>> {quote}
>> Set video output: QGstreamerVideoRenderer(0x153fe0)
>> Current sink: fakesink0 0x13e590 pending:  0x0 new sink: fakesink0
>> 0x13e590
>> Video sink has not changed, skip video output reconfiguration
>> Video sink has chaged, reload video output
>> void QGstreamerPlayerSession::setVideoRenderer(QObject*)
>> Set video output: QGstreamerVideoRenderer(0x153fe0)
>> Current sink: fakesink0 0x13e590 pending:  0x0 new sink:
>> qvideosurfacegstsink0 0x1bc470
>> Reconfigure video output
>> The pipeline has not started yet, pending state:
>> QMediaPlayer::StoppedState
>> Video sink has chaged, reload video output
>> void QGstreamerPlayerSession::setVideoRenderer(QObject*)
>> Set video output: QGstreamerVideoRenderer(0x153fe0)
>> Current sink: qvideosurfacegstsink0 0x1bc470 pending:  0x0 new sink:
>> qvideosurfacegstsink0 0x1bc470
>> Video sink has not changed, skip video output reconfiguration
>> {quote}
>>
>> what is it 'fakesink', 'qvideosurfacegstsink0' ? where is any of desired:
>>
>> {quote}
>> root at apalis-t30:~/Downloads# gst-inspect | grep sink | grep nv
>> omx:  nv_omx_audiosink: OpenMAX IL audiosink element
>> omx:  nv_gstbin_videosink: OpenMAX IL videosink element
>> omx:  nv_omx_videosink: OpenMAX IL videosink element
>> omx:  nv_omx_hdmi_videosink: OpenMAX IL hdmi videosink element
>> omx:  nv_omx_overlaysink: OpenMAX IL overlaysink element
>> omx:  nv_gl_eglimagesink: OpenMAX IL videosink element
>> nvxvimagesink:  nvxvimagesink: Video sink
>> root at apalis-t30:~/Downloads#
>> {quote}
>>
>> sinks? I do not understand nothing.. o_O How I can see the selected sink
>> name?
>>
>>
>>
>>
>> 2016-05-25 14:35 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:
>>
>>> > BTW: As I remember, I saw some environment variable which allows to
>>> setup custom video sink... I need to see sources of QtMM... if I'm not
>>> mistaked..
>>> maybe it will help...
>>>
>>> I have found this env variables:
>>>
>>> * QT_GSTREAMER_WIDGET_VIDEOSINK
>>> * QT_GSTREAMER_WINDOW_VIDEOSINK
>>>
>>> and try to use it:
>>>
>>> $export QT_GSTREAMER_WIDGET_VIDEOSINK=nv_omx_hdmi_videosink
>>> $export QT_GSTREAMER_WINDOW_VIDEOSINK=nv_omx_hdmi_videosink
>>> $./my-qml-video-player
>>>
>>> but, this does not help...
>>>
>>> BR,
>>> Denis
>>>
>>>
>>>
>>> 2016-05-25 8:30 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:
>>>
>>>> > Gstreamer 1.0+ goes some way to addressing these issues....
>>>>
>>>> The Apalis T30 board has GPU from the NVidia. So, when I use
>>>> 'gst-inspect' I see some of 'nv' sinks.
>>>> So, as I understand, the easy ways to do it are:
>>>>
>>>> 1) I should try to use GStreamer 1.0 (instead of 0.10) "as is" with
>>>> QtMM (rebuild QtMM with this GStreamer), at first stage.
>>>>
>>>> 2) I need try to change QtMM sources and try to set the specific
>>>> vendor-video sinks there, at second stage... (in case the first stage does
>>>> not work).
>>>>
>>>> Is it?
>>>>
>>>> BTW: As I remember, I saw some environment variable which allows to
>>>> setup custom video sink... I need to see sources of QtMM... if I'm not
>>>> mistaked..
>>>> maybe it will help...
>>>>
>>>> > but even within the standard packages there still seems to be
>>>> multiple APIs for codecs which produce EGL surfaces and none of them have
>>>> been utilized in QtMultimedia yet
>>>>
>>>> Could you please provide an examples of this API which is not adopted
>>>> in QtMM?
>>>>
>>>>
>>>>
>>>> BR,
>>>> Denis
>>>>
>>>>
>>>> 25.05.2016 4:44, Andrew den Exter пишет:
>>>>
>>>> The short answer is some amount of platform adaptation will be required
>>>> to enable hardware acceleration on your board.  Looking at the page you
>>>> linked to; that adaptation might be achieved by creating an implementation
>>>> of QGstBufferPoolPlugin (gstreamer 0.10 only) which exposes EGL surfaces
>>>> associated with the x-nv-yuv buffers produced by the accelerated
>>>> codec.  Or you might also need to create your own video node by
>>>> implementing QSGVideoNodeFactoryPlugin or something more invasive may
>>>> be required, it really depends on how the vendor implemented their
>>>> gstreamer plugins
>>>>
>>>> The reason hardware acceleration doesn't just work is that hardware
>>>> vendors have traditionally had to create extensions to gstreamer in order
>>>> to implement their hardware accelerated plugins, and these extensions tend
>>>> to require a vendor specific video sink to work.  If all you care about is
>>>> rendering to the full dimensions of a native window then autovideosink will
>>>> generally resolve to that vendor specific video sink and that's why the
>>>> standard example command line examples tend to work.  But when it comes to
>>>> something more complicated like compositing into an opengl scene or even as
>>>> simple as rendering to a sub rect of a window then you're into the realm of
>>>> vendor extensions and without a port to your specific hardware it probably
>>>> won't work.  Gstreamer 1.0+ goes some way to addressing these issues, but
>>>> even within the standard packages there still seems to be multiple APIs for
>>>> codecs which produce EGL surfaces and none of them have been utilized in
>>>> QtMultimedia yet.
>>>>
>>>>
>>>> Andrew
>>>>
>>>>
>>>> On Wed, May 25, 2016 at 12:40 AM, Denis Shienkov <
>>>> denis.shienkov at gmail.com> wrote:
>>>>
>>>>> Hi all.
>>>>>
>>>>> I have the Toradex (Apalis T30) embedded board:
>>>>>
>>>>> https://www.toradex.com/computer-on-modules/apalis-arm-family/nvidia-tegra-3
>>>>>
>>>>> which has:
>>>>>
>>>>> http://developer.toradex.com/files/toradex-dev/uploads/media/Colibri/Linux/Images/Apalis_T30_LinuxImageV2.6Beta1_20160331.tar.bz2
>>>>>
>>>>> Linux Image, with the GStreamer 0.10.
>>>>>
>>>>> Also I have compiled Qt 5.6 and installed it to the board.
>>>>>
>>>>> When I try to play any *.MP4 video file from the QML Video item:
>>>>> http://doc.qt.io/qt-5/qml-qtmultimedia-video.html
>>>>>
>>>>> I see, that my video has very-very lags, and I see (with htop utility)
>>>>> that CPU loading is ~300% (
>>>>> three cores running on ~100%).
>>>>>
>>>>> But, when I try to play the video, using the 'nvgstplayer' utility, I
>>>>> see that CPU loading is ~46% and all fine. Seems, the QML player loads the
>>>>> SW codecs instead of HW codecs...
>>>>>
>>>>> Is there are any way to say to QML player to use a HW codecs? Or,
>>>>> maybe, I need to fix a sources of QtMM (where I need to change it?) Or,
>>>>> maybe, is there are another way to do this?
>>>>>
>>>>> Otherwise, is just a **HELL** and there is no sense in QML
>>>>> QtMultimedia at all (But I need QML for anumations). And I do not know, how
>>>>> to fix it (I'm not expert in GStreamer).
>>>>>
>>>>> PS: I have tried the 'gst-launch' pipelines instead of 'nvgstplayer',
>>>>> as described here:
>>>>> http://developer.toradex.com/knowledge-base/video-playback-%28linux%29
>>>>> but it crashes with sigsegv....
>>>>>
>>>>> BR,
>>>>> Denis
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Development mailing list
>>>>> Development at qt-project.org
>>>>> http://lists.qt-project.org/mailman/listinfo/development
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20160525/253c6dbf/attachment.html>