[Development] [QtMultimedia] How to make to use a HW codecs by QML media player on Linux?

Denis Shienkov denis.shienkov at gmail.com
Thu May 26 11:01:13 CEST 2016 

UPD: Seems the problem is ***NOT IN QtMM***..

For example, when I have compiled an usual cpp-based QMediaPlayer example,
and to set there:

{quote}
root at apalis-t30:~/Downloads# export
QT_GSTREAMER_WIDGET_VIDEOSINK=nv_omx_hdmi_videosink
root at apalis-t30:~/Downloads# export
QT_GSTREAMER_WINDOW_VIDEOSINK=nv_omx_hdmi_videosink
{quote}

then CPU loas ~50%, I is good!

So, seems, the problem is in QML && OpenGL!!! It is very-very sad....:(

BR,
Denis


2016-05-25 17:12 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:

> Now, I have added this output:
>
> {code}
> qDebug() << "*** TEST Best sink name" << elementName << ", m_videoSink:"
> << m_videoSink;
> {code}
>
> to the ctor of QGStreamerVideoOverlay in file
> /src/gsttools/qgstreamervideooverlay.cpp,
> and now I see this output:
>
> {quote}
> root at apalis-t30:~/Downloads# ./video-player welcome.avi
> QIconvCodec::convertToUnicode: using Latin-1 for conversion, iconv_open
> failed
> QIconvCodec::convertFromUnicode: using Latin-1 for conversion, iconv_open
> failed
> Unable to query physical screen size, defaulting to 100 dpi.
> To override, set QT_QPA_EGLFS_PHYSICAL_WIDTH and
> QT_QPA_EGLFS_PHYSICAL_HEIGHT (in millimeters).
> Setting videosink to  "nv_omx_hdmi_videosink"
> NvxLiteH264DecoderInit : Opening TVMR H264 block
> NvxLiteH264DecoderInit : Opening TVMR H264 block
> *** TEST Best sink name "nv_omx_hdmi_videosink" , m_videoSink: 0x15e1e8
> *** TEST Best sink name "nv_omx_hdmi_videosink" , m_videoSink: 0x15e5c0
> NvMMLiteOpen : Block : BlockType = 260
> ++++++ NvAvpOpen +++++++
> NvMMLiteBlockCreate : Block : BlockType = 260
> ++++++++++++ TVMRFrameDelivery +++++++++++++++
> BeginSequence 1920x720
> pnvsi->nDecodeBuffers = 3
> Display Resolution : (1920x720)
> Display Aspect Ratio : (1920x720)
> cbBeginSequence at 433: SurfaceLayout = 2
> pStreamInfo->NumOfSurfaces = 7, MaxDPB = 24, InteraceStream = 0,
> InterlaceEnabled = 0
> Allocating new output: 1920x720 (x 9)
> Warning: "A lot of buffers are being dropped."
> Warning: "A lot of buffers are being dropped."
> Warning: "A lot of buffers are being dropped."
> Warning: "A lot of buffers are being dropped."
> ^Croot at apalis-t30:~/Downloads#
> {quote}
>
> where are:
>
> {quote}
> *** TEST Best sink name "nv_omx_hdmi_videosink" , m_videoSink: 0x15e1e8
> *** TEST Best sink name "nv_omx_hdmi_videosink" , m_videoSink: 0x15e5c0
> {quote}
>
> BUT: Nothing changes, the CPU loading ~300% as before... So, seems, the
> problem is *NOT IN VIDEO-SINK* selection...
>
> So, now, I am confused.. WTF? WTF? WTF? :(
>
> BR,
> Denis
>
>
>
> 2016-05-25 16:53 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:
>
>> The solution from:
>>
>> https://devtalk.nvidia.com/default/topic/894891/jetson-tk1/gstreamer-1-0-and-qt5-video-playback/
>>
>> with adding the:
>>
>> {code}
>>         qDebug() << "Setting videosink to " << videoSinkString;
>>         qputenv("QT_GSTREAMER_WINDOW_VIDEOSINK",
>> videoSinkString.toUtf8());
>> {code}
>>
>> to the main.cpp, where videoSinkString is nv_omx_hdmi_videosink, does not
>> work too...
>>
>> WTF?
>>
>>
>>
>> 2016-05-25 15:07 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:
>>
>>> Now, I have recompiled the QtMM plugin (with default gstreamer 0.10),
>>> but with adding of debug macro DEBUG_PLAYBIN.
>>> Then I got following log: https://paste.kde.org/p6ztesnsq
>>>
>>> Where I do not see any mentions about the selected gst sinks.. I see
>>> only this:
>>>
>>> {quote}
>>> Set video output: QGstreamerVideoRenderer(0x153fe0)
>>> Current sink: fakesink0 0x13e590 pending:  0x0 new sink: fakesink0
>>> 0x13e590
>>> Video sink has not changed, skip video output reconfiguration
>>> Video sink has chaged, reload video output
>>> void QGstreamerPlayerSession::setVideoRenderer(QObject*)
>>> Set video output: QGstreamerVideoRenderer(0x153fe0)
>>> Current sink: fakesink0 0x13e590 pending:  0x0 new sink:
>>> qvideosurfacegstsink0 0x1bc470
>>> Reconfigure video output
>>> The pipeline has not started yet, pending state:
>>> QMediaPlayer::StoppedState
>>> Video sink has chaged, reload video output
>>> void QGstreamerPlayerSession::setVideoRenderer(QObject*)
>>> Set video output: QGstreamerVideoRenderer(0x153fe0)
>>> Current sink: qvideosurfacegstsink0 0x1bc470 pending:  0x0 new sink:
>>> qvideosurfacegstsink0 0x1bc470
>>> Video sink has not changed, skip video output reconfiguration
>>> {quote}
>>>
>>> what is it 'fakesink', 'qvideosurfacegstsink0' ? where is any of desired:
>>>
>>> {quote}
>>> root at apalis-t30:~/Downloads# gst-inspect | grep sink | grep nv
>>> omx:  nv_omx_audiosink: OpenMAX IL audiosink element
>>> omx:  nv_gstbin_videosink: OpenMAX IL videosink element
>>> omx:  nv_omx_videosink: OpenMAX IL videosink element
>>> omx:  nv_omx_hdmi_videosink: OpenMAX IL hdmi videosink element
>>> omx:  nv_omx_overlaysink: OpenMAX IL overlaysink element
>>> omx:  nv_gl_eglimagesink: OpenMAX IL videosink element
>>> nvxvimagesink:  nvxvimagesink: Video sink
>>> root at apalis-t30:~/Downloads#
>>> {quote}
>>>
>>> sinks? I do not understand nothing.. o_O How I can see the selected sink
>>> name?
>>>
>>>
>>>
>>>
>>> 2016-05-25 14:35 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:
>>>
>>>> > BTW: As I remember, I saw some environment variable which allows to
>>>> setup custom video sink... I need to see sources of QtMM... if I'm not
>>>> mistaked..
>>>> maybe it will help...
>>>>
>>>> I have found this env variables:
>>>>
>>>> * QT_GSTREAMER_WIDGET_VIDEOSINK
>>>> * QT_GSTREAMER_WINDOW_VIDEOSINK
>>>>
>>>> and try to use it:
>>>>
>>>> $export QT_GSTREAMER_WIDGET_VIDEOSINK=nv_omx_hdmi_videosink
>>>> $export QT_GSTREAMER_WINDOW_VIDEOSINK=nv_omx_hdmi_videosink
>>>> $./my-qml-video-player
>>>>
>>>> but, this does not help...
>>>>
>>>> BR,
>>>> Denis
>>>>
>>>>
>>>>
>>>> 2016-05-25 8:30 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:
>>>>
>>>>> > Gstreamer 1.0+ goes some way to addressing these issues....
>>>>>
>>>>> The Apalis T30 board has GPU from the NVidia. So, when I use
>>>>> 'gst-inspect' I see some of 'nv' sinks.
>>>>> So, as I understand, the easy ways to do it are:
>>>>>
>>>>> 1) I should try to use GStreamer 1.0 (instead of 0.10) "as is" with
>>>>> QtMM (rebuild QtMM with this GStreamer), at first stage.
>>>>>
>>>>> 2) I need try to change QtMM sources and try to set the specific
>>>>> vendor-video sinks there, at second stage... (in case the first stage does
>>>>> not work).
>>>>>
>>>>> Is it?
>>>>>
>>>>> BTW: As I remember, I saw some environment variable which allows to
>>>>> setup custom video sink... I need to see sources of QtMM... if I'm not
>>>>> mistaked..
>>>>> maybe it will help...
>>>>>
>>>>> > but even within the standard packages there still seems to be
>>>>> multiple APIs for codecs which produce EGL surfaces and none of them have
>>>>> been utilized in QtMultimedia yet
>>>>>
>>>>> Could you please provide an examples of this API which is not adopted
>>>>> in QtMM?
>>>>>
>>>>>
>>>>>
>>>>> BR,
>>>>> Denis
>>>>>
>>>>>
>>>>> 25.05.2016 4:44, Andrew den Exter пишет:
>>>>>
>>>>> The short answer is some amount of platform adaptation will be
>>>>> required to enable hardware acceleration on your board.  Looking at the
>>>>> page you linked to; that adaptation might be achieved by creating an
>>>>> implementation of QGstBufferPoolPlugin (gstreamer 0.10 only) which exposes
>>>>> EGL surfaces associated with the x-nv-yuv buffers produced by the
>>>>> accelerated codec.  Or you might also need to create your own video node by
>>>>> implementing QSGVideoNodeFactoryPlugin or something more invasive may
>>>>> be required, it really depends on how the vendor implemented their
>>>>> gstreamer plugins
>>>>>
>>>>> The reason hardware acceleration doesn't just work is that hardware
>>>>> vendors have traditionally had to create extensions to gstreamer in order
>>>>> to implement their hardware accelerated plugins, and these extensions tend
>>>>> to require a vendor specific video sink to work.  If all you care about is
>>>>> rendering to the full dimensions of a native window then autovideosink will
>>>>> generally resolve to that vendor specific video sink and that's why the
>>>>> standard example command line examples tend to work.  But when it comes to
>>>>> something more complicated like compositing into an opengl scene or even as
>>>>> simple as rendering to a sub rect of a window then you're into the realm of
>>>>> vendor extensions and without a port to your specific hardware it probably
>>>>> won't work.  Gstreamer 1.0+ goes some way to addressing these issues, but
>>>>> even within the standard packages there still seems to be multiple APIs for
>>>>> codecs which produce EGL surfaces and none of them have been utilized in
>>>>> QtMultimedia yet.
>>>>>
>>>>>
>>>>> Andrew
>>>>>
>>>>>
>>>>> On Wed, May 25, 2016 at 12:40 AM, Denis Shienkov <
>>>>> denis.shienkov at gmail.com> wrote:
>>>>>
>>>>>> Hi all.
>>>>>>
>>>>>> I have the Toradex (Apalis T30) embedded board:
>>>>>>
>>>>>> https://www.toradex.com/computer-on-modules/apalis-arm-family/nvidia-tegra-3
>>>>>>
>>>>>> which has:
>>>>>>
>>>>>> http://developer.toradex.com/files/toradex-dev/uploads/media/Colibri/Linux/Images/Apalis_T30_LinuxImageV2.6Beta1_20160331.tar.bz2
>>>>>>
>>>>>> Linux Image, with the GStreamer 0.10.
>>>>>>
>>>>>> Also I have compiled Qt 5.6 and installed it to the board.
>>>>>>
>>>>>> When I try to play any *.MP4 video file from the QML Video item:
>>>>>> http://doc.qt.io/qt-5/qml-qtmultimedia-video.html
>>>>>>
>>>>>> I see, that my video has very-very lags, and I see (with htop
>>>>>> utility) that CPU loading is ~300% (
>>>>>> three cores running on ~100%).
>>>>>>
>>>>>> But, when I try to play the video, using the 'nvgstplayer' utility, I
>>>>>> see that CPU loading is ~46% and all fine. Seems, the QML player loads the
>>>>>> SW codecs instead of HW codecs...
>>>>>>
>>>>>> Is there are any way to say to QML player to use a HW codecs? Or,
>>>>>> maybe, I need to fix a sources of QtMM (where I need to change it?) Or,
>>>>>> maybe, is there are another way to do this?
>>>>>>
>>>>>> Otherwise, is just a **HELL** and there is no sense in QML
>>>>>> QtMultimedia at all (But I need QML for anumations). And I do not know, how
>>>>>> to fix it (I'm not expert in GStreamer).
>>>>>>
>>>>>> PS: I have tried the 'gst-launch' pipelines instead of 'nvgstplayer',
>>>>>> as described here:
>>>>>> http://developer.toradex.com/knowledge-base/video-playback-%28linux%29
>>>>>> but it crashes with sigsegv....
>>>>>>
>>>>>> BR,
>>>>>> Denis
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Development mailing list
>>>>>> Development at qt-project.org
>>>>>> http://lists.qt-project.org/mailman/listinfo/development
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20160526/b3b965be/attachment.html>

More information about the Development mailing list