[Development] [QtMultimedia] How to make to use a HW codecs by QML media player on Linux?

Wed May 25 15:53:10 CEST 2016

The solution from:
https://devtalk.nvidia.com/default/topic/894891/jetson-tk1/gstreamer-1-0-and-qt5-video-playback/

with adding the:

{code}
        qDebug() << "Setting videosink to " << videoSinkString;
        qputenv("QT_GSTREAMER_WINDOW_VIDEOSINK", videoSinkString.toUtf8());
{code}

to the main.cpp, where videoSinkString is nv_omx_hdmi_videosink, does not
work too...

WTF?

2016-05-25 15:07 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:

> Now, I have recompiled the QtMM plugin (with default gstreamer 0.10), but
> with adding of debug macro DEBUG_PLAYBIN.
> Then I got following log: https://paste.kde.org/p6ztesnsq
>
> Where I do not see any mentions about the selected gst sinks.. I see only
> this:
>
> {quote}
> Set video output: QGstreamerVideoRenderer(0x153fe0)
> Current sink: fakesink0 0x13e590 pending:  0x0 new sink: fakesink0 0x13e590
> Video sink has not changed, skip video output reconfiguration
> Video sink has chaged, reload video output
> void QGstreamerPlayerSession::setVideoRenderer(QObject*)
> Set video output: QGstreamerVideoRenderer(0x153fe0)
> Current sink: fakesink0 0x13e590 pending:  0x0 new sink:
> qvideosurfacegstsink0 0x1bc470
> Reconfigure video output
> The pipeline has not started yet, pending state: QMediaPlayer::StoppedState
> Video sink has chaged, reload video output
> void QGstreamerPlayerSession::setVideoRenderer(QObject*)
> Set video output: QGstreamerVideoRenderer(0x153fe0)
> Current sink: qvideosurfacegstsink0 0x1bc470 pending:  0x0 new sink:
> qvideosurfacegstsink0 0x1bc470
> Video sink has not changed, skip video output reconfiguration
> {quote}
>
> what is it 'fakesink', 'qvideosurfacegstsink0' ? where is any of desired:
>
> {quote}
> root at apalis-t30:~/Downloads# gst-inspect | grep sink | grep nv
> omx:  nv_omx_audiosink: OpenMAX IL audiosink element
> omx:  nv_gstbin_videosink: OpenMAX IL videosink element
> omx:  nv_omx_videosink: OpenMAX IL videosink element
> omx:  nv_omx_hdmi_videosink: OpenMAX IL hdmi videosink element
> omx:  nv_omx_overlaysink: OpenMAX IL overlaysink element
> omx:  nv_gl_eglimagesink: OpenMAX IL videosink element
> nvxvimagesink:  nvxvimagesink: Video sink
> root at apalis-t30:~/Downloads#
> {quote}
>
> sinks? I do not understand nothing.. o_O How I can see the selected sink
> name?
>
>
>
>
> 2016-05-25 14:35 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:
>
>> > BTW: As I remember, I saw some environment variable which allows to
>> setup custom video sink... I need to see sources of QtMM... if I'm not
>> mistaked..
>> maybe it will help...
>>
>> I have found this env variables:
>>
>> * QT_GSTREAMER_WIDGET_VIDEOSINK
>> * QT_GSTREAMER_WINDOW_VIDEOSINK
>>
>> and try to use it:
>>
>> $export QT_GSTREAMER_WIDGET_VIDEOSINK=nv_omx_hdmi_videosink
>> $export QT_GSTREAMER_WINDOW_VIDEOSINK=nv_omx_hdmi_videosink
>> $./my-qml-video-player
>>
>> but, this does not help...
>>
>> BR,
>> Denis
>>
>>
>>
>> 2016-05-25 8:30 GMT+03:00 Denis Shienkov <denis.shienkov at gmail.com>:
>>
>>> > Gstreamer 1.0+ goes some way to addressing these issues....
>>>
>>> The Apalis T30 board has GPU from the NVidia. So, when I use
>>> 'gst-inspect' I see some of 'nv' sinks.
>>> So, as I understand, the easy ways to do it are:
>>>
>>> 1) I should try to use GStreamer 1.0 (instead of 0.10) "as is" with QtMM
>>> (rebuild QtMM with this GStreamer), at first stage.
>>>
>>> 2) I need try to change QtMM sources and try to set the specific
>>> vendor-video sinks there, at second stage... (in case the first stage does
>>> not work).
>>>
>>> Is it?
>>>
>>> BTW: As I remember, I saw some environment variable which allows to
>>> setup custom video sink... I need to see sources of QtMM... if I'm not
>>> mistaked..
>>> maybe it will help...
>>>
>>> > but even within the standard packages there still seems to be multiple
>>> APIs for codecs which produce EGL surfaces and none of them have been
>>> utilized in QtMultimedia yet
>>>
>>> Could you please provide an examples of this API which is not adopted in
>>> QtMM?
>>>
>>>
>>>
>>> BR,
>>> Denis
>>>
>>>
>>> 25.05.2016 4:44, Andrew den Exter пишет:
>>>
>>> The short answer is some amount of platform adaptation will be required
>>> to enable hardware acceleration on your board.  Looking at the page you
>>> linked to; that adaptation might be achieved by creating an implementation
>>> of QGstBufferPoolPlugin (gstreamer 0.10 only) which exposes EGL surfaces
>>> associated with the x-nv-yuv buffers produced by the accelerated
>>> codec.  Or you might also need to create your own video node by
>>> implementing QSGVideoNodeFactoryPlugin or something more invasive may
>>> be required, it really depends on how the vendor implemented their
>>> gstreamer plugins
>>>
>>> The reason hardware acceleration doesn't just work is that hardware
>>> vendors have traditionally had to create extensions to gstreamer in order
>>> to implement their hardware accelerated plugins, and these extensions tend
>>> to require a vendor specific video sink to work.  If all you care about is
>>> rendering to the full dimensions of a native window then autovideosink will
>>> generally resolve to that vendor specific video sink and that's why the
>>> standard example command line examples tend to work.  But when it comes to
>>> something more complicated like compositing into an opengl scene or even as
>>> simple as rendering to a sub rect of a window then you're into the realm of
>>> vendor extensions and without a port to your specific hardware it probably
>>> won't work.  Gstreamer 1.0+ goes some way to addressing these issues, but
>>> even within the standard packages there still seems to be multiple APIs for
>>> codecs which produce EGL surfaces and none of them have been utilized in
>>> QtMultimedia yet.
>>>
>>>
>>> Andrew
>>>
>>>
>>> On Wed, May 25, 2016 at 12:40 AM, Denis Shienkov <
>>> denis.shienkov at gmail.com> wrote:
>>>
>>>> Hi all.
>>>>
>>>> I have the Toradex (Apalis T30) embedded board:
>>>>
>>>> https://www.toradex.com/computer-on-modules/apalis-arm-family/nvidia-tegra-3
>>>>
>>>> which has:
>>>>
>>>> http://developer.toradex.com/files/toradex-dev/uploads/media/Colibri/Linux/Images/Apalis_T30_LinuxImageV2.6Beta1_20160331.tar.bz2
>>>>
>>>> Linux Image, with the GStreamer 0.10.
>>>>
>>>> Also I have compiled Qt 5.6 and installed it to the board.
>>>>
>>>> When I try to play any *.MP4 video file from the QML Video item:
>>>> http://doc.qt.io/qt-5/qml-qtmultimedia-video.html
>>>>
>>>> I see, that my video has very-very lags, and I see (with htop utility)
>>>> that CPU loading is ~300% (
>>>> three cores running on ~100%).
>>>>
>>>> But, when I try to play the video, using the 'nvgstplayer' utility, I
>>>> see that CPU loading is ~46% and all fine. Seems, the QML player loads the
>>>> SW codecs instead of HW codecs...
>>>>
>>>> Is there are any way to say to QML player to use a HW codecs? Or,
>>>> maybe, I need to fix a sources of QtMM (where I need to change it?) Or,
>>>> maybe, is there are another way to do this?
>>>>
>>>> Otherwise, is just a **HELL** and there is no sense in QML QtMultimedia
>>>> at all (But I need QML for anumations). And I do not know, how to fix it
>>>> (I'm not expert in GStreamer).
>>>>
>>>> PS: I have tried the 'gst-launch' pipelines instead of 'nvgstplayer',
>>>> as described here:
>>>> http://developer.toradex.com/knowledge-base/video-playback-%28linux%29
>>>> but it crashes with sigsegv....
>>>>
>>>> BR,
>>>> Denis
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Development mailing list
>>>> Development at qt-project.org
>>>> http://lists.qt-project.org/mailman/listinfo/development
>>>>
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20160525/f366d65e/attachment.html>