Gstreamer-imx plugin with Buildroot

Our i.MX6 platforms have great multimedia features and this post will describe how to leverage them using the most used multimedia framework Gstreamer along with a Buildroot filesystem.

What is it?

• Gstreamer: This is a pipeline-based multimedia framework written in the C programming language with the type system based on GObject.
• Gstreamer-imx: This is a set of GStreamer 1.0 plugins for Freescale's i.MX platform, which make use of the i.MX multimedia capabilities.
• Buildroot: Buildroot is a nice, simple, and efficient embedded Linux build system. You can learn more about this project by looking at this presentation from the ELC2014: Dive into Buildroot core.

For the impatient

If you just want to boot up Buildroot with gstreamer1, we’ve uploaded an image that will run on our BD-SL-i.MX6 (SABRE Lite), Nitrogen6x or Nitrogen6_Max board.The image contains Freescale-licensed content, so you will need to register on our web-site and log in before you can accept the license agreement and download the image from here:

• 20150331-nitrogen-3.10.17-buildroot-gstreamer1.ext2.gz

Programming the image

Buildroot generates partition files which requires the block device, SDCard or SATA drive, to be partitioned already. Here is a command that allows to create a full size partition to the device:

$ sudo parted -a optimal /dev/sdc mkpart primary 0% 100%
$ sudo partprobe

Once the device has at least one partition of 4GB, the pre-built image can be flashed:

$ sudo umount /dev/sdc*
$ zcat 20150331-nitrogen*.ext2.gz | sudo dd of=/dev/sdc1 bs=1M
$ sync

You can plug in the freshly imaged device to the board and power it up.
Note: The user login is root with no password.

Building from sources

For those interested in building the same image, we provide the procedure to reproduce it:

~$ curl https://buildroot.uclibc.org/downloads/buildroot-2015.02.tar.gz | tar xz
~$ cd buildroot-2015.02
~/buildroot-2015.02$ curl https://storage.googleapis.com/boundarydevices.com/20150331-nitrogen-3.10.17-buildroot-gstreamer1.patch | patch -p1
~/buildroot-2015.02$ curl https://storage.googleapis.com/boundarydevices.com/20150331-nitrogen-3.10.17-buildroot-gstreamer1.config > .config
~/buildroot-2015.02$ make

Note: This patch is currently under review to be part of the next release of Buildroot:
https://article.gmane.org/gmane.comp.lib.uclibc.buildroot/111069The output image will be located in output/images/rootfs.ext2.gz and can be flashed just like the prebuilt image.Also, we invite you to discover the menu Buildroot offers to add and/or modify the target packages:

~/buildroot-2015.02$ make menuconfig

Using Gstreamer

Once booted up, the system allows you to issue any Gstreamer command, our usual video test uses the Tears of Steel movie in 1080p.

$ wget https://media.xiph.org/mango/tears_of_steel_1080p.webm -P /root/

Below are some commands used to test the different i.MX sinks available with the plugin:

$ gst-launch-1.0 playbin uri=file:///root/tears_of_steel_1080p.webm
$ gst-launch-1.0 filesrc location=/root/tears_of_steel_1080p.webm !
matroskademux ! imxvpudec ! imxipuvideosink
$ gst-launch-1.0 filesrc location=/root/tears_of_steel_1080p.webm !
matroskademux ! imxvpudec ! imxeglvivsink

If you have a camera connected to the board, you can also use some of the gstreamer-imx plugin to capture the video:

$ modprobe ov5640_camera_mipi
$ modprobe ov5642_camera
$ gst-launch-1.0 imxv4l2src device=/dev/video0 ! imxeglvivsink

Below is an example of a pipeline leveraging the VPU for the video decoding, the IPU for cropping the video output as well as capturing the stream from the Camera and the GPU that converts and display the Camera stream

$ gst-launch-1.0 playbin uri=file:///root/tears_of_steel_1080p.webm
video-sink="imxipuvideosink window-width=640 window-height=480"
imxv4l2src device=/dev/video0 ! imxeglvivsink window-x-coord=600
window-y-coord=400

Gstreamer cheat sheet

For those not really familiar with Gstreamer and its tools, here is a non-exhaustive list of tools that can help you get started.

gst-inspect

Provides information about a particular plugin

• Very useful to find and debug capabilities
• Useful to see which GObject properties or which signals an element supports

It can list all the plugins present on the target:

$ gst-inspect-1.0
lame: lamemp3enc: L.A.M.E. mp3 encoder
videoscale: videoscale: Video scaler
volume: volume: Volume
videotestsrc: videotestsrc: Video test source
...

Or it can give specifics about one plugin:

$ gst-inspect-1.0 imxvpudec
Factory Details:
 Rank primary + 1 (257)
 Long-name Freescale VPU video decoder
...

gst-launch

Used to create and test pipelines (prototyping).

$ gst-launch-1.0 filesrc location=/root/tears_of_steel_1080p.webm !
matroskademux ! imxvpudec ! imxipuvideosink

This tool will also allow you to easily troubleshoot issues by adding some log using the GST_DEBUG shell variable. It will then change the log level whose value range between 0 and 5 (5 will show all messages).

$ GST_DEBUG=5 gst-launch-1.0 playbin uri=file:///root/tears_of_steel_1080p.webm

You can specify which plugin you want to add traces to:

$ GST_DEBUG=imxvpudec:5 gst-launch-1.0 playbin
uri=file:///root/tears_of_steel_1080p.webm