code:
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later

.. _overlay:

***********************
Video Overlay Interface
***********************

**Also known as Framebuffer Overlay or Previewing.**

Video overlay devices have the ability to genlock (TV-)video into the
(VGA-)video signal of a graphics card, or to store captured images
directly in video memory of a graphics card, typically with clipping.
This can be considerable more efficient than capturing images and
displaying them by other means. In the old days when only nuclear power
plants needed cooling towers this used to be the only way to put live
video into a window.

Video overlay devices are accessed through the same character special
files as :ref:`video capture <capture>` devices.

.. note::

   The default function of a ``/dev/video`` device is video
   capturing. The overlay function is only available after calling
   the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl.

The driver may support simultaneous overlay and capturing using the
read/write and streaming I/O methods. If so, operation at the nominal
frame rate of the video standard is not guaranteed. Frames may be
directed away from overlay to capture, or one field may be used for
overlay and the other for capture if the capture parameters permit this.

Applications should use different file descriptors for capturing and
overlay. This must be supported by all drivers capable of simultaneous
capturing and overlay. Optionally these drivers may also permit
capturing and overlay with a single file descriptor for compatibility
with V4L and earlier versions of V4L2. [#f1]_

A common application of two file descriptors is the X11
:ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application.
While the X server controls video overlay, the application can take
advantage of memory mapping and DMA.

Querying Capabilities
=====================

Devices supporting the video overlay interface set the
``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct
:c:type:`v4l2_capability` returned by the
:ref:`VIDIOC_QUERYCAP` ioctl. The overlay I/O
method specified below must be supported. Tuners and audio inputs are
optional.

Supplemental Functions
======================

Video overlay devices shall support :ref:`audio input <audio>`,
:ref:`tuner`, :ref:`controls <control>`,
:ref:`cropping and scaling <crop>` and
:ref:`streaming parameter <streaming-par>` ioctls as needed. The
:ref:`video input <video>` and :ref:`video standard <standard>`
ioctls must be supported by all video overlay devices.

Setup
=====

Before overlay can commence applications must program the driver with
frame buffer parameters, namely the address and size of the frame buffer
and the image format, for example RGB 5:6:5. The
:ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and
:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and
set these parameters, respectively. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is
privileged because it allows to set up DMA into physical memory,
bypassing the memory protection mechanisms of the kernel. Only the
superuser can change the frame buffer address and size. Users are not
supposed to run TV applications as root or with SUID bit set. A small
helper application with suitable privileges should query the graphics
system and program the V4L2 driver at the appropriate time.

Some devices add the video overlay to the output signal of the graphics
card. In this case the frame buffer is not modified by the video device,
and the frame buffer address and pixel format are not needed by the
driver. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is not privileged. An application
can check for this type of device by calling the :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`
ioctl.

A driver may support any (or none) of five clipping/blending methods:

1. Chroma-keying displays the overlaid image only where pixels in the
   primary graphics surface assume a certain color.

2. A bitmap can be specified where each bit corresponds to a pixel in
   the overlaid image. When the bit is set, the corresponding video
   pixel is displayed, otherwise a pixel of the graphics surface.

3. A list of clipping rectangles can be specified. In these regions *no*
   video is displayed, so the graphics surface can be seen here.

4. The framebuffer has an alpha channel that can be used to clip or
   blend the framebuffer with the video.

5. A global alpha value can be specified to blend the framebuffer
   contents with video images.

When simultaneous capturing and overlay is supported and the hardware
prohibits different image and frame buffer formats, the format requested
first takes precedence. The attempt to capture
(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay
(:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an ``EBUSY`` error
code or return accordingly modified parameters..

Overlay Window
==============

The overlaid image is determined by cropping and overlay window
parameters. The former select an area of the video picture to capture,
the latter how images are overlaid and clipped. Cropping initialization
at minimum requires to reset the parameters to defaults. An example is
given in :ref:`crop`.

The overlay window is described by a struct
:c:type:`v4l2_window`. It defines the size of the image,
its position over the graphics surface and the clipping to be applied.
To get the current parameters applications set the ``type`` field of a
struct :c:type:`v4l2_format` to
``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the
:ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the
struct :c:type:`v4l2_window` substructure named ``win``. It is not
possible to retrieve a previously programmed clipping list or bitmap.

To program the overlay window applications set the ``type`` field of a
struct :c:type:`v4l2_format` to
``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and
call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver
adjusts the parameters against hardware limits and returns the actual
parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does. Like :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`, the
:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn
about driver capabilities without actually changing driver state. Unlike
:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` this also works after the overlay has been enabled.

The scaling factor of the overlaid image is implied by the width and
height given in struct :c:type:`v4l2_window` and the size
of the cropping rectangle. For more information see :ref:`crop`.

When simultaneous capturing and overlay is supported and the hardware
prohibits different image and window sizes, the size requested first
takes precedence. The attempt to capture or overlay as well
(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an ``EBUSY`` error
code or return accordingly modified parameters.

.. c:type:: v4l2_window

struct v4l2_window
------------------

``struct v4l2_rect w``
    Size and position of the window relative to the top, left corner of
    the frame buffer defined with
    :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the
    frame buffer width and height, the ``x`` and ``y`` coordinates can
    be negative, and it can lie completely outside the frame buffer. The
    driver clips the window accordingly, or if that is not possible,
    modifies its size and/or position.

``enum v4l2_field field``
    Applications set this field to determine which video field shall be
    overlaid, typically one of ``V4L2_FIELD_ANY`` (0),
    ``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or
    ``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different
    field order and return the actual setting here.

``__u32 chromakey``
    When chroma-keying has been negotiated with
    :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field
    to the desired pixel value for the chroma key. The format is the
    same as the pixel format of the framebuffer (struct
    :c:type:`v4l2_framebuffer` ``fmt.pixelformat``
    field), with bytes in host order. E. g. for
    :ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should
    be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host.

``struct v4l2_clip * clips``
    When chroma-keying has *not* been negotiated and
    :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
    applications can set this field to point to an array of clipping
    rectangles.

    Like the window coordinates w, clipping rectangles are defined
    relative to the top, left corner of the frame buffer. However
    clipping rectangles must not extend the frame buffer width and
    height, and they must not overlap. If possible applications
    should merge adjacent rectangles. Whether this must create
    x-y or y-x bands, or the order of rectangles, is not defined. When
    clip lists are not supported the driver ignores this field. Its
    contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`
    are undefined.

``__u32 clipcount``
    When the application set the ``clips`` field, this field must
    contain the number of clipping rectangles in the list. When clip
    lists are not supported the driver ignores this field, its contents
    after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are undefined. When clip lists are
    supported but no clipping is desired this field must be set to zero.

``void * bitmap``
    When chroma-keying has *not* been negotiated and
    :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
    applications can set this field to point to a clipping bit mask.

It must be of the same size as the window, ``w.width`` and ``w.height``.
Each bit corresponds to a pixel in the overlaid image, which is
displayed only when the bit is *set*. Pixel coordinates translate to
bits like:

.. code-block:: c

    ((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7))

where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [#f2]_

When a clipping bit mask is not supported the driver ignores this field,
its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are
undefined. When a bit mask is supported but no clipping is desired this
field must be set to ``NULL``.

Applications need not create a clip list or bit mask. When they pass
both, or despite negotiating chroma-keying, the results are undefined.
Regardless of the chosen method, the clipping abilities of the hardware
may be limited in quantity or quality. The results when these limits are
exceeded are undefined. [#f3]_

``__u8 global_alpha``
    The global alpha value used to blend the framebuffer with video
    images, if global alpha blending has been negotiated
    (``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see
    :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`,
    :ref:`framebuffer-flags`).

.. note::

   This field was added in Linux 2.6.23, extending the
   structure. However the :ref:`VIDIOC_[G|S|TRY]_FMT <VIDIOC_G_FMT>`
   ioctls, which take a pointer to a :c:type:`v4l2_format`
   parent structure with padding bytes at the end, are not affected.

.. c:type:: v4l2_clip

struct v4l2_clip [#f4]_
-----------------------

``struct v4l2_rect c``
    Coordinates of the clipping rectangle, relative to the top, left
    corner of the frame buffer. Only window pixels *outside* all
    clipping rectangles are displayed.

``struct v4l2_clip * next``
    Pointer to the next clipping rectangle, ``NULL`` when this is the last
    rectangle. Drivers ignore this field, it cannot be used to pass a
    linked list of clipping rectangles.

.. c:type:: v4l2_rect

struct v4l2_rect
----------------

``__s32 left``
    Horizontal offset of the top, left corner of the rectangle, in
    pixels.

``__s32 top``
    Vertical offset of the top, left corner of the rectangle, in pixels.
    Offsets increase to the right and down.

``__u32 width``
    Width of the rectangle, in pixels.

``__u32 height``
    Height of the rectangle, in pixels.

Enabling Overlay
================

To start or stop the frame buffer overlay applications call the
:ref:`VIDIOC_OVERLAY` ioctl.

.. [#f1]
   In the opinion of the designers of this API, no driver writer taking
   the efforts to support simultaneous capturing and overlay will
   restrict this ability by requiring a single file descriptor, as in
   V4L and earlier versions of V4L2. Making this optional means
   applications depending on two file descriptors need backup routines
   to be compatible with all drivers, which is considerable more work
   than using two fds in applications which do not. Also two fd's fit
   the general concept of one file descriptor for each logical stream.
   Hence as a complexity trade-off drivers *must* support two file
   descriptors and *may* support single fd operation.

.. [#f2]
   Should we require ``w.width`` to be a multiple of eight?

.. [#f3]
   When the image is written into frame buffer memory it will be
   undesirable if the driver clips out less pixels than expected,
   because the application and graphics system are not aware these
   regions need to be refreshed. The driver should clip out more pixels
   or not write the image at all.

.. [#f4]
   The X Window system defines "regions" which are vectors of ``struct
   BoxRec { short x1, y1, x2, y2; }`` with ``width = x2 - x1`` and
   ``height = y2 - y1``, so one cannot pass X11 clip lists directly.


code:
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later

.. _radio:

***************
Radio Interface
***************

This interface is intended for AM and FM (analog) radio receivers and
transmitters.

Conventionally V4L2 radio devices are accessed through character device
special files named ``/dev/radio`` and ``/dev/radio0`` to
``/dev/radio63`` with major number 81 and minor numbers 64 to 127.

Querying Capabilities
=====================

Devices supporting the radio interface set the ``V4L2_CAP_RADIO`` and
``V4L2_CAP_TUNER`` or ``V4L2_CAP_MODULATOR`` flag in the
``capabilities`` field of struct
:c:type:`v4l2_capability` returned by the
:ref:`VIDIOC_QUERYCAP` ioctl. Other combinations of
capability flags are reserved for future extensions.

Supplemental Functions
======================

Radio devices can support :ref:`controls <control>`, and must support
the :ref:`tuner or modulator <tuner>` ioctls.

They do not support the video input or output, audio input or output,
video standard, cropping and scaling, compression and streaming
parameter, or overlay ioctls. All other ioctls and I/O methods are
reserved for future extensions.

Programming
===========

Radio devices may have a couple audio controls (as discussed in
:ref:`control`) such as a volume control, possibly custom controls.
Further all radio devices have one tuner or modulator (these are
discussed in :ref:`tuner`) with index number zero to select the radio
frequency and to determine if a monaural or FM stereo program is
received/emitted. Drivers switch automatically between AM and FM
depending on the selected frequency. The
:ref:`VIDIOC_G_TUNER <VIDIOC_G_TUNER>` or
:ref:`VIDIOC_G_MODULATOR <VIDIOC_G_MODULATOR>` ioctl reports the
supported frequency range.

repo_name: angery/linux


code:
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
.. c:namespace:: V4L

.. _raw-vbi:

**********************
Raw VBI Data Interface
**********************

VBI is an abbreviation of Vertical Blanking Interval, a gap in the
sequence of lines of an analog video signal. During VBI no picture
information is transmitted, allowing some time while the electron beam
of a cathode ray tube TV returns to the top of the screen. Using an
oscilloscope you will find here the vertical synchronization pulses and
short data packages ASK modulated [#f1]_ onto the video signal. These are
transmissions of services such as Teletext or Closed Caption.

Subject of this interface type is raw VBI data, as sampled off a video
signal, or to be added to a signal for output. The data format is
similar to uncompressed video images, a number of lines times a number
of samples per line, we call this a VBI image.

Conventionally V4L2 VBI devices are accessed through character device
special files named ``/dev/vbi`` and ``/dev/vbi0`` to ``/dev/vbi31``
with major number 81 and minor numbers 224 to 255. ``/dev/vbi`` is
typically a symbolic link to the preferred VBI device. This convention
applies to both input and output devices.

To address the problems of finding related video and VBI devices VBI
capturing and output is also available as device function under
``/dev/video``. To capture or output raw VBI data with these devices
applications must call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl.
Accessed as ``/dev/vbi``, raw VBI capturing or output is the default
device function.

Querying Capabilities
=====================

Devices supporting the raw VBI capturing or output API set the
``V4L2_CAP_VBI_CAPTURE`` or ``V4L2_CAP_VBI_OUTPUT`` flags, respectively,
in the ``capabilities`` field of struct
:c:type:`v4l2_capability` returned by the
:ref:`VIDIOC_QUERYCAP` ioctl. At least one of the
read/write or streaming I/O methods must be supported. VBI
devices may or may not have a tuner or modulator.

Supplemental Functions
======================

VBI devices shall support :ref:`video input or output <video>`,
:ref:`tuner or modulator <tuner>`, and :ref:`controls <control>`
ioctls as needed. The :ref:`video standard <standard>` ioctls provide
information vital to program a VBI device, therefore must be supported.

Raw VBI Format Negotiation
==========================

Raw VBI sampling abilities can vary, in particular the sampling
frequency. To properly interpret the data V4L2 specifies an ioctl to
query the sampling parameters. Moreover, to allow for some flexibility
applications can also suggest different parameters.

As usual these parameters are *not* reset at :c:func:`open()`
time to permit Unix tool chains, programming a device and then reading
from it as if it was a plain file. Well written V4L2 applications should
always ensure they really get what they want, requesting reasonable
parameters and then checking if the actual parameters are suitable.

To query the current raw VBI capture parameters applications set the
``type`` field of a struct :c:type:`v4l2_format` to
``V4L2_BUF_TYPE_VBI_CAPTURE`` or ``V4L2_BUF_TYPE_VBI_OUTPUT``, and call
the :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl with a pointer to this
structure. Drivers fill the struct
:c:type:`v4l2_vbi_format` ``vbi`` member of the
``fmt`` union.

To request different parameters applications set the ``type`` field of a
struct :c:type:`v4l2_format` as above and initialize all
fields of the struct :c:type:`v4l2_vbi_format`
``vbi`` member of the ``fmt`` union, or better just modify the results
of :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>`, and call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`
ioctl with a pointer to this structure. Drivers return an ``EINVAL`` error
code only when the given parameters are ambiguous, otherwise they modify
the parameters according to the hardware capabilities and return the
actual parameters. When the driver allocates resources at this point, it
may return an ``EBUSY`` error code to indicate the returned parameters are
valid but the required resources are currently not available. That may
happen for instance when the video and VBI areas to capture would
overlap, or when the driver supports multiple opens and another process
already requested VBI capturing or output. Anyway, applications must
expect other resource allocation points which may return ``EBUSY``, at the
:ref:`VIDIOC_STREAMON` ioctl and the first :c:func:`read()`
, :c:func:`write()` and :c:func:`select()` calls.

VBI devices must implement both the :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` and
:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl, even if :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ignores all requests
and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does.
:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` is optional.

.. tabularcolumns:: |p{1.6cm}|p{4.2cm}|p{11.5cm}|

.. c:type:: v4l2_vbi_format

.. cssclass:: longtable

.. flat-table:: struct v4l2_vbi_format
    :header-rows:  0
    :stub-columns: 0
    :widths:       1 1 2

    * - __u32
      - ``sampling_rate``
      - Samples per second, i. e. unit 1 Hz.
    * - __u32
      - ``offset``
      - Horizontal offset of the VBI image, relative to the leading edge
	of the line synchronization pulse and counted in samples: The
	first sample in the VBI image will be located ``offset`` /
	``sampling_rate`` seconds following the leading edge. See also
	:ref:`vbi-hsync`.
    * - __u32
      - ``samples_per_line``
      -
    * - __u32
      - ``sample_format``
      - Defines the sample format as in :ref:`pixfmt`, a
	four-character-code. [#f2]_ Usually this is ``V4L2_PIX_FMT_GREY``,
	i. e. each sample consists of 8 bits with lower values oriented
	towards the black level. Do not assume any other correlation of
	values with the signal level. For example, the MSB does not
	necessarily indicate if the signal is 'high' or 'low' because 128
	may not be the mean value of the signal. Drivers shall not convert
	the sample format by software.
    * - __u32
      - ``start``\ [#f2]_
      - This is the scanning system line number associated with the first
	line of the VBI image, of the first and the second field
	respectively. See :ref:`vbi-525` and :ref:`vbi-625` for valid
	values. The ``V4L2_VBI_ITU_525_F1_START``,
	``V4L2_VBI_ITU_525_F2_START``, ``V4L2_VBI_ITU_625_F1_START`` and
	``V4L2_VBI_ITU_625_F2_START`` defines give the start line numbers
	for each field for each 525 or 625 line format as a convenience.
	Don't forget that ITU line numbering starts at 1, not 0. VBI input
	drivers can return start values 0 if the hardware cannot reliable
	identify scanning lines, VBI acquisition may not require this
	information.
    * - __u32
      - ``count``\ [#f2]_
      - The number of lines in the first and second field image,
	respectively.
    * - :cspan:`2`

	Drivers should be as flexibility as possible. For example, it may
	be possible to extend or move the VBI capture window down to the
	picture area, implementing a 'full field mode' to capture data
	service transmissions embedded in the picture.

	An application can set the first or second ``count`` value to zero
	if no data is required from the respective field; ``count``\ [1]
	if the scanning system is progressive, i. e. not interlaced. The
	corresponding start value shall be ignored by the application and
	driver. Anyway, drivers may not support single field capturing and
	return both count values non-zero.

	Both ``count`` values set to zero, or line numbers are outside the
	bounds depicted\ [#f4]_, or a field image covering lines of two
	fields, are invalid and shall not be returned by the driver.

	To initialize the ``start`` and ``count`` fields, applications
	must first determine the current video standard selection. The
	:ref:`v4l2_std_id <v4l2-std-id>` or the ``framelines`` field
	of struct :c:type:`v4l2_standard` can be evaluated
	for this purpose.
    * - __u32
      - ``flags``
      - See :ref:`vbifmt-flags` below. Currently only drivers set flags,
	applications must set this field to zero.
    * - __u32
      - ``reserved``\ [#f2]_
      - This array is reserved for future extensions. Drivers and
	applications must set it to zero.

.. tabularcolumns:: |p{4.4cm}|p{1.5cm}|p{11.4cm}|

.. _vbifmt-flags:

.. flat-table:: Raw VBI Format Flags
    :header-rows:  0
    :stub-columns: 0
    :widths:       3 1 4

    * - ``V4L2_VBI_UNSYNC``
      - 0x0001
      - This flag indicates hardware which does not properly distinguish
	between fields. Normally the VBI image stores the first field
	(lower scanning line numbers) first in memory. This may be a top
	or bottom field depending on the video standard. When this flag is
	set the first or second field may be stored first, however the
	fields are still in correct temporal order with the older field
	first in memory. [#f3]_
    * - ``V4L2_VBI_INTERLACED``
      - 0x0002
      - By default the two field images will be passed sequentially; all
	lines of the first field followed by all lines of the second field
	(compare :ref:`field-order` ``V4L2_FIELD_SEQ_TB`` and
	``V4L2_FIELD_SEQ_BT``, whether the top or bottom field is first in
	memory depends on the video standard). When this flag is set, the
	two fields are interlaced (cf. ``V4L2_FIELD_INTERLACED``). The
	first line of the first field followed by the first line of the
	second field, then the two second lines, and so on. Such a layout
	may be necessary when the hardware has been programmed to capture
	or output interlaced video images and is unable to separate the
	fields for VBI capturing at the same time. For simplicity setting
	this flag implies that both ``count`` values are equal and
	non-zero.

.. _vbi-hsync:

.. kernel-figure:: vbi_hsync.svg
    :alt:   vbi_hsync.svg
    :align: center

    **Figure 4.1. Line synchronization**

.. _vbi-525:

.. kernel-figure:: vbi_525.svg
    :alt:   vbi_525.svg
    :align: center

    **Figure 4.2. ITU-R 525 line numbering (M/NTSC and M/PAL)**

.. _vbi-625:

.. kernel-figure:: vbi_625.svg
    :alt:   vbi_625.svg
    :align: center

    **Figure 4.3. ITU-R 625 line numbering**

Remember the VBI image format depends on the selected video standard,
therefore the application must choose a new standard or query the
current standard first. Attempts to read or write data ahead of format
negotiation, or after switching the video standard which may invalidate
the negotiated VBI parameters, should be refused by the driver. A format
change during active I/O is not permitted.

Reading and writing VBI images
==============================

To assure synchronization with the field number and easier
implementation, the smallest unit of data passed at a time is one frame,
consisting of two fields of VBI images immediately following in memory.

The total size of a frame computes as follows:

.. code-block:: c

    (count[0] + count[1]) * samples_per_line * sample size in bytes

The sample size is most likely always one byte, applications must check
the ``sample_format`` field though, to function properly with other
drivers.

A VBI device may support :ref:`read/write <rw>` and/or streaming
(:ref:`memory mapping <mmap>` or :ref:`user pointer <userp>`) I/O.
The latter bears the possibility of synchronizing video and VBI data by
using buffer timestamps.

Remember the :ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>` ioctl and the
first :c:func:`read()`, :c:func:`write()` and
:c:func:`select()` call can be resource allocation
points returning an ``EBUSY`` error code if the required hardware resources
are temporarily unavailable, for example the device is already in use by
another process.

.. [#f1]
   ASK: Amplitude-Shift Keying. A high signal level represents a '1'
   bit, a low level a '0' bit.

.. [#f2]
   A few devices may be unable to sample VBI data at all but can extend
   the video capture window to the VBI region.

.. [#f3]
   Most VBI services transmit on both fields, but some have different
   semantics depending on the field number. These cannot be reliable
   decoded or encoded when ``V4L2_VBI_UNSYNC`` is set.

.. [#f4]
   The valid values ar shown at :ref:`vbi-525` and :ref:`vbi-625`.


code:
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
.. c:namespace:: V4L

.. _rds:

*************
RDS Interface
*************

The Radio Data System transmits supplementary information in binary
format, for example the station name or travel information, on an
inaudible audio subcarrier of a radio program. This interface is aimed
at devices capable of receiving and/or transmitting RDS information.

For more information see the core RDS standard :ref:`iec62106` and the
RBDS standard :ref:`nrsc4`.

.. note::

   Note that the RBDS standard as is used in the USA is almost
   identical to the RDS standard. Any RDS decoder/encoder can also handle
   RBDS. Only some of the fields have slightly different meanings. See the
   RBDS standard for more information.

The RBDS standard also specifies support for MMBS (Modified Mobile
Search). This is a proprietary format which seems to be discontinued.
The RDS interface does not support this format. Should support for MMBS
(or the so-called 'E blocks' in general) be needed, then please contact
the linux-media mailing list:
`https://linuxtv.org/lists.php <https://linuxtv.org/lists.php>`__.

Querying Capabilities
=====================

Devices supporting the RDS capturing API set the
``V4L2_CAP_RDS_CAPTURE`` flag in the ``capabilities`` field of struct
:c:type:`v4l2_capability` returned by the
:ref:`VIDIOC_QUERYCAP` ioctl. Any tuner that
supports RDS will set the ``V4L2_TUNER_CAP_RDS`` flag in the
``capability`` field of struct :c:type:`v4l2_tuner`. If the
driver only passes RDS blocks without interpreting the data the
``V4L2_TUNER_CAP_RDS_BLOCK_IO`` flag has to be set, see
:ref:`Reading RDS data <reading-rds-data>`. For future use the flag
``V4L2_TUNER_CAP_RDS_CONTROLS`` has also been defined. However, a driver
for a radio tuner with this capability does not yet exist, so if you are
planning to write such a driver you should discuss this on the
linux-media mailing list:
`https://linuxtv.org/lists.php <https://linuxtv.org/lists.php>`__.

Whether an RDS signal is present can be detected by looking at the
``rxsubchans`` field of struct :c:type:`v4l2_tuner`: the
``V4L2_TUNER_SUB_RDS`` will be set if RDS data was detected.

Devices supporting the RDS output API set the ``V4L2_CAP_RDS_OUTPUT``
flag in the ``capabilities`` field of struct
:c:type:`v4l2_capability` returned by the
:ref:`VIDIOC_QUERYCAP` ioctl. Any modulator that
supports RDS will set the ``V4L2_TUNER_CAP_RDS`` flag in the
``capability`` field of struct
:c:type:`v4l2_modulator`. In order to enable the RDS
transmission one must set the ``V4L2_TUNER_SUB_RDS`` bit in the
``txsubchans`` field of struct
:c:type:`v4l2_modulator`. If the driver only passes RDS
blocks without interpreting the data the ``V4L2_TUNER_CAP_RDS_BLOCK_IO``
flag has to be set. If the tuner is capable of handling RDS entities
like program identification codes and radio text, the flag
``V4L2_TUNER_CAP_RDS_CONTROLS`` should be set, see
:ref:`Writing RDS data <writing-rds-data>` and
:ref:`FM Transmitter Control Reference <fm-tx-controls>`.

.. _reading-rds-data:

Reading RDS data
================

RDS data can be read from the radio device with the
:c:func:`read()` function. The data is packed in groups of
three bytes.

.. _writing-rds-data:

Writing RDS data
================

RDS data can be written to the radio device with the
:c:func:`write()` function. The data is packed in groups of
three bytes, as follows:

RDS datastructures
==================

.. c:type:: v4l2_rds_data

.. flat-table:: struct v4l2_rds_data
    :header-rows:  0
    :stub-columns: 0
    :widths:       1 1 5

    * - __u8
      - ``lsb``
      - Least Significant Byte of RDS Block
    * - __u8
      - ``msb``
      - Most Significant Byte of RDS Block
    * - __u8
      - ``block``
      - Block description

.. _v4l2-rds-block:

.. tabularcolumns:: |p{2.9cm}|p{14.6cm}|

.. flat-table:: Block description
    :header-rows:  0
    :stub-columns: 0
    :widths:       1 5

    * - Bits 0-2
      - Block (aka offset) of the received data.
    * - Bits 3-5
      - Deprecated. Currently identical to bits 0-2. Do not use these
	bits.
    * - Bit 6
      - Corrected bit. Indicates that an error was corrected for this data
	block.
    * - Bit 7
      - Error bit. Indicates that an uncorrectable error occurred during
	reception of this block.

.. _v4l2-rds-block-codes:

.. tabularcolumns:: |p{6.4cm}|p{2.0cm}|p{1.2cm}|p{7.0cm}|

.. flat-table:: Block defines
    :header-rows:  0
    :stub-columns: 0
    :widths:       1 1 1 5

    * - V4L2_RDS_BLOCK_MSK
      -
      - 7
      - Mask for bits 0-2 to get the block ID.
    * - V4L2_RDS_BLOCK_A
      -
      - 0
      - Block A.
    * - V4L2_RDS_BLOCK_B
      -
      - 1
      - Block B.
    * - V4L2_RDS_BLOCK_C
      -
      - 2
      - Block C.
    * - V4L2_RDS_BLOCK_D
      -
      - 3
      - Block D.
    * - V4L2_RDS_BLOCK_C_ALT
      -
      - 4
      - Block C'.
    * - V4L2_RDS_BLOCK_INVALID
      - read-only
      - 7
      - An invalid block.
    * - V4L2_RDS_BLOCK_CORRECTED
      - read-only
      - 0x40
      - A bit error was detected but corrected.
    * - V4L2_RDS_BLOCK_ERROR
      - read-only
      - 0x80
      - An uncorrectable error occurred.


code:
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later

.. _sdr:

**************************************
Software Defined Radio Interface (SDR)
**************************************

SDR is an abbreviation of Software Defined Radio, the radio device which
uses application software for modulation or demodulation. This interface
is intended for controlling and data streaming of such devices.

SDR devices are accessed through character device special files named
``/dev/swradio0`` to ``/dev/swradio255`` with major number 81 and
dynamically allocated minor numbers 0 to 255.

Querying Capabilities
=====================

Devices supporting the SDR receiver interface set the
``V4L2_CAP_SDR_CAPTURE`` and ``V4L2_CAP_TUNER`` flag in the
``capabilities`` field of struct
:c:type:`v4l2_capability` returned by the
:ref:`VIDIOC_QUERYCAP` ioctl. That flag means the
device has an Analog to Digital Converter (ADC), which is a mandatory
element for the SDR receiver.

Devices supporting the SDR transmitter interface set the
``V4L2_CAP_SDR_OUTPUT`` and ``V4L2_CAP_MODULATOR`` flag in the
``capabilities`` field of struct
:c:type:`v4l2_capability` returned by the
:ref:`VIDIOC_QUERYCAP` ioctl. That flag means the
device has an Digital to Analog Converter (DAC), which is a mandatory
element for the SDR transmitter.

At least one of the read/write or streaming I/O methods
must be supported.

Supplemental Functions
======================

SDR devices can support :ref:`controls <control>`, and must support
the :ref:`tuner` ioctls. Tuner ioctls are used for setting the
ADC/DAC sampling rate (sampling frequency) and the possible radio
frequency (RF).

The ``V4L2_TUNER_SDR`` tuner type is used for setting SDR device ADC/DAC
frequency, and the ``V4L2_TUNER_RF`` tuner type is used for setting
radio frequency. The tuner index of the RF tuner (if any) must always
follow the SDR tuner index. Normally the SDR tuner is #0 and the RF
tuner is #1.

The :ref:`VIDIOC_S_HW_FREQ_SEEK` ioctl is
not supported.

Data Format Negotiation
=======================

The SDR device uses the :ref:`format` ioctls to select the
capture and output format. Both the sampling resolution and the data
streaming format are bound to that selectable format. In addition to the
basic :ref:`format` ioctls, the
:ref:`VIDIOC_ENUM_FMT` ioctl must be supported as
well.

To use the :ref:`format` ioctls applications set the ``type``
field of a struct :c:type:`v4l2_format` to
``V4L2_BUF_TYPE_SDR_CAPTURE`` or ``V4L2_BUF_TYPE_SDR_OUTPUT`` and use
the struct :c:type:`v4l2_sdr_format` ``sdr`` member
of the ``fmt`` union as needed per the desired operation. Currently
there are two fields, ``pixelformat`` and ``buffersize``, of
struct :c:type:`v4l2_sdr_format` which are used.
Content of the ``pixelformat`` is V4L2 FourCC code of the data format.
The ``buffersize`` field is maximum buffer size in bytes required for
data transfer, set by the driver in order to inform application.

.. c:type:: v4l2_sdr_format

.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.5cm}|

.. flat-table:: struct v4l2_sdr_format
    :header-rows:  0
    :stub-columns: 0
    :widths:       1 1 2

    * - __u32
      - ``pixelformat``
      - The data format or type of compression, set by the application.
	This is a little endian
	:ref:`four character code <v4l2-fourcc>`. V4L2 defines SDR
	formats in :ref:`sdr-formats`.
    * - __u32
      - ``buffersize``
      - Maximum size in bytes required for data. Value is set by the
	driver.
    * - __u8
      - ``reserved[24]``
      - This array is reserved for future extensions. Drivers and
	applications must set it to zero.

An SDR device may support :ref:`read/write <rw>` and/or streaming
(:ref:`memory mapping <mmap>` or :ref:`user pointer <userp>`) I/O.


code:
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
.. c:namespace:: V4L

.. _sliced:

*************************
Sliced VBI Data Interface
*************************

VBI stands for Vertical Blanking Interval, a gap in the sequence of
lines of an analog video signal. During VBI no picture information is
transmitted, allowing some time while the electron beam of a cathode ray
tube TV returns to the top of the screen.

Sliced VBI devices use hardware to demodulate data transmitted in the
VBI. V4L2 drivers shall *not* do this by software, see also the
:ref:`raw VBI interface <raw-vbi>`. The data is passed as short
packets of fixed size, covering one scan line each. The number of
packets per video frame is variable.

Sliced VBI capture and output devices are accessed through the same
character special files as raw VBI devices. When a driver supports both
interfaces, the default function of a ``/dev/vbi`` device is *raw* VBI
capturing or output, and the sliced VBI function is only available after
calling the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl as defined
below. Likewise a ``/dev/video`` device may support the sliced VBI API,
however the default function here is video capturing or output.
Different file descriptors must be used to pass raw and sliced VBI data
simultaneously, if this is supported by the driver.

Querying Capabilities
=====================

Devices supporting the sliced VBI capturing or output API set the
``V4L2_CAP_SLICED_VBI_CAPTURE`` or ``V4L2_CAP_SLICED_VBI_OUTPUT`` flag
respectively, in the ``capabilities`` field of struct
:c:type:`v4l2_capability` returned by the
:ref:`VIDIOC_QUERYCAP` ioctl. At least one of the
read/write or streaming :ref:`I/O methods <io>` must be
supported. Sliced VBI devices may have a tuner or modulator.

Supplemental Functions
======================

Sliced VBI devices shall support :ref:`video input or output <video>`
and :ref:`tuner or modulator <tuner>` ioctls if they have these
capabilities, and they may support :ref:`control` ioctls.
The :ref:`video standard <standard>` ioctls provide information vital
to program a sliced VBI device, therefore must be supported.

.. _sliced-vbi-format-negotitation:

Sliced VBI Format Negotiation
=============================

To find out which data services are supported by the hardware
applications can call the
:ref:`VIDIOC_G_SLICED_VBI_CAP <VIDIOC_G_SLICED_VBI_CAP>` ioctl.
All drivers implementing the sliced VBI interface must support this
ioctl. The results may differ from those of the
:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl when the number of VBI
lines the hardware can capture or output per frame, or the number of
services it can identify on a given line are limited. For example on PAL
line 16 the hardware may be able to look for a VPS or Teletext signal,
but not both at the same time.

To determine the currently selected services applications set the
``type`` field of struct :c:type:`v4l2_format` to
``V4L2_BUF_TYPE_SLICED_VBI_CAPTURE`` or
``V4L2_BUF_TYPE_SLICED_VBI_OUTPUT``, and the
:ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl fills the ``fmt.sliced``
member, a struct
:c:type:`v4l2_sliced_vbi_format`.

Applications can request different parameters by initializing or
modifying the ``fmt.sliced`` member and calling the
:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl with a pointer to the
struct :c:type:`v4l2_format` structure.

The sliced VBI API is more complicated than the raw VBI API because the
hardware must be told which VBI service to expect on each scan line. Not
all services may be supported by the hardware on all lines (this is
especially true for VBI output where Teletext is often unsupported and
other services can only be inserted in one specific line). In many
cases, however, it is sufficient to just set the ``service_set`` field
to the required services and let the driver fill the ``service_lines``
array according to hardware capabilities. Only if more precise control
is needed should the programmer set the ``service_lines`` array
explicitly.

The :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl modifies the parameters
according to hardware capabilities. When the driver allocates resources
at this point, it may return an ``EBUSY`` error code if the required
resources are temporarily unavailable. Other resource allocation points
which may return ``EBUSY`` can be the
:ref:`VIDIOC_STREAMON` ioctl and the first
:c:func:`read()`, :c:func:`write()` and
:c:func:`select()` call.

.. c:type:: v4l2_sliced_vbi_format

struct v4l2_sliced_vbi_format
-----------------------------

.. raw:: latex

    \begingroup
    \scriptsize
    \setlength{\tabcolsep}{2pt}

.. tabularcolumns:: |p{.85cm}|p{3.3cm}|p{4.45cm}|p{4.45cm}|p{4.45cm}|

.. cssclass:: longtable

.. flat-table::
    :header-rows:  0
    :stub-columns: 0
    :widths:       3 3 2 2 2

    * - __u16
      - ``service_set``
      - :cspan:`2`

	If ``service_set`` is non-zero when passed with
	:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` or
	:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>`, the ``service_lines``
	array will be filled by the driver according to the services
	specified in this field. For example, if ``service_set`` is
	initialized with ``V4L2_SLICED_TELETEXT_B | V4L2_SLICED_WSS_625``,
	a driver for the cx25840 video decoder sets lines 7-22 of both
	fields [#f1]_ to ``V4L2_SLICED_TELETEXT_B`` and line 23 of the first
	field to ``V4L2_SLICED_WSS_625``. If ``service_set`` is set to
	zero, then the values of ``service_lines`` will be used instead.

	On return the driver sets this field to the union of all elements
	of the returned ``service_lines`` array. It may contain less
	services than requested, perhaps just one, if the hardware cannot
	handle more services simultaneously. It may be empty (zero) if
	none of the requested services are supported by the hardware.
    * - __u16
      - ``service_lines``\ [2][24]
      - :cspan:`2`

	Applications initialize this array with sets of data services the
	driver shall look for or insert on the respective scan line.
	Subject to hardware capabilities drivers return the requested set,
	a subset, which may be just a single service, or an empty set.
	When the hardware cannot handle multiple services on the same line
	the driver shall choose one. No assumptions can be made on which
	service the driver chooses.

	Data services are defined in :ref:`vbi-services2`. Array indices
	map to ITU-R line numbers\ [#f2]_ as follows:
    * -
      -
      - Element
      - 525 line systems
      - 625 line systems
    * -
      -
      - ``service_lines``\ [0][1]
      - 1
      - 1
    * -
      -
      - ``service_lines``\ [0][23]
      - 23
      - 23
    * -
      -
      - ``service_lines``\ [1][1]
      - 264
      - 314
    * -
      -
      - ``service_lines``\ [1][23]
      - 286
      - 336
    * -
      -
      - :cspan:`2` Drivers must set ``service_lines`` [0][0] and
	``service_lines``\ [1][0] to zero. The
	``V4L2_VBI_ITU_525_F1_START``, ``V4L2_VBI_ITU_525_F2_START``,
	``V4L2_VBI_ITU_625_F1_START`` and ``V4L2_VBI_ITU_625_F2_START``
	defines give the start line numbers for each field for each 525 or
	625 line format as a convenience. Don't forget that ITU line
	numbering starts at 1, not 0.
    * - __u32
      - ``io_size``
      - :cspan:`2` Maximum number of bytes passed by one
	:c:func:`read()` or :c:func:`write()` call,
	and the buffer size in bytes for the
	:ref:`VIDIOC_QBUF` and
	:ref:`VIDIOC_DQBUF <VIDIOC_QBUF>` ioctl. Drivers set this field
	to the size of struct
	:c:type:`v4l2_sliced_vbi_data` times the
	number of non-zero elements in the returned ``service_lines``
	array (that is the number of lines potentially carrying data).
    * - __u32
      - ``reserved``\ [2]
      - :cspan:`2` This array is reserved for future extensions.

	Applications and drivers must set it to zero.

.. raw:: latex

    \endgroup

.. _vbi-services2:

Sliced VBI services
-------------------

.. raw:: latex

    \footnotesize

.. tabularcolumns:: |p{4.2cm}|p{1.1cm}|p{2.1cm}|p{2.0cm}|p{6.5cm}|

.. flat-table::
    :header-rows:  1
    :stub-columns: 0
    :widths:       2 1 1 2 2

    * - Symbol
      - Value
      - Reference
      - Lines, usually
      - Payload
    * - ``V4L2_SLICED_TELETEXT_B`` (Teletext System B)
      - 0x0001
      - :ref:`ets300706`,

	:ref:`itu653`
      - PAL/SECAM line 7-22, 320-335 (second field 7-22)
      - Last 42 of the 45 byte Teletext packet, that is without clock
	run-in and framing code, lsb first transmitted.
    * - ``V4L2_SLICED_VPS``
      - 0x0400
      - :ref:`ets300231`
      - PAL line 16
      - Byte number 3 to 15 according to Figure 9 of ETS 300 231, lsb
	first transmitted.
    * - ``V4L2_SLICED_CAPTION_525``
      - 0x1000
      - :ref:`cea608`
      - NTSC line 21, 284 (second field 21)
      - Two bytes in transmission order, including parity bit, lsb first
	transmitted.
    * - ``V4L2_SLICED_WSS_625``
      - 0x4000
      - :ref:`itu1119`,

	:ref:`en300294`
      - PAL/SECAM line 23
      -  See :ref:`v4l2-sliced-wss-625-payload` below.
    * - ``V4L2_SLICED_VBI_525``
      - 0x1000
      - :cspan:`2` Set of services applicable to 525 line systems.
    * - ``V4L2_SLICED_VBI_625``
      - 0x4401
      - :cspan:`2` Set of services applicable to 625 line systems.

.. raw:: latex

    \normalsize

Drivers may return an ``EINVAL`` error code when applications attempt to
read or write data without prior format negotiation, after switching the
video standard (which may invalidate the negotiated VBI parameters) and
after switching the video input (which may change the video standard as
a side effect). The :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl may
return an ``EBUSY`` error code when applications attempt to change the
format while i/o is in progress (between a
:ref:`VIDIOC_STREAMON` and
:ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` call, and after the first
:c:func:`read()` or :c:func:`write()` call).

.. _v4l2-sliced-wss-625-payload:

V4L2_SLICED_WSS_625 payload
~~~~~~~~~~~~~~~~~~~~~~~~~~~

The payload for ``V4L2_SLICED_WSS_625`` is:

           +-----+------------------+-----------------------+
	   |Byte |        0         |           1           |
           +-----+--------+---------+-----------+-----------+
	   |     | msb    | lsb     | msb       | lsb       |
           |     +-+-+-+--+--+-+-+--+--+-+--+---+---+--+-+--+
	   | Bit |7|6|5|4 | 3|2|1|0 | x|x|13|12 | 11|10|9|8 |
           +-----+-+-+-+--+--+-+-+--+--+-+--+---+---+--+-+--+

Reading and writing sliced VBI data
===================================

A single :c:func:`read()` or :c:func:`write()`
call must pass all data belonging to one video frame. That is an array
of struct :c:type:`v4l2_sliced_vbi_data` structures with one or
more elements and a total size not exceeding ``io_size`` bytes. Likewise
in streaming I/O mode one buffer of ``io_size`` bytes must contain data
of one video frame. The ``id`` of unused
struct :c:type:`v4l2_sliced_vbi_data` elements must be zero.

.. c:type:: v4l2_sliced_vbi_data

struct v4l2_sliced_vbi_data
---------------------------

.. tabularcolumns:: |p{1.2cm}|p{2.2cm}|p{13.9cm}|

.. flat-table::
    :header-rows:  0
    :stub-columns: 0
    :widths:       3 1 4

    * - __u32
      - ``id``
      - A flag from :ref:`vbi-services` identifying the type of data in
	this packet. Only a single bit must be set. When the ``id`` of a
	captured packet is zero, the packet is empty and the contents of
	other fields are undefined. Applications shall ignore empty
	packets. When the ``id`` of a packet for output is zero the
	contents of the ``data`` field are undefined and the driver must
	no longer insert data on the requested ``field`` and ``line``.
    * - __u32
      - ``field``
      - The video field number this data has been captured from, or shall
	be inserted at. ``0`` for the first field, ``1`` for the second
	field.
    * - __u32
      - ``line``
      - The field (as opposed to frame) line number this data has been
	captured from, or shall be inserted at. See :ref:`vbi-525` and
	:ref:`vbi-625` for valid values. Sliced VBI capture devices can
	set the line number of all packets to ``0`` if the hardware cannot
	reliably identify scan lines. The field number must always be
	valid.
    * - __u32
      - ``reserved``
      - This field is reserved for future extensions. Applications and
	drivers must set it to zero.
    * - __u8
      - ``data``\ [48]
      - The packet payload. See :ref:`vbi-services` for the contents and
	number of bytes passed for each data type. The contents of padding
	bytes at the end of this array are undefined, drivers and
	applications shall ignore them.

Packets are always passed in ascending line number order, without
duplicate line numbers. The :c:func:`write()` function and
the :ref:`VIDIOC_QBUF` ioctl must return an ``EINVAL``
error code when applications violate this rule. They must also return an
EINVAL error code when applications pass an incorrect field or line
number, or a combination of ``field``, ``line`` and ``id`` which has not
been negotiated with the :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` or
:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. When the line numbers are
unknown the driver must pass the packets in transmitted order. The
driver can insert empty packets with ``id`` set to zero anywhere in the
packet array.

To assure synchronization and to distinguish from frame dropping, when a
captured frame does not carry any of the requested data services drivers
must pass one or more empty packets. When an application fails to pass
VBI data in time for output, the driver must output the last VPS and WSS
packet again, and disable the output of Closed Caption and Teletext
data, or output data which is ignored by Closed Caption and Teletext
decoders.

A sliced VBI device may support :ref:`read/write <rw>` and/or
streaming (:ref:`memory mapping <mmap>` and/or
:ref:`user pointer <userp>`) I/O. The latter bears the possibility of
synchronizing video and VBI data by using buffer timestamps.

Sliced VBI Data in MPEG Streams
===============================

If a device can produce an MPEG output stream, it may be capable of
providing
:ref:`negotiated sliced VBI services <sliced-vbi-format-negotitation>`
as data embedded in the MPEG stream. Users or applications control this
sliced VBI data insertion with the
:ref:`V4L2_CID_MPEG_STREAM_VBI_FMT <v4l2-mpeg-stream-vbi-fmt>`
control.

If the driver does not provide the
:ref:`V4L2_CID_MPEG_STREAM_VBI_FMT <v4l2-mpeg-stream-vbi-fmt>`
control, or only allows that control to be set to
:ref:`V4L2_MPEG_STREAM_VBI_FMT_NONE <v4l2-mpeg-stream-vbi-fmt>`,
then the device cannot embed sliced VBI data in the MPEG stream.

The
:ref:`V4L2_CID_MPEG_STREAM_VBI_FMT <v4l2-mpeg-stream-vbi-fmt>`
control does not implicitly set the device driver to capture nor cease
capturing sliced VBI data. The control only indicates to embed sliced
VBI data in the MPEG stream, if an application has negotiated sliced VBI
service be captured.

It may also be the case that a device can embed sliced VBI data in only
certain types of MPEG streams: for example in an MPEG-2 PS but not an
MPEG-2 TS. In this situation, if sliced VBI data insertion is requested,
the sliced VBI data will be embedded in MPEG stream types when
supported, and silently omitted from MPEG stream types where sliced VBI
data insertion is not supported by the device.

The following subsections specify the format of the embedded sliced VBI
data.

MPEG Stream Embedded, Sliced VBI Data Format: NONE
--------------------------------------------------

The
:ref:`V4L2_MPEG_STREAM_VBI_FMT_NONE <v4l2-mpeg-stream-vbi-fmt>`
embedded sliced VBI format shall be interpreted by drivers as a control
to cease embedding sliced VBI data in MPEG streams. Neither the device
nor driver shall insert "empty" embedded sliced VBI data packets in the
MPEG stream when this format is set. No MPEG stream data structures are
specified for this format.

MPEG Stream Embedded, Sliced VBI Data Format: IVTV
--------------------------------------------------

The
:ref:`V4L2_MPEG_STREAM_VBI_FMT_IVTV <v4l2-mpeg-stream-vbi-fmt>`
embedded sliced VBI format, when supported, indicates to the driver to
embed up to 36 lines of sliced VBI data per frame in an MPEG-2 *Private
Stream 1 PES* packet encapsulated in an MPEG-2 *Program Pack* in the
MPEG stream.

*Historical context*: This format specification originates from a
custom, embedded, sliced VBI data format used by the ``ivtv`` driver.
This format has already been informally specified in the kernel sources
in the file ``Documentation/userspace-api/media/drivers/cx2341x-uapi.rst`` . The
maximum size of the payload and other aspects of this format are driven
by the CX23415 MPEG decoder's capabilities and limitations with respect
to extracting, decoding, and displaying sliced VBI data embedded within
an MPEG stream.

This format's use is *not* exclusive to the ``ivtv`` driver *nor*
exclusive to CX2341x devices, as the sliced VBI data packet insertion
into the MPEG stream is implemented in driver software. At least the
``cx18`` driver provides sliced VBI data insertion into an MPEG-2 PS in
this format as well.

The following definitions specify the payload of the MPEG-2 *Private
Stream 1 PES* packets that contain sliced VBI data when
:ref:`V4L2_MPEG_STREAM_VBI_FMT_IVTV <v4l2-mpeg-stream-vbi-fmt>`
is set. (The MPEG-2 *Private Stream 1 PES* packet header and
encapsulating MPEG-2 *Program Pack* header are not detailed here. Please
refer to the MPEG-2 specifications for details on those packet headers.)

The payload of the MPEG-2 *Private Stream 1 PES* packets that contain
sliced VBI data is specified by struct
:c:type:`v4l2_mpeg_vbi_fmt_ivtv`. The
payload is variable length, depending on the actual number of lines of
sliced VBI data present in a video frame. The payload may be padded at
the end with unspecified fill bytes to align the end of the payload to a
4-byte boundary. The payload shall never exceed 1552 bytes (2 fields
with 18 lines/field with 43 bytes of data/line and a 4 byte magic
number).

.. c:type:: v4l2_mpeg_vbi_fmt_ivtv

struct v4l2_mpeg_vbi_fmt_ivtv
-----------------------------

.. tabularcolumns:: |p{4.2cm}|p{2.0cm}|p{11.1cm}|

.. flat-table::
    :header-rows:  0
    :stub-columns: 0
    :widths:       1 1 2

    * - __u8
      - ``magic``\ [4]
      - A "magic" constant from :ref:`v4l2-mpeg-vbi-fmt-ivtv-magic` that
	indicates this is a valid sliced VBI data payload and also
	indicates which member of the anonymous union, ``itv0`` or
	``ITV0``, to use for the payload data.
    * - union {
      - (anonymous)
    * - struct :c:type:`v4l2_mpeg_vbi_itv0`
      - ``itv0``
      - The primary form of the sliced VBI data payload that contains
	anywhere from 1 to 35 lines of sliced VBI data. Line masks are
	provided in this form of the payload indicating which VBI lines
	are provided.
    * - struct :ref:`v4l2_mpeg_vbi_ITV0 <v4l2-mpeg-vbi-itv0-1>`
      - ``ITV0``
      - An alternate form of the sliced VBI data payload used when 36
	lines of sliced VBI data are present. No line masks are provided
	in this form of the payload; all valid line mask bits are
	implcitly set.
    * - }
      -

.. _v4l2-mpeg-vbi-fmt-ivtv-magic:

Magic Constants for struct v4l2_mpeg_vbi_fmt_ivtv magic field
-------------------------------------------------------------

.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.5cm}|

.. flat-table::
    :header-rows:  1
    :stub-columns: 0
    :widths:       3 1 4

    * - Defined Symbol
      - Value
      - Description
    * - ``V4L2_MPEG_VBI_IVTV_MAGIC0``
      - "itv0"
      - Indicates the ``itv0`` member of the union in struct
	:c:type:`v4l2_mpeg_vbi_fmt_ivtv` is
	valid.
    * - ``V4L2_MPEG_VBI_IVTV_MAGIC1``
      - "ITV0"
      - Indicates the ``ITV0`` member of the union in struct
	:c:type:`v4l2_mpeg_vbi_fmt_ivtv` is
	valid and that 36 lines of sliced VBI data are present.

.. c:type:: v4l2_mpeg_vbi_itv0

.. c:type:: v4l2_mpeg_vbi_ITV0

structs v4l2_mpeg_vbi_itv0 and v4l2_mpeg_vbi_ITV0
-------------------------------------------------

.. raw:: latex

   \footnotesize

.. tabularcolumns:: |p{4.6cm}|p{2.0cm}|p{10.7cm}|

.. flat-table::
    :header-rows:  0
    :stub-columns: 0
    :widths:       1 1 2

    * - __le32
      - ``linemask``\ [2]
      - Bitmasks indicating the VBI service lines present. These
	``linemask`` values are stored in little endian byte order in the
	MPEG stream. Some reference ``linemask`` bit positions with their
	corresponding VBI line number and video field are given below.
	b\ :sub:`0` indicates the least significant bit of a ``linemask``
	value:

	::

	    linemask[0] b0:     line  6  first field
	    linemask[0] b17:    line 23  first field
	    linemask[0] b18:    line  6  second field
	    linemask[0] b31:    line 19  second field
	    linemask[1] b0:     line 20  second field
	    linemask[1] b3:     line 23  second field
	    linemask[1] b4-b31: unused and set to 0
    * - struct
	:c:type:`v4l2_mpeg_vbi_itv0_line`
      - ``line``\ [35]
      - This is a variable length array that holds from 1 to 35 lines of
	sliced VBI data. The sliced VBI data lines present correspond to
	the bits set in the ``linemask`` array, starting from b\ :sub:`0`
	of ``linemask``\ [0] up through b\ :sub:`31` of ``linemask``\ [0],
	and from b\ :sub:`0` of ``linemask``\ [1] up through b\ :sub:`3` of
	``linemask``\ [1]. ``line``\ [0] corresponds to the first bit
	found set in the ``linemask`` array, ``line``\ [1] corresponds to
	the second bit found set in the ``linemask`` array, etc. If no
	``linemask`` array bits are set, then ``line``\ [0] may contain
	one line of unspecified data that should be ignored by
	applications.

.. raw:: latex

   \normalsize

.. _v4l2-mpeg-vbi-itv0-1:

struct v4l2_mpeg_vbi_ITV0
-------------------------

.. tabularcolumns:: |p{5.2cm}|p{2.4cm}|p{9.7cm}|

.. flat-table::
    :header-rows:  0
    :stub-columns: 0
    :widths:       1 1 2

    * - struct
	:c:type:`v4l2_mpeg_vbi_itv0_line`
      - ``line``\ [36]
      - A fixed length array of 36 lines of sliced VBI data. ``line``\ [0]
	through ``line``\ [17] correspond to lines 6 through 23 of the
	first field. ``line``\ [18] through ``line``\ [35] corresponds to
	lines 6 through 23 of the second field.

.. c:type:: v4l2_mpeg_vbi_itv0_line

struct v4l2_mpeg_vbi_itv0_line
------------------------------

.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.5cm}|

.. flat-table::
    :header-rows:  0
    :stub-columns: 0
    :widths:       1 1 2

    * - __u8
      - ``id``
      - A line identifier value from
	:ref:`ITV0-Line-Identifier-Constants` that indicates the type of
	sliced VBI data stored on this line.
    * - __u8
      - ``data``\ [42]
      - The sliced VBI data for the line.

.. _ITV0-Line-Identifier-Constants:

Line Identifiers for struct v4l2_mpeg_vbi_itv0_line id field
------------------------------------------------------------

.. tabularcolumns:: |p{7.0cm}|p{1.8cm}|p{8.5cm}|

.. flat-table::
    :header-rows:  1
    :stub-columns: 0
    :widths:       3 1 4

    * - Defined Symbol
      - Value
      - Description
    * - ``V4L2_MPEG_VBI_IVTV_TELETEXT_B``
      - 1
      - Refer to :ref:`Sliced VBI services <vbi-services2>` for a
	description of the line payload.
    * - ``V4L2_MPEG_VBI_IVTV_CAPTION_525``
      - 4
      - Refer to :ref:`Sliced VBI services <vbi-services2>` for a
	description of the line payload.
    * - ``V4L2_MPEG_VBI_IVTV_WSS_625``
      - 5
      - Refer to :ref:`Sliced VBI services <vbi-services2>` for a
	description of the line payload.
    * - ``V4L2_MPEG_VBI_IVTV_VPS``
      - 7
      - Refer to :ref:`Sliced VBI services <vbi-services2>` for a
	description of the line payload.

.. [#f1]
   According to :ref:`ETS 300 706 <ets300706>` lines 6-22 of the first
   field and lines 5-22 of the second field may carry Teletext data.

.. [#f2]
   See also :ref:`vbi-525` and :ref:`vbi-625`.


code:
.. SPDX-License-Identifier: GPL-2.0

.. _stateless_decoder:

**************************************************
Memory-to-memory Stateless Video Decoder Interface
**************************************************

A stateless decoder is a decoder that works without retaining any kind of state
between processed frames. This means that each frame is decoded independently
of any previous and future frames, and that the client is responsible for
maintaining the decoding state and providing it to the decoder with each
decoding request. This is in contrast to the stateful video decoder interface,
where the hardware and driver maintain the decoding state and all the client
has to do is to provide the raw encoded stream and dequeue decoded frames in
display order.

This section describes how user-space ("the client") is expected to communicate
with stateless decoders in order to successfully decode an encoded stream.
Compared to stateful codecs, the decoder/client sequence is simpler, but the
cost of this simplicity is extra complexity in the client which is responsible
for maintaining a consistent decoding state.

Stateless decoders make use of the :ref:`media-request-api`. A stateless
decoder must expose the ``V4L2_BUF_CAP_SUPPORTS_REQUESTS`` capability on its
``OUTPUT`` queue when :c:func:`VIDIOC_REQBUFS` or :c:func:`VIDIOC_CREATE_BUFS`
are invoked.

Depending on the encoded formats supported by the decoder, a single decoded
frame may be the result of several decode requests (for instance, H.264 streams
with multiple slices per frame). Decoders that support such formats must also
expose the ``V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF`` capability on their
``OUTPUT`` queue.

Querying capabilities
=====================

1. To enumerate the set of coded formats supported by the decoder, the client
   calls :c:func:`VIDIOC_ENUM_FMT` on the ``OUTPUT`` queue.

   * The driver must always return the full set of supported ``OUTPUT`` formats,
     irrespective of the format currently set on the ``CAPTURE`` queue.

   * Simultaneously, the driver must restrain the set of values returned by
     codec-specific capability controls (such as H.264 profiles) to the set
     actually supported by the hardware.

2. To enumerate the set of supported raw formats, the client calls
   :c:func:`VIDIOC_ENUM_FMT` on the ``CAPTURE`` queue.

   * The driver must return only the formats supported for the format currently
     active on the ``OUTPUT`` queue.

   * Depending on the currently set ``OUTPUT`` format, the set of supported raw
     formats may depend on the value of some codec-dependent controls.
     The client is responsible for making sure that these controls are set
     before querying the ``CAPTURE`` queue. Failure to do so will result in the
     default values for these controls being used, and a returned set of formats
     that may not be usable for the media the client is trying to decode.

3. The client may use :c:func:`VIDIOC_ENUM_FRAMESIZES` to detect supported
   resolutions for a given format, passing desired pixel format in
   :c:type:`v4l2_frmsizeenum`'s ``pixel_format``.

4. Supported profiles and levels for the current ``OUTPUT`` format, if
   applicable, may be queried using their respective controls via
   :c:func:`VIDIOC_QUERYCTRL`.

Initialization
==============

1. Set the coded format on the ``OUTPUT`` queue via :c:func:`VIDIOC_S_FMT`.

   * **Required fields:**

     ``type``
         a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``.

     ``pixelformat``
         a coded pixel format.

     ``width``, ``height``
         coded width and height parsed from the stream.

     other fields
         follow standard semantics.

   .. note::

      Changing the ``OUTPUT`` format may change the currently set ``CAPTURE``
      format. The driver will derive a new ``CAPTURE`` format from the
      ``OUTPUT`` format being set, including resolution, colorimetry
      parameters, etc. If the client needs a specific ``CAPTURE`` format,
      it must adjust it afterwards.

2. Call :c:func:`VIDIOC_S_EXT_CTRLS` to set all the controls (parsed headers,
   etc.) required by the ``OUTPUT`` format to enumerate the ``CAPTURE`` formats.

3. Call :c:func:`VIDIOC_G_FMT` for ``CAPTURE`` queue to get the format for the
   destination buffers parsed/decoded from the bytestream.

   * **Required fields:**

     ``type``
         a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``.

   * **Returned fields:**

     ``width``, ``height``
         frame buffer resolution for the decoded frames.

     ``pixelformat``
         pixel format for decoded frames.

     ``num_planes`` (for _MPLANE ``type`` only)
         number of planes for pixelformat.

     ``sizeimage``, ``bytesperline``
         as per standard semantics; matching frame buffer format.

   .. note::

      The value of ``pixelformat`` may be any pixel format supported for the
      ``OUTPUT`` format, based on the hardware capabilities. It is suggested
      that the driver chooses the preferred/optimal format for the current
      configuration. For example, a YUV format may be preferred over an RGB
      format, if an additional conversion step would be required for RGB.

4. *[optional]* Enumerate ``CAPTURE`` formats via :c:func:`VIDIOC_ENUM_FMT` on
   the ``CAPTURE`` queue. The client may use this ioctl to discover which
   alternative raw formats are supported for the current ``OUTPUT`` format and
   select one of them via :c:func:`VIDIOC_S_FMT`.

   .. note::

      The driver will return only formats supported for the currently selected
      ``OUTPUT`` format and currently set controls, even if more formats may be
      supported by the decoder in general.

      For example, a decoder may support YUV and RGB formats for
      resolutions 1920x1088 and lower, but only YUV for higher resolutions (due
      to hardware limitations). After setting a resolution of 1920x1088 or lower
      as the ``OUTPUT`` format, :c:func:`VIDIOC_ENUM_FMT` may return a set of
      YUV and RGB pixel formats, but after setting a resolution higher than
      1920x1088, the driver will not return RGB pixel formats, since they are
      unsupported for this resolution.

5. *[optional]* Choose a different ``CAPTURE`` format than suggested via
   :c:func:`VIDIOC_S_FMT` on ``CAPTURE`` queue. It is possible for the client to
   choose a different format than selected/suggested by the driver in
   :c:func:`VIDIOC_G_FMT`.

    * **Required fields:**

      ``type``
          a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``.

      ``pixelformat``
          a raw pixel format.

      ``width``, ``height``
         frame buffer resolution of the decoded stream; typically unchanged from
         what was returned with :c:func:`VIDIOC_G_FMT`, but it may be different
         if the hardware supports composition and/or scaling.

   After performing this step, the client must perform step 3 again in order
   to obtain up-to-date information about the buffers size and layout.

6. Allocate source (bytestream) buffers via :c:func:`VIDIOC_REQBUFS` on
   ``OUTPUT`` queue.

    * **Required fields:**

      ``count``
          requested number of buffers to allocate; greater than zero.

      ``type``
          a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``.

      ``memory``
          follows standard semantics.

    * **Return fields:**

      ``count``
          actual number of buffers allocated.

    * If required, the driver will adjust ``count`` to be equal or bigger to the
      minimum of required number of ``OUTPUT`` buffers for the given format and
      requested count. The client must check this value after the ioctl returns
      to get the actual number of buffers allocated.

7. Allocate destination (raw format) buffers via :c:func:`VIDIOC_REQBUFS` on the
   ``CAPTURE`` queue.

    * **Required fields:**

      ``count``
          requested number of buffers to allocate; greater than zero. The client
          is responsible for deducing the minimum number of buffers required
          for the stream to be properly decoded (taking e.g. reference frames
          into account) and pass an equal or bigger number.

      ``type``
          a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``.

      ``memory``
          follows standard semantics. ``V4L2_MEMORY_USERPTR`` is not supported
          for ``CAPTURE`` buffers.

    * **Return fields:**

      ``count``
          adjusted to allocated number of buffers, in case the codec requires
          more buffers than requested.

    * The driver must adjust count to the minimum of required number of
      ``CAPTURE`` buffers for the current format, stream configuration and
      requested count. The client must check this value after the ioctl
      returns to get the number of buffers allocated.

8. Allocate requests (likely one per ``OUTPUT`` buffer) via
    :c:func:`MEDIA_IOC_REQUEST_ALLOC` on the media device.

9. Start streaming on both ``OUTPUT`` and ``CAPTURE`` queues via
    :c:func:`VIDIOC_STREAMON`.

Decoding
========

For each frame, the client is responsible for submitting at least one request to
which the following is attached:

* The amount of encoded data expected by the codec for its current
  configuration, as a buffer submitted to the ``OUTPUT`` queue. Typically, this
  corresponds to one frame worth of encoded data, but some formats may allow (or
  require) different amounts per unit.
* All the metadata needed to decode the submitted encoded data, in the form of
  controls relevant to the format being decoded.

The amount of data and contents of the source ``OUTPUT`` buffer, as well as the
controls that must be set on the request, depend on the active coded pixel
format and might be affected by codec-specific extended controls, as stated in
documentation of each format.

If there is a possibility that the decoded frame will require one or more
decode requests after the current one in order to be produced, then the client
must set the ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` flag on the ``OUTPUT``
buffer. This will result in the (potentially partially) decoded ``CAPTURE``
buffer not being made available for dequeueing, and reused for the next decode
request if the timestamp of the next ``OUTPUT`` buffer has not changed.

A typical frame would thus be decoded using the following sequence:

1. Queue an ``OUTPUT`` buffer containing one unit of encoded bytestream data for
   the decoding request, using :c:func:`VIDIOC_QBUF`.

    * **Required fields:**

      ``index``
          index of the buffer being queued.

      ``type``
          type of the buffer.

      ``bytesused``
          number of bytes taken by the encoded data frame in the buffer.

      ``flags``
          the ``V4L2_BUF_FLAG_REQUEST_FD`` flag must be set. Additionally, if
          we are not sure that the current decode request is the last one needed
          to produce a fully decoded frame, then
          ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` must also be set.

      ``request_fd``
          must be set to the file descriptor of the decoding request.

      ``timestamp``
          must be set to a unique value per frame. This value will be propagated
          into the decoded frame's buffer and can also be used to use this frame
          as the reference of another. If using multiple decode requests per
          frame, then the timestamps of all the ``OUTPUT`` buffers for a given
          frame must be identical. If the timestamp changes, then the currently
          held ``CAPTURE`` buffer will be made available for dequeuing and the
          current request will work on a new ``CAPTURE`` buffer.

2. Set the codec-specific controls for the decoding request, using
   :c:func:`VIDIOC_S_EXT_CTRLS`.

    * **Required fields:**

      ``which``
          must be ``V4L2_CTRL_WHICH_REQUEST_VAL``.

      ``request_fd``
          must be set to the file descriptor of the decoding request.

      other fields
          other fields are set as usual when setting controls. The ``controls``
          array must contain all the codec-specific controls required to decode
          a frame.

   .. note::

      It is possible to specify the controls in different invocations of
      :c:func:`VIDIOC_S_EXT_CTRLS`, or to overwrite a previously set control, as
      long as ``request_fd`` and ``which`` are properly set. The controls state
      at the moment of request submission is the one that will be considered.

   .. note::

      The order in which steps 1 and 2 take place is interchangeable.

3. Submit the request by invoking :c:func:`MEDIA_REQUEST_IOC_QUEUE` on the
   request FD.

    If the request is submitted without an ``OUTPUT`` buffer, or if some of the
    required controls are missing from the request, then
    :c:func:`MEDIA_REQUEST_IOC_QUEUE` will return ``-ENOENT``. If more than one
    ``OUTPUT`` buffer is queued, then it will return ``-EINVAL``.
    :c:func:`MEDIA_REQUEST_IOC_QUEUE` returning non-zero means that no
    ``CAPTURE`` buffer will be produced for this request.

``CAPTURE`` buffers must not be part of the request, and are queued
independently. They are returned in decode order (i.e. the same order as coded
frames were submitted to the ``OUTPUT`` queue).

Runtime decoding errors are signaled by the dequeued ``CAPTURE`` buffers
carrying the ``V4L2_BUF_FLAG_ERROR`` flag. If a decoded reference frame has an
error, then all following decoded frames that refer to it also have the
``V4L2_BUF_FLAG_ERROR`` flag set, although the decoder will still try to
produce (likely corrupted) frames.

Buffer management while decoding
================================
Contrary to stateful decoders, a stateless decoder does not perform any kind of
buffer management: it only guarantees that dequeued ``CAPTURE`` buffers can be
used by the client for as long as they are not queued again. "Used" here
encompasses using the buffer for compositing or display.

A dequeued capture buffer can also be used as the reference frame of another
buffer.

A frame is specified as reference by converting its timestamp into nanoseconds,
and storing it into the relevant member of a codec-dependent control structure.
The :c:func:`v4l2_timeval_to_ns` function must be used to perform that
conversion. The timestamp of a frame can be used to reference it as soon as all
its units of encoded data are successfully submitted to the ``OUTPUT`` queue.

A decoded buffer containing a reference frame must not be reused as a decoding
target until all the frames referencing it have been decoded. The safest way to
achieve this is to refrain from queueing a reference buffer until all the
decoded frames referencing it have been dequeued. However, if the driver can
guarantee that buffers queued to the ``CAPTURE`` queue are processed in queued
order, then user-space can take advantage of this guarantee and queue a
reference buffer when the following conditions are met:

1. All the requests for frames affected by the reference frame have been
   queued, and

2. A sufficient number of ``CAPTURE`` buffers to cover all the decoded
   referencing frames have been queued.

When queuing a decoding request, the driver will increase the reference count of
all the resources associated with reference frames. This means that the client
can e.g. close the DMABUF file descriptors of reference frame buffers if it
won't need them afterwards.

Seeking
=======
In order to seek, the client just needs to submit requests using input buffers
corresponding to the new stream position. It must however be aware that
resolution may have changed and follow the dynamic resolution change sequence in
that case. Also depending on the codec used, picture parameters (e.g. SPS/PPS
for H.264) may have changed and the client is responsible for making sure that a
valid state is sent to the decoder.

The client is then free to ignore any returned ``CAPTURE`` buffer that comes
from the pre-seek position.

Pausing
=======

In order to pause, the client can just cease queuing buffers onto the ``OUTPUT``
queue. Without source bytestream data, there is no data to process and the codec
will remain idle.

Dynamic resolution change
=========================

If the client detects a resolution change in the stream, it will need to perform
the initialization sequence again with the new resolution:

1. If the last submitted request resulted in a ``CAPTURE`` buffer being
   held by the use of the ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` flag, then the
   last frame is not available on the ``CAPTURE`` queue. In this case, a
   ``V4L2_DEC_CMD_FLUSH`` command shall be sent. This will make the driver
   dequeue the held ``CAPTURE`` buffer.

2. Wait until all submitted requests have completed and dequeue the
   corresponding output buffers.

3. Call :c:func:`VIDIOC_STREAMOFF` on both the ``OUTPUT`` and ``CAPTURE``
   queues.

4. Free all ``CAPTURE`` buffers by calling :c:func:`VIDIOC_REQBUFS` on the
   ``CAPTURE`` queue with a buffer count of zero.

5. Perform the initialization sequence again (minus the allocation of
   ``OUTPUT`` buffers), with the new resolution set on the ``OUTPUT`` queue.
   Note that due to resolution constraints, a different format may need to be
   picked on the ``CAPTURE`` queue.

Drain
=====

If the last submitted request resulted in a ``CAPTURE`` buffer being
held by the use of the ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` flag, then the
last frame is not available on the ``CAPTURE`` queue. In this case, a
``V4L2_DEC_CMD_FLUSH`` command shall be sent. This will make the driver
dequeue the held ``CAPTURE`` buffer.

After that, in order to drain the stream on a stateless decoder, the client
just needs to wait until all the submitted requests are completed.


code:
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later

.. _subdev:

********************
Sub-device Interface
********************

The complex nature of V4L2 devices, where hardware is often made of
several integrated circuits that need to interact with each other in a
controlled way, leads to complex V4L2 drivers. The drivers usually
reflect the hardware model in software, and model the different hardware
components as software blocks called sub-devices.

V4L2 sub-devices are usually kernel-only objects. If the V4L2 driver
implements the media device API, they will automatically inherit from
media entities. Applications will be able to enumerate the sub-devices
and discover the hardware topology using the media entities, pads and
links enumeration API.

In addition to make sub-devices discoverable, drivers can also choose to
make them directly configurable by applications. When both the
sub-device driver and the V4L2 device driver support this, sub-devices
will feature a character device node on which ioctls can be called to

-  query, read and write sub-devices controls

-  subscribe and unsubscribe to events and retrieve them

-  negotiate image formats on individual pads

Sub-device character device nodes, conventionally named
``/dev/v4l-subdev*``, use major number 81.

Drivers may opt to limit the sub-device character devices to only expose
operations that do not modify the device state. In such a case the sub-devices
are referred to as ``read-only`` in the rest of this documentation, and the
related restrictions are documented in individual ioctls.

Controls
========

Most V4L2 controls are implemented by sub-device hardware. Drivers
usually merge all controls and expose them through video device nodes.
Applications can control all sub-devices through a single interface.

Complex devices sometimes implement the same control in different pieces
of hardware. This situation is common in embedded platforms, where both
sensors and image processing hardware implement identical functions,
such as contrast adjustment, white balance or faulty pixels correction.
As the V4L2 controls API doesn't support several identical controls in a
single device, all but one of the identical controls are hidden.

Applications can access those hidden controls through the sub-device
node with the V4L2 control API described in :ref:`control`. The ioctls
behave identically as when issued on V4L2 device nodes, with the
exception that they deal only with controls implemented in the
sub-device.

Depending on the driver, those controls might also be exposed through
one (or several) V4L2 device nodes.

Events
======

V4L2 sub-devices can notify applications of events as described in
:ref:`event`. The API behaves identically as when used on V4L2 device
nodes, with the exception that it only deals with events generated by
the sub-device. Depending on the driver, those events might also be
reported on one (or several) V4L2 device nodes.

.. _pad-level-formats:

Pad-level Formats
=================

.. warning::

    Pad-level formats are only applicable to very complex devices that
    need to expose low-level format configuration to user space. Generic
    V4L2 applications do *not* need to use the API described in this
    section.

.. note::

    For the purpose of this section, the term *format* means the
    combination of media bus data format, frame width and frame height.

Image formats are typically negotiated on video capture and output
devices using the format and
:ref:`selection <VIDIOC_SUBDEV_G_SELECTION>` ioctls. The driver is
responsible for configuring every block in the video pipeline according
to the requested format at the pipeline input and/or output.

For complex devices, such as often found in embedded systems, identical
image sizes at the output of a pipeline can be achieved using different
hardware configurations. One such example is shown on
:ref:`pipeline-scaling`, where image scaling can be performed on both
the video sensor and the host image processing hardware.

.. _pipeline-scaling:

.. kernel-figure:: pipeline.dot
    :alt:   pipeline.dot
    :align: center

    Image Format Negotiation on Pipelines

    High quality and high speed pipeline configuration

The sensor scaler is usually of less quality than the host scaler, but
scaling on the sensor is required to achieve higher frame rates.
Depending on the use case (quality vs. speed), the pipeline must be
configured differently. Applications need to configure the formats at
every point in the pipeline explicitly.

Drivers that implement the :ref:`media API <media-controller-intro>`
can expose pad-level image format configuration to applications. When
they do, applications can use the
:ref:`VIDIOC_SUBDEV_G_FMT <VIDIOC_SUBDEV_G_FMT>` and
:ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls. to
negotiate formats on a per-pad basis.

Applications are responsible for configuring coherent parameters on the
whole pipeline and making sure that connected pads have compatible
formats. The pipeline is checked for formats mismatch at
:ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>` time, and an ``EPIPE`` error
code is then returned if the configuration is invalid.

Pad-level image format configuration support can be tested by calling
the :ref:`VIDIOC_SUBDEV_G_FMT` ioctl on pad
0. If the driver returns an ``EINVAL`` error code pad-level format
configuration is not supported by the sub-device.

Format Negotiation
------------------

Acceptable formats on pads can (and usually do) depend on a number of
external parameters, such as formats on other pads, active links, or
even controls. Finding a combination of formats on all pads in a video
pipeline, acceptable to both application and driver, can't rely on
formats enumeration only. A format negotiation mechanism is required.

Central to the format negotiation mechanism are the get/set format
operations. When called with the ``which`` argument set to
:ref:`V4L2_SUBDEV_FORMAT_TRY <VIDIOC_SUBDEV_G_FMT>`, the
:ref:`VIDIOC_SUBDEV_G_FMT <VIDIOC_SUBDEV_G_FMT>` and
:ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls operate on
a set of formats parameters that are not connected to the hardware
configuration. Modifying those 'try' formats leaves the device state
untouched (this applies to both the software state stored in the driver
and the hardware state stored in the device itself).

While not kept as part of the device state, try formats are stored in
the sub-device file handles. A
:ref:`VIDIOC_SUBDEV_G_FMT <VIDIOC_SUBDEV_G_FMT>` call will return
the last try format set *on the same sub-device file handle*. Several
applications querying the same sub-device at the same time will thus not
interact with each other.

To find out whether a particular format is supported by the device,
applications use the
:ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctl. Drivers
verify and, if needed, change the requested ``format`` based on device
requirements and return the possibly modified value. Applications can
then choose to try a different format or accept the returned value and
continue.

Formats returned by the driver during a negotiation iteration are
guaranteed to be supported by the device. In particular, drivers
guarantee that a returned format will not be further changed if passed
to an :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` call as-is
(as long as external parameters, such as formats on other pads or links'
configuration are not changed).

Drivers automatically propagate formats inside sub-devices. When a try
or active format is set on a pad, corresponding formats on other pads of
the same sub-device can be modified by the driver. Drivers are free to
modify formats as required by the device. However, they should comply
with the following rules when possible:

-  Formats should be propagated from sink pads to source pads. Modifying
   a format on a source pad should not modify the format on any sink
   pad.

-  Sub-devices that scale frames using variable scaling factors should
   reset the scale factors to default values when sink pads formats are
   modified. If the 1:1 scaling ratio is supported, this means that
   source pads formats should be reset to the sink pads formats.

Formats are not propagated across links, as that would involve
propagating them from one sub-device file handle to another.
Applications must then take care to configure both ends of every link
explicitly with compatible formats. Identical formats on the two ends of
a link are guaranteed to be compatible. Drivers are free to accept
different formats matching device requirements as being compatible.

:ref:`sample-pipeline-config` shows a sample configuration sequence
for the pipeline described in :ref:`pipeline-scaling` (table columns
list entity names and pad numbers).

.. raw:: latex

    \begingroup
    \scriptsize
    \setlength{\tabcolsep}{2pt}

.. tabularcolumns:: |p{2.0cm}|p{2.1cm}|p{2.1cm}|p{2.1cm}|p{2.1cm}|p{2.1cm}|p{2.1cm}|

.. _sample-pipeline-config:

.. flat-table:: Sample Pipeline Configuration
    :header-rows:  1
    :stub-columns: 0
    :widths: 5 5 5 5 5 5 5

    * -
      - Sensor/0

        format
      - Frontend/0

        format
      - Frontend/1

        format
      - Scaler/0

        format
      - Scaler/0

        compose selection rectangle
      - Scaler/1

        format
    * - Initial state
      - 2048x1536

        SGRBG8_1X8
      - (default)
      - (default)
      - (default)
      - (default)
      - (default)
    * - Configure frontend sink format
      - 2048x1536

        SGRBG8_1X8
      - *2048x1536*

        *SGRBG8_1X8*
      - *2046x1534*

        *SGRBG8_1X8*
      - (default)
      - (default)
      - (default)
    * - Configure scaler sink format
      - 2048x1536

        SGRBG8_1X8
      - 2048x1536

        SGRBG8_1X8
      - 2046x1534

        SGRBG8_1X8
      - *2046x1534*

        *SGRBG8_1X8*
      - *0,0/2046x1534*
      - *2046x1534*

        *SGRBG8_1X8*
    * - Configure scaler sink compose selection
      - 2048x1536

        SGRBG8_1X8
      - 2048x1536

        SGRBG8_1X8
      - 2046x1534

        SGRBG8_1X8
      - 2046x1534

        SGRBG8_1X8
      - *0,0/1280x960*
      - *1280x960*

        *SGRBG8_1X8*

.. raw:: latex

    \endgroup

1. Initial state. The sensor source pad format is set to its native 3MP
   size and V4L2_MBUS_FMT_SGRBG8_1X8 media bus code. Formats on the
   host frontend and scaler sink and source pads have the default
   values, as well as the compose rectangle on the scaler's sink pad.

2. The application configures the frontend sink pad format's size to
   2048x1536 and its media bus code to V4L2_MBUS_FMT_SGRBG_1X8. The
   driver propagates the format to the frontend source pad.

3. The application configures the scaler sink pad format's size to
   2046x1534 and the media bus code to V4L2_MBUS_FMT_SGRBG_1X8 to
   match the frontend source size and media bus code. The media bus code
   on the sink pad is set to V4L2_MBUS_FMT_SGRBG_1X8. The driver
   propagates the size to the compose selection rectangle on the
   scaler's sink pad, and the format to the scaler source pad.

4. The application configures the size of the compose selection
   rectangle of the scaler's sink pad 1280x960. The driver propagates
   the size to the scaler's source pad format.

When satisfied with the try results, applications can set the active
formats by setting the ``which`` argument to
``V4L2_SUBDEV_FORMAT_ACTIVE``. Active formats are changed exactly as try
formats by drivers. To avoid modifying the hardware state during format
negotiation, applications should negotiate try formats first and then
modify the active settings using the try formats returned during the
last negotiation iteration. This guarantees that the active format will
be applied as-is by the driver without being modified.

.. _v4l2-subdev-selections:

Selections: cropping, scaling and composition
---------------------------------------------

Many sub-devices support cropping frames on their input or output pads
(or possible even on both). Cropping is used to select the area of
interest in an image, typically on an image sensor or a video decoder.
It can also be used as part of digital zoom implementations to select
the area of the image that will be scaled up.

Crop settings are defined by a crop rectangle and represented in a
struct :c:type:`v4l2_rect` by the coordinates of the top
left corner and the rectangle size. Both the coordinates and sizes are
expressed in pixels.

As for pad formats, drivers store try and active rectangles for the
selection targets :ref:`v4l2-selections-common`.

On sink pads, cropping is applied relative to the current pad format.
The pad format represents the image size as received by the sub-device
from the previous block in the pipeline, and the crop rectangle
represents the sub-image that will be transmitted further inside the
sub-device for processing.

The scaling operation changes the size of the image by scaling it to new
dimensions. The scaling ratio isn't specified explicitly, but is implied
from the original and scaled image sizes. Both sizes are represented by
struct :c:type:`v4l2_rect`.

Scaling support is optional. When supported by a subdev, the crop
rectangle on the subdev's sink pad is scaled to the size configured
using the
:ref:`VIDIOC_SUBDEV_S_SELECTION <VIDIOC_SUBDEV_G_SELECTION>` IOCTL
using ``V4L2_SEL_TGT_COMPOSE`` selection target on the same pad. If the
subdev supports scaling but not composing, the top and left values are
not used and must always be set to zero.

On source pads, cropping is similar to sink pads, with the exception
that the source size from which the cropping is performed, is the
COMPOSE rectangle on the sink pad. In both sink and source pads, the
crop rectangle must be entirely contained inside the source image size
for the crop operation.

The drivers should always use the closest possible rectangle the user
requests on all selection targets, unless specifically told otherwise.
``V4L2_SEL_FLAG_GE`` and ``V4L2_SEL_FLAG_LE`` flags may be used to round
the image size either up or down. :ref:`v4l2-selection-flags`

Types of selection targets
--------------------------

Actual targets
^^^^^^^^^^^^^^

Actual targets (without a postfix) reflect the actual hardware
configuration at any point of time. There is a BOUNDS target
corresponding to every actual target.

BOUNDS targets
^^^^^^^^^^^^^^

BOUNDS targets is the smallest rectangle that contains all valid actual
rectangles. It may not be possible to set the actual rectangle as large
as the BOUNDS rectangle, however. This may be because e.g. a sensor's
pixel array is not rectangular but cross-shaped or round. The maximum
size may also be smaller than the BOUNDS rectangle.

Order of configuration and format propagation
---------------------------------------------

Inside subdevs, the order of image processing steps will always be from
the sink pad towards the source pad. This is also reflected in the order
in which the configuration must be performed by the user: the changes
made will be propagated to any subsequent stages. If this behaviour is
not desired, the user must set ``V4L2_SEL_FLAG_KEEP_CONFIG`` flag. This
flag causes no propagation of the changes are allowed in any
circumstances. This may also cause the accessed rectangle to be adjusted
by the driver, depending on the properties of the underlying hardware.

The coordinates to a step always refer to the actual size of the
previous step. The exception to this rule is the sink compose
rectangle, which refers to the sink compose bounds rectangle --- if it
is supported by the hardware.

1. Sink pad format. The user configures the sink pad format. This format
   defines the parameters of the image the entity receives through the
   pad for further processing.

2. Sink pad actual crop selection. The sink pad crop defines the crop
   performed to the sink pad format.

3. Sink pad actual compose selection. The size of the sink pad compose
   rectangle defines the scaling ratio compared to the size of the sink
   pad crop rectangle. The location of the compose rectangle specifies
   the location of the actual sink compose rectangle in the sink compose
   bounds rectangle.

4. Source pad actual crop selection. Crop on the source pad defines crop
   performed to the image in the sink compose bounds rectangle.

5. Source pad format. The source pad format defines the output pixel
   format of the subdev, as well as the other parameters with the
   exception of the image width and height. Width and height are defined
   by the size of the source pad actual crop selection.

Accessing any of the above rectangles not supported by the subdev will
return ``EINVAL``. Any rectangle referring to a previous unsupported
rectangle coordinates will instead refer to the previous supported
rectangle. For example, if sink crop is not supported, the compose
selection will refer to the sink pad format dimensions instead.

.. _subdev-image-processing-crop:

.. kernel-figure:: subdev-image-processing-crop.svg
    :alt:   subdev-image-processing-crop.svg
    :align: center

    **Figure 4.5. Image processing in subdevs: simple crop example**

In the above example, the subdev supports cropping on its sink pad. To
configure it, the user sets the media bus format on the subdev's sink
pad. Now the actual crop rectangle can be set on the sink pad --- the
location and size of this rectangle reflect the location and size of a
rectangle to be cropped from the sink format. The size of the sink crop
rectangle will also be the size of the format of the subdev's source
pad.

.. _subdev-image-processing-scaling-multi-source:

.. kernel-figure:: subdev-image-processing-scaling-multi-source.svg
    :alt:   subdev-image-processing-scaling-multi-source.svg
    :align: center

    **Figure 4.6. Image processing in subdevs: scaling with multiple sources**

In this example, the subdev is capable of first cropping, then scaling
and finally cropping for two source pads individually from the resulting
scaled image. The location of the scaled image in the cropped image is
ignored in sink compose target. Both of the locations of the source crop
rectangles refer to the sink scaling rectangle, independently cropping
an area at location specified by the source crop rectangle from it.

.. _subdev-image-processing-full:

.. kernel-figure:: subdev-image-processing-full.svg
    :alt:    subdev-image-processing-full.svg
    :align:  center

    **Figure 4.7. Image processing in subdevs: scaling and composition with multiple sinks and sources**

The subdev driver supports two sink pads and two source pads. The images
from both of the sink pads are individually cropped, then scaled and
further composed on the composition bounds rectangle. From that, two
independent streams are cropped and sent out of the subdev from the
source pads.

.. toctree::
    :maxdepth: 1

    subdev-formats


code:
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later

.. _touch:

*************
Touch Devices
*************

Touch devices are accessed through character device special files named
``/dev/v4l-touch0`` to ``/dev/v4l-touch255`` with major number 81 and
dynamically allocated minor numbers 0 to 255.

Overview
========

Sensors may be Optical, or Projected Capacitive touch (PCT).

Processing is required to analyse the raw data and produce input events. In
some systems, this may be performed on the ASIC and the raw data is purely a
side-channel for diagnostics or tuning. In other systems, the ASIC is a simple
analogue front end device which delivers touch data at high rate, and any touch
processing must be done on the host.

For capacitive touch sensing, the touchscreen is composed of an array of
horizontal and vertical conductors (alternatively called rows/columns, X/Y
lines, or tx/rx). Mutual Capacitance measured is at the nodes where the
conductors cross. Alternatively, Self Capacitance measures the signal from each
column and row independently.

A touch input may be determined by comparing the raw capacitance measurement to
a no-touch reference (or "baseline") measurement:

Delta = Raw - Reference

The reference measurement takes account of variations in the capacitance across
the touch sensor matrix, for example manufacturing irregularities,
environmental or edge effects.

Querying Capabilities
=====================

Devices supporting the touch interface set the ``V4L2_CAP_VIDEO_CAPTURE`` flag
and the ``V4L2_CAP_TOUCH`` flag in the ``capabilities`` field of
:c:type:`v4l2_capability` returned by the
:ref:`VIDIOC_QUERYCAP` ioctl.

At least one of the read/write or streaming I/O methods must be
supported.

The formats supported by touch devices are documented in
:ref:`Touch Formats <tch-formats>`.

Data Format Negotiation
=======================

A touch device may support any I/O method.


code:
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later

.. _devices:

**********
Interfaces
**********

.. toctree::
    :maxdepth: 1

    dev-capture
    dev-overlay
    dev-output
    dev-osd
    dev-mem2mem
    dev-raw-vbi
    dev-sliced-vbi
    dev-radio
    dev-rds
    dev-sdr
    dev-touch
    dev-event
    dev-subdev
    dev-meta