code:
.. SPDX-License-Identifier: GPL-2.0

Media Controller devices
------------------------

Media Controller
~~~~~~~~~~~~~~~~

The media controller userspace API is documented in
:ref:`the Media Controller uAPI book <media_controller>`. This document focus
on the kernel-side implementation of the media framework.

Abstract media device model
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Discovering a device internal topology, and configuring it at runtime, is one
of the goals of the media framework. To achieve this, hardware devices are
modelled as an oriented graph of building blocks called entities connected
through pads.

An entity is a basic media hardware building block. It can correspond to
a large variety of logical blocks such as physical hardware devices
(CMOS sensor for instance), logical hardware devices (a building block
in a System-on-Chip image processing pipeline), DMA channels or physical
connectors.

A pad is a connection endpoint through which an entity can interact with
other entities. Data (not restricted to video) produced by an entity
flows from the entity's output to one or more entity inputs. Pads should
not be confused with physical pins at chip boundaries.

A link is a point-to-point oriented connection between two pads, either
on the same entity or on different entities. Data flows from a source
pad to a sink pad.

Media device
^^^^^^^^^^^^

A media device is represented by a struct media_device
instance, defined in ``include/media/media-device.h``.
Allocation of the structure is handled by the media device driver, usually by
embedding the :c:type:`media_device` instance in a larger driver-specific
structure.

Drivers initialise media device instances by calling
:c:func:`media_device_init()`. After initialising a media device instance, it is
registered by calling :c:func:`__media_device_register()` via the macro
``media_device_register()`` and unregistered by calling
:c:func:`media_device_unregister()`. An initialised media device must be
eventually cleaned up by calling :c:func:`media_device_cleanup()`.

Note that it is not allowed to unregister a media device instance that was not
previously registered, or clean up a media device instance that was not
previously initialised.

Entities
^^^^^^^^

Entities are represented by a struct media_entity
instance, defined in ``include/media/media-entity.h``. The structure is usually
embedded into a higher-level structure, such as
:c:type:`v4l2_subdev` or :c:type:`video_device`
instances, although drivers can allocate entities directly.

Drivers initialize entity pads by calling
:c:func:`media_entity_pads_init()`.

Drivers register entities with a media device by calling
:c:func:`media_device_register_entity()`
and unregistered by calling
:c:func:`media_device_unregister_entity()`.

Interfaces
^^^^^^^^^^

Interfaces are represented by a
struct media_interface instance, defined in
``include/media/media-entity.h``. Currently, only one type of interface is
defined: a device node. Such interfaces are represented by a
struct media_intf_devnode.

Drivers initialize and create device node interfaces by calling
:c:func:`media_devnode_create()`
and remove them by calling:
:c:func:`media_devnode_remove()`.

Pads
^^^^
Pads are represented by a struct media_pad instance,
defined in ``include/media/media-entity.h``. Each entity stores its pads in
a pads array managed by the entity driver. Drivers usually embed the array in
a driver-specific structure.

Pads are identified by their entity and their 0-based index in the pads
array.

Both information are stored in the struct media_pad,
making the struct media_pad pointer the canonical way
to store and pass link references.

Pads have flags that describe the pad capabilities and state.

``MEDIA_PAD_FL_SINK`` indicates that the pad supports sinking data.
``MEDIA_PAD_FL_SOURCE`` indicates that the pad supports sourcing data.

.. note::

  One and only one of ``MEDIA_PAD_FL_SINK`` or ``MEDIA_PAD_FL_SOURCE`` must
  be set for each pad.

Links
^^^^^

Links are represented by a struct media_link instance,
defined in ``include/media/media-entity.h``. There are two types of links:

**1. pad to pad links**:

Associate two entities via their PADs. Each entity has a list that points
to all links originating at or targeting any of its pads.
A given link is thus stored twice, once in the source entity and once in
the target entity.

Drivers create pad to pad links by calling:
:c:func:`media_create_pad_link()` and remove with
:c:func:`media_entity_remove_links()`.

**2. interface to entity links**:

Associate one interface to a Link.

Drivers create interface to entity links by calling:
:c:func:`media_create_intf_link()` and remove with
:c:func:`media_remove_intf_links()`.

.. note::

   Links can only be created after having both ends already created.

Links have flags that describe the link capabilities and state. The
valid values are described at :c:func:`media_create_pad_link()` and
:c:func:`media_create_intf_link()`.

Graph traversal
^^^^^^^^^^^^^^^

The media framework provides APIs to iterate over entities in a graph.

To iterate over all entities belonging to a media device, drivers can use
the media_device_for_each_entity macro, defined in
``include/media/media-device.h``.

..  code-block:: c

    struct media_entity *entity;

    media_device_for_each_entity(entity, mdev) {
    // entity will point to each entity in turn
    ...
    }

Drivers might also need to iterate over all entities in a graph that can be
reached only through enabled links starting at a given entity. The media
framework provides a depth-first graph traversal API for that purpose.

.. note::

   Graphs with cycles (whether directed or undirected) are **NOT**
   supported by the graph traversal API. To prevent infinite loops, the graph
   traversal code limits the maximum depth to ``MEDIA_ENTITY_ENUM_MAX_DEPTH``,
   currently defined as 16.

Drivers initiate a graph traversal by calling
:c:func:`media_graph_walk_start()`

The graph structure, provided by the caller, is initialized to start graph
traversal at the given entity.

Drivers can then retrieve the next entity by calling
:c:func:`media_graph_walk_next()`

When the graph traversal is complete the function will return ``NULL``.

Graph traversal can be interrupted at any moment. No cleanup function call
is required and the graph structure can be freed normally.

Helper functions can be used to find a link between two given pads, or a pad
connected to another pad through an enabled link
(:c:func:`media_entity_find_link()`, :c:func:`media_pad_remote_pad_first()`,
:c:func:`media_entity_remote_source_pad_unique()` and
:c:func:`media_pad_remote_pad_unique()`).

Use count and power handling
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Due to the wide differences between drivers regarding power management
needs, the media controller does not implement power management. However,
the struct media_entity includes a ``use_count``
field that media drivers
can use to track the number of users of every entity for power management
needs.

The :c:type:`media_entity<media_entity>`.\ ``use_count`` field is owned by
media drivers and must not be
touched by entity drivers. Access to the field must be protected by the
:c:type:`media_device`.\ ``graph_mutex`` lock.

Links setup
^^^^^^^^^^^

Link properties can be modified at runtime by calling
:c:func:`media_entity_setup_link()`.

Pipelines and media streams
^^^^^^^^^^^^^^^^^^^^^^^^^^^

A media stream is a stream of pixels or metadata originating from one or more
source devices (such as a sensors) and flowing through media entity pads
towards the final sinks. The stream can be modified on the route by the
devices (e.g. scaling or pixel format conversions), or it can be split into
multiple branches, or multiple branches can be merged.

A media pipeline is a set of media streams which are interdependent. This
interdependency can be caused by the hardware (e.g. configuration of a second
stream cannot be changed if the first stream has been enabled) or by the driver
due to the software design. Most commonly a media pipeline consists of a single
stream which does not branch.

When starting streaming, drivers must notify all entities in the pipeline to
prevent link states from being modified during streaming by calling
:c:func:`media_pipeline_start()`.

The function will mark all the pads which are part of the pipeline as streaming.

The struct media_pipeline instance pointed to by
the pipe argument will be stored in every pad in the pipeline.
Drivers should embed the struct media_pipeline
in higher-level pipeline structures and can then access the
pipeline through the struct media_pad
pipe field.

Calls to :c:func:`media_pipeline_start()` can be nested.
The pipeline pointer must be identical for all nested calls to the function.

:c:func:`media_pipeline_start()` may return an error. In that case,
it will clean up any of the changes it did by itself.

When stopping the stream, drivers must notify the entities with
:c:func:`media_pipeline_stop()`.

If multiple calls to :c:func:`media_pipeline_start()` have been
made the same number of :c:func:`media_pipeline_stop()` calls
are required to stop streaming.
The :c:type:`media_entity`.\ ``pipe`` field is reset to ``NULL`` on the last
nested stop call.

Link configuration will fail with ``-EBUSY`` by default if either end of the
link is a streaming entity. Links that can be modified while streaming must
be marked with the ``MEDIA_LNK_FL_DYNAMIC`` flag.

If other operations need to be disallowed on streaming entities (such as
changing entities configuration parameters) drivers can explicitly check the
media_entity stream_count field to find out if an entity is streaming. This
operation must be done with the media_device graph_mutex held.

Link validation
^^^^^^^^^^^^^^^

Link validation is performed by :c:func:`media_pipeline_start()`
for any entity which has sink pads in the pipeline. The
:c:type:`media_entity`.\ ``link_validate()`` callback is used for that
purpose. In ``link_validate()`` callback, entity driver should check
that the properties of the source pad of the connected entity and its own
sink pad match. It is up to the type of the entity (and in the end, the
properties of the hardware) what matching actually means.

Subsystems should facilitate link validation by providing subsystem specific
helper functions to provide easy access for commonly needed information, and
in the end provide a way to use driver-specific callbacks.

Media Controller Device Allocator API
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When the media device belongs to more than one driver, the shared media
device is allocated with the shared struct device as the key for look ups.

The shared media device should stay in registered state until the last
driver unregisters it. In addition, the media device should be released when
all the references are released. Each driver gets a reference to the media
device during probe, when it allocates the media device. If media device is
already allocated, the allocate API bumps up the refcount and returns the
existing media device. The driver puts the reference back in its disconnect
routine when it calls :c:func:`media_device_delete()`.

The media device is unregistered and cleaned up from the kref put handler to
ensure that the media device stays in registered state until the last driver
unregisters the media device.

**Driver Usage**

Drivers should use the appropriate media-core routines to manage the shared
media device life-time handling the two states:
1. allocate -> register -> delete
2. get reference to already registered device -> delete

call :c:func:`media_device_delete()` routine to make sure the shared media
device delete is handled correctly.

**driver probe:**
Call :c:func:`media_device_usb_allocate()` to allocate or get a reference
Call :c:func:`media_device_register()`, if media devnode isn't registered

**driver disconnect:**
Call :c:func:`media_device_delete()` to free the media_device. Freeing is
handled by the kref put handler.

API Definitions
^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/media-device.h

.. kernel-doc:: include/media/media-devnode.h

.. kernel-doc:: include/media/media-entity.h

.. kernel-doc:: include/media/media-request.h

.. kernel-doc:: include/media/media-dev-allocator.h


code:
.. SPDX-License-Identifier: GPL-2.0

Remote Controller devices
-------------------------

Remote Controller core
~~~~~~~~~~~~~~~~~~~~~~

The remote controller core implements infrastructure to receive and send
remote controller keyboard keystrokes and mouse events.

Every time a key is pressed on a remote controller, a scan code is produced.
Also, on most hardware, keeping a key pressed for more than a few dozens of
milliseconds produce a repeat key event. That's somewhat similar to what
a normal keyboard or mouse is handled internally on Linux\ [#f1]_. So, the
remote controller core is implemented on the top of the linux input/evdev
interface.

.. [#f1]

   The main difference is that, on keyboard events, the keyboard controller
   produces one event for a key press and another one for key release. On
   infrared-based remote controllers, there's no key release event. Instead,
   an extra code is produced to indicate key repeats.

However, most of the remote controllers use infrared (IR) to transmit signals.
As there are several protocols used to modulate infrared signals, one
important part of the core is dedicated to adjust the driver and the core
system to support the infrared protocol used by the emitter.

The infrared transmission is done by blinking a infrared emitter using a
carrier. The carrier can be switched on or off by the IR transmitter
hardware. When the carrier is switched on, it is called *PULSE*.
When the carrier is switched off, it is called *SPACE*.

In other words, a typical IR transmission can be viewed as a sequence of
*PULSE* and *SPACE* events, each with a given duration.

The carrier parameters (frequency, duty cycle) and the intervals for
*PULSE* and *SPACE* events depend on the protocol.
For example, the NEC protocol uses a carrier of 38kHz, and transmissions
start with a 9ms *PULSE* and a 4.5ms SPACE. It then transmits 16 bits of
scan code, being 8 bits for address (usually it is a fixed number for a
given remote controller), followed by 8 bits of code. A bit "1" is modulated
with 560µs *PULSE* followed by 1690µs *SPACE* and a bit "0"  is modulated
with 560µs *PULSE* followed by 560µs *SPACE*.

At receiver, a simple low-pass filter can be used to convert the received
signal in a sequence of *PULSE/SPACE* events, filtering out the carrier
frequency. Due to that, the receiver doesn't care about the carrier's
actual frequency parameters: all it has to do is to measure the amount
of time it receives *PULSE/SPACE* events.
So, a simple IR receiver hardware will just provide a sequence of timings
for those events to the Kernel. The drivers for hardware with such kind of
receivers are identified by  ``RC_DRIVER_IR_RAW``, as defined by
:c:type:`rc_driver_type`\ [#f2]_. Other hardware come with a
microcontroller that decode the *PULSE/SPACE* sequence and return scan
codes to the Kernel. Such kind of receivers are identified
by ``RC_DRIVER_SCANCODE``.

.. [#f2]

   The RC core also supports devices that have just IR emitters,
   without any receivers. Right now, all such devices work only in
   raw TX mode. Such kind of hardware is identified as
   ``RC_DRIVER_IR_RAW_TX``.

When the RC core receives events produced by ``RC_DRIVER_IR_RAW`` IR
receivers, it needs to decode the IR protocol, in order to obtain the
corresponding scan code. The protocols supported by the RC core are
defined at enum :c:type:`rc_proto`.

When the RC code receives a scan code (either directly, by a driver
of the type ``RC_DRIVER_SCANCODE``, or via its IR decoders), it needs
to convert into a Linux input event code. This is done via a mapping
table.

The Kernel has support for mapping tables available on most media
devices. It also supports loading a table in runtime, via some
sysfs nodes. See the :ref:`RC userspace API <Remote_controllers_Intro>`
for more details.

Remote controller data structures and functions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/rc-core.h

.. kernel-doc:: include/media/rc-map.h

repo_name: angery/linux


code:
.. SPDX-License-Identifier: GPL-2.0

.. _transmitter-receiver:

Pixel data transmitter and receiver drivers
===========================================

V4L2 supports various devices that transmit and receive pixel data. Examples of
these devices include a camera sensor, a TV tuner and a parallel or a CSI-2
receiver in an SoC.

Bus types
---------

The following busses are the most common. This section discusses these two only.

MIPI CSI-2
^^^^^^^^^^

CSI-2 is a data bus intended for transferring images from cameras to
the host SoC. It is defined by the `MIPI alliance`_.

.. _`MIPI alliance`: https://www.mipi.org/

Parallel
^^^^^^^^

`BT.601`_ and `BT.656`_ are the most common parallel busses.

.. _`BT.601`: https://en.wikipedia.org/wiki/Rec._601
.. _`BT.656`: https://en.wikipedia.org/wiki/ITU-R_BT.656

Transmitter drivers
-------------------

Transmitter drivers generally need to provide the receiver drivers with the
configuration of the transmitter. What is required depends on the type of the
bus. These are common for both busses.

Media bus pixel code
^^^^^^^^^^^^^^^^^^^^

See :ref:`v4l2-mbus-pixelcode`.

Link frequency
^^^^^^^^^^^^^^

The :ref:`V4L2_CID_LINK_FREQ <v4l2-cid-link-freq>` control is used to tell the
receiver the frequency of the bus (i.e. it is not the same as the symbol rate).

``.s_stream()`` callback
^^^^^^^^^^^^^^^^^^^^^^^^

The struct struct v4l2_subdev_video_ops->s_stream() callback is used by the
receiver driver to control the transmitter driver's streaming state.

CSI-2 transmitter drivers
-------------------------

Pixel rate
^^^^^^^^^^

The pixel rate on the bus is calculated as follows::

	pixel_rate = link_freq * 2 * nr_of_lanes * 16 / k / bits_per_sample

where

.. list-table:: variables in pixel rate calculation
   :header-rows: 1

   * - variable or constant
     - description
   * - link_freq
     - The value of the ``V4L2_CID_LINK_FREQ`` integer64 menu item.
   * - nr_of_lanes
     - Number of data lanes used on the CSI-2 link. This can
       be obtained from the OF endpoint configuration.
   * - 2
     - Data is transferred on both rising and falling edge of the signal.
   * - bits_per_sample
     - Number of bits per sample.
   * - k
     - 16 for D-PHY and 7 for C-PHY

.. note::

	The pixel rate calculated this way is **not** the same thing as the
	pixel rate on the camera sensor's pixel array which is indicated by the
	:ref:`V4L2_CID_PIXEL_RATE <v4l2-cid-pixel-rate>` control.

LP-11 and LP-111 modes
^^^^^^^^^^^^^^^^^^^^^^

As part of transitioning to high speed mode, a CSI-2 transmitter typically
briefly sets the bus to LP-11 or LP-111 state, depending on the PHY. This period
may be as short as 100 µs, during which the receiver observes this state and
proceeds its own part of high speed mode transition.

Most receivers are capable of autonomously handling this once the software has
configured them to do so, but there are receivers which require software
involvement in observing LP-11 or LP-111 state. 100 µs is a brief period to hit
in software, especially when there is no interrupt telling something is
happening.

One way to address this is to configure the transmitter side explicitly to LP-11
or LP-111 mode, which requires support from the transmitter hardware. This is
not universally available. Many devices return to this state once streaming is
stopped while the state after power-on is LP-00 or LP-000.

The ``.pre_streamon()`` callback may be used to prepare a transmitter for
transitioning to streaming state, but not yet start streaming. Similarly, the
``.post_streamoff()`` callback is used to undo what was done by the
``.pre_streamon()`` callback. The caller of ``.pre_streamon()`` is thus required
to call ``.post_streamoff()`` for each successful call of ``.pre_streamon()``.

In the context of CSI-2, the ``.pre_streamon()`` callback is used to transition
the transmitter to the LP-11 or LP-111 mode. This also requires powering on the
device, so this should be only done when it is needed.

Receiver drivers that do not need explicit LP-11 or LP-111 mode setup are waived
from calling the two callbacks.

Stopping the transmitter
^^^^^^^^^^^^^^^^^^^^^^^^

A transmitter stops sending the stream of images as a result of
calling the ``.s_stream()`` callback. Some transmitters may stop the
stream at a frame boundary whereas others stop immediately,
effectively leaving the current frame unfinished. The receiver driver
should not make assumptions either way, but function properly in both
cases.


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 async kAPI
^^^^^^^^^^^^^^^
.. kernel-doc:: include/media/v4l2-async.h


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 common functions and data structures
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/v4l2-common.h

.. kernel-doc:: include/media/v4l2-ioctl.h


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 Controls
=============

Introduction
------------

The V4L2 control API seems simple enough, but quickly becomes very hard to
implement correctly in drivers. But much of the code needed to handle controls
is actually not driver specific and can be moved to the V4L core framework.

After all, the only part that a driver developer is interested in is:

1) How do I add a control?
2) How do I set the control's value? (i.e. s_ctrl)

And occasionally:

3) How do I get the control's value? (i.e. g_volatile_ctrl)
4) How do I validate the user's proposed control value? (i.e. try_ctrl)

All the rest is something that can be done centrally.

The control framework was created in order to implement all the rules of the
V4L2 specification with respect to controls in a central place. And to make
life as easy as possible for the driver developer.

Note that the control framework relies on the presence of a struct
:c:type:`v4l2_device` for V4L2 drivers and struct v4l2_subdev for
sub-device drivers.

Objects in the framework
------------------------

There are two main objects:

The :c:type:`v4l2_ctrl` object describes the control properties and keeps
track of the control's value (both the current value and the proposed new
value).

:c:type:`v4l2_ctrl_handler` is the object that keeps track of controls. It
maintains a list of v4l2_ctrl objects that it owns and another list of
references to controls, possibly to controls owned by other handlers.

Basic usage for V4L2 and sub-device drivers
-------------------------------------------

1) Prepare the driver:

.. code-block:: c

	#include <media/v4l2-ctrls.h>

1.1) Add the handler to your driver's top-level struct:

For V4L2 drivers:

.. code-block:: c

	struct foo_dev {
		...
		struct v4l2_device v4l2_dev;
		...
		struct v4l2_ctrl_handler ctrl_handler;
		...
	};

For sub-device drivers:

.. code-block:: c

	struct foo_dev {
		...
		struct v4l2_subdev sd;
		...
		struct v4l2_ctrl_handler ctrl_handler;
		...
	};

1.2) Initialize the handler:

.. code-block:: c

	v4l2_ctrl_handler_init(&foo->ctrl_handler, nr_of_controls);

The second argument is a hint telling the function how many controls this
handler is expected to handle. It will allocate a hashtable based on this
information. It is a hint only.

1.3) Hook the control handler into the driver:

For V4L2 drivers:

.. code-block:: c

	foo->v4l2_dev.ctrl_handler = &foo->ctrl_handler;

For sub-device drivers:

.. code-block:: c

	foo->sd.ctrl_handler = &foo->ctrl_handler;

1.4) Clean up the handler at the end:

.. code-block:: c

	v4l2_ctrl_handler_free(&foo->ctrl_handler);

2) Add controls:

You add non-menu controls by calling :c:func:`v4l2_ctrl_new_std`:

.. code-block:: c

	struct v4l2_ctrl *v4l2_ctrl_new_std(struct v4l2_ctrl_handler *hdl,
			const struct v4l2_ctrl_ops *ops,
			u32 id, s32 min, s32 max, u32 step, s32 def);

Menu and integer menu controls are added by calling
:c:func:`v4l2_ctrl_new_std_menu`:

.. code-block:: c

	struct v4l2_ctrl *v4l2_ctrl_new_std_menu(struct v4l2_ctrl_handler *hdl,
			const struct v4l2_ctrl_ops *ops,
			u32 id, s32 max, s32 skip_mask, s32 def);

Menu controls with a driver specific menu are added by calling
:c:func:`v4l2_ctrl_new_std_menu_items`:

.. code-block:: c

       struct v4l2_ctrl *v4l2_ctrl_new_std_menu_items(
                       struct v4l2_ctrl_handler *hdl,
                       const struct v4l2_ctrl_ops *ops, u32 id, s32 max,
                       s32 skip_mask, s32 def, const char * const *qmenu);

Standard compound controls can be added by calling
:c:func:`v4l2_ctrl_new_std_compound`:

.. code-block:: c

       struct v4l2_ctrl *v4l2_ctrl_new_std_compound(struct v4l2_ctrl_handler *hdl,
                       const struct v4l2_ctrl_ops *ops, u32 id,
                       const union v4l2_ctrl_ptr p_def);

Integer menu controls with a driver specific menu can be added by calling
:c:func:`v4l2_ctrl_new_int_menu`:

.. code-block:: c

	struct v4l2_ctrl *v4l2_ctrl_new_int_menu(struct v4l2_ctrl_handler *hdl,
			const struct v4l2_ctrl_ops *ops,
			u32 id, s32 max, s32 def, const s64 *qmenu_int);

These functions are typically called right after the
:c:func:`v4l2_ctrl_handler_init`:

.. code-block:: c

	static const s64 exp_bias_qmenu[] = {
	       -2, -1, 0, 1, 2
	};
	static const char * const test_pattern[] = {
		"Disabled",
		"Vertical Bars",
		"Solid Black",
		"Solid White",
	};

	v4l2_ctrl_handler_init(&foo->ctrl_handler, nr_of_controls);
	v4l2_ctrl_new_std(&foo->ctrl_handler, &foo_ctrl_ops,
			V4L2_CID_BRIGHTNESS, 0, 255, 1, 128);
	v4l2_ctrl_new_std(&foo->ctrl_handler, &foo_ctrl_ops,
			V4L2_CID_CONTRAST, 0, 255, 1, 128);
	v4l2_ctrl_new_std_menu(&foo->ctrl_handler, &foo_ctrl_ops,
			V4L2_CID_POWER_LINE_FREQUENCY,
			V4L2_CID_POWER_LINE_FREQUENCY_60HZ, 0,
			V4L2_CID_POWER_LINE_FREQUENCY_DISABLED);
	v4l2_ctrl_new_int_menu(&foo->ctrl_handler, &foo_ctrl_ops,
			V4L2_CID_EXPOSURE_BIAS,
			ARRAY_SIZE(exp_bias_qmenu) - 1,
			ARRAY_SIZE(exp_bias_qmenu) / 2 - 1,
			exp_bias_qmenu);
	v4l2_ctrl_new_std_menu_items(&foo->ctrl_handler, &foo_ctrl_ops,
			V4L2_CID_TEST_PATTERN, ARRAY_SIZE(test_pattern) - 1, 0,
			0, test_pattern);
	...
	if (foo->ctrl_handler.error) {
		int err = foo->ctrl_handler.error;

		v4l2_ctrl_handler_free(&foo->ctrl_handler);
		return err;
	}

The :c:func:`v4l2_ctrl_new_std` function returns the v4l2_ctrl pointer to
the new control, but if you do not need to access the pointer outside the
control ops, then there is no need to store it.

The :c:func:`v4l2_ctrl_new_std` function will fill in most fields based on
the control ID except for the min, max, step and default values. These are
passed in the last four arguments. These values are driver specific while
control attributes like type, name, flags are all global. The control's
current value will be set to the default value.

The :c:func:`v4l2_ctrl_new_std_menu` function is very similar but it is
used for menu controls. There is no min argument since that is always 0 for
menu controls, and instead of a step there is a skip_mask argument: if bit
X is 1, then menu item X is skipped.

The :c:func:`v4l2_ctrl_new_int_menu` function creates a new standard
integer menu control with driver-specific items in the menu. It differs
from v4l2_ctrl_new_std_menu in that it doesn't have the mask argument and
takes as the last argument an array of signed 64-bit integers that form an
exact menu item list.

The :c:func:`v4l2_ctrl_new_std_menu_items` function is very similar to
v4l2_ctrl_new_std_menu but takes an extra parameter qmenu, which is the
driver specific menu for an otherwise standard menu control. A good example
for this control is the test pattern control for capture/display/sensors
devices that have the capability to generate test patterns. These test
patterns are hardware specific, so the contents of the menu will vary from
device to device.

Note that if something fails, the function will return NULL or an error and
set ctrl_handler->error to the error code. If ctrl_handler->error was already
set, then it will just return and do nothing. This is also true for
v4l2_ctrl_handler_init if it cannot allocate the internal data structure.

This makes it easy to init the handler and just add all controls and only check
the error code at the end. Saves a lot of repetitive error checking.

It is recommended to add controls in ascending control ID order: it will be
a bit faster that way.

3) Optionally force initial control setup:

.. code-block:: c

	v4l2_ctrl_handler_setup(&foo->ctrl_handler);

This will call s_ctrl for all controls unconditionally. Effectively this
initializes the hardware to the default control values. It is recommended
that you do this as this ensures that both the internal data structures and
the hardware are in sync.

4) Finally: implement the :c:type:`v4l2_ctrl_ops`

.. code-block:: c

	static const struct v4l2_ctrl_ops foo_ctrl_ops = {
		.s_ctrl = foo_s_ctrl,
	};

Usually all you need is s_ctrl:

.. code-block:: c

	static int foo_s_ctrl(struct v4l2_ctrl *ctrl)
	{
		struct foo *state = container_of(ctrl->handler, struct foo, ctrl_handler);

		switch (ctrl->id) {
		case V4L2_CID_BRIGHTNESS:
			write_reg(0x123, ctrl->val);
			break;
		case V4L2_CID_CONTRAST:
			write_reg(0x456, ctrl->val);
			break;
		}
		return 0;
	}

The control ops are called with the v4l2_ctrl pointer as argument.
The new control value has already been validated, so all you need to do is
to actually update the hardware registers.

You're done! And this is sufficient for most of the drivers we have. No need
to do any validation of control values, or implement QUERYCTRL, QUERY_EXT_CTRL
and QUERYMENU. And G/S_CTRL as well as G/TRY/S_EXT_CTRLS are automatically supported.

.. note::

   The remainder sections deal with more advanced controls topics and scenarios.
   In practice the basic usage as described above is sufficient for most drivers.

Inheriting Sub-device Controls
------------------------------

When a sub-device is registered with a V4L2 driver by calling
v4l2_device_register_subdev() and the ctrl_handler fields of both v4l2_subdev
and v4l2_device are set, then the controls of the subdev will become
automatically available in the V4L2 driver as well. If the subdev driver
contains controls that already exist in the V4L2 driver, then those will be
skipped (so a V4L2 driver can always override a subdev control).

What happens here is that v4l2_device_register_subdev() calls
v4l2_ctrl_add_handler() adding the controls of the subdev to the controls
of v4l2_device.

Accessing Control Values
------------------------

The following union is used inside the control framework to access control
values:

.. code-block:: c

	union v4l2_ctrl_ptr {
		s32 *p_s32;
		s64 *p_s64;
		char *p_char;
		void *p;
	};

The v4l2_ctrl struct contains these fields that can be used to access both
current and new values:

.. code-block:: c

	s32 val;
	struct {
		s32 val;
	} cur;

	union v4l2_ctrl_ptr p_new;
	union v4l2_ctrl_ptr p_cur;

If the control has a simple s32 type, then:

.. code-block:: c

	&ctrl->val == ctrl->p_new.p_s32
	&ctrl->cur.val == ctrl->p_cur.p_s32

For all other types use ctrl->p_cur.p<something>. Basically the val
and cur.val fields can be considered an alias since these are used so often.

Within the control ops you can freely use these. The val and cur.val speak for
themselves. The p_char pointers point to character buffers of length
ctrl->maximum + 1, and are always 0-terminated.

Unless the control is marked volatile the p_cur field points to the
current cached control value. When you create a new control this value is made
identical to the default value. After calling v4l2_ctrl_handler_setup() this
value is passed to the hardware. It is generally a good idea to call this
function.

Whenever a new value is set that new value is automatically cached. This means
that most drivers do not need to implement the g_volatile_ctrl() op. The
exception is for controls that return a volatile register such as a signal
strength read-out that changes continuously. In that case you will need to
implement g_volatile_ctrl like this:

.. code-block:: c

	static int foo_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
	{
		switch (ctrl->id) {
		case V4L2_CID_BRIGHTNESS:
			ctrl->val = read_reg(0x123);
			break;
		}
	}

Note that you use the 'new value' union as well in g_volatile_ctrl. In general
controls that need to implement g_volatile_ctrl are read-only controls. If they
are not, a V4L2_EVENT_CTRL_CH_VALUE will not be generated when the control
changes.

To mark a control as volatile you have to set V4L2_CTRL_FLAG_VOLATILE:

.. code-block:: c

	ctrl = v4l2_ctrl_new_std(&sd->ctrl_handler, ...);
	if (ctrl)
		ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;

For try/s_ctrl the new values (i.e. as passed by the user) are filled in and
you can modify them in try_ctrl or set them in s_ctrl. The 'cur' union
contains the current value, which you can use (but not change!) as well.

If s_ctrl returns 0 (OK), then the control framework will copy the new final
values to the 'cur' union.

While in g_volatile/s/try_ctrl you can access the value of all controls owned
by the same handler since the handler's lock is held. If you need to access
the value of controls owned by other handlers, then you have to be very careful
not to introduce deadlocks.

Outside of the control ops you have to go through to helper functions to get
or set a single control value safely in your driver:

.. code-block:: c

	s32 v4l2_ctrl_g_ctrl(struct v4l2_ctrl *ctrl);
	int v4l2_ctrl_s_ctrl(struct v4l2_ctrl *ctrl, s32 val);

These functions go through the control framework just as VIDIOC_G/S_CTRL ioctls
do. Don't use these inside the control ops g_volatile/s/try_ctrl, though, that
will result in a deadlock since these helpers lock the handler as well.

You can also take the handler lock yourself:

.. code-block:: c

	mutex_lock(&state->ctrl_handler.lock);
	pr_info("String value is '%s'\n", ctrl1->p_cur.p_char);
	pr_info("Integer value is '%s'\n", ctrl2->cur.val);
	mutex_unlock(&state->ctrl_handler.lock);

Menu Controls
-------------

The v4l2_ctrl struct contains this union:

.. code-block:: c

	union {
		u32 step;
		u32 menu_skip_mask;
	};

For menu controls menu_skip_mask is used. What it does is that it allows you
to easily exclude certain menu items. This is used in the VIDIOC_QUERYMENU
implementation where you can return -EINVAL if a certain menu item is not
present. Note that VIDIOC_QUERYCTRL always returns a step value of 1 for
menu controls.

A good example is the MPEG Audio Layer II Bitrate menu control where the
menu is a list of standardized possible bitrates. But in practice hardware
implementations will only support a subset of those. By setting the skip
mask you can tell the framework which menu items should be skipped. Setting
it to 0 means that all menu items are supported.

You set this mask either through the v4l2_ctrl_config struct for a custom
control, or by calling v4l2_ctrl_new_std_menu().

Custom Controls
---------------

Driver specific controls can be created using v4l2_ctrl_new_custom():

.. code-block:: c

	static const struct v4l2_ctrl_config ctrl_filter = {
		.ops = &ctrl_custom_ops,
		.id = V4L2_CID_MPEG_CX2341X_VIDEO_SPATIAL_FILTER,
		.name = "Spatial Filter",
		.type = V4L2_CTRL_TYPE_INTEGER,
		.flags = V4L2_CTRL_FLAG_SLIDER,
		.max = 15,
		.step = 1,
	};

	ctrl = v4l2_ctrl_new_custom(&foo->ctrl_handler, &ctrl_filter, NULL);

The last argument is the priv pointer which can be set to driver-specific
private data.

The v4l2_ctrl_config struct also has a field to set the is_private flag.

If the name field is not set, then the framework will assume this is a standard
control and will fill in the name, type and flags fields accordingly.

Active and Grabbed Controls
---------------------------

If you get more complex relationships between controls, then you may have to
activate and deactivate controls. For example, if the Chroma AGC control is
on, then the Chroma Gain control is inactive. That is, you may set it, but
the value will not be used by the hardware as long as the automatic gain
control is on. Typically user interfaces can disable such input fields.

You can set the 'active' status using v4l2_ctrl_activate(). By default all
controls are active. Note that the framework does not check for this flag.
It is meant purely for GUIs. The function is typically called from within
s_ctrl.

The other flag is the 'grabbed' flag. A grabbed control means that you cannot
change it because it is in use by some resource. Typical examples are MPEG
bitrate controls that cannot be changed while capturing is in progress.

If a control is set to 'grabbed' using v4l2_ctrl_grab(), then the framework
will return -EBUSY if an attempt is made to set this control. The
v4l2_ctrl_grab() function is typically called from the driver when it
starts or stops streaming.

Control Clusters
----------------

By default all controls are independent from the others. But in more
complex scenarios you can get dependencies from one control to another.
In that case you need to 'cluster' them:

.. code-block:: c

	struct foo {
		struct v4l2_ctrl_handler ctrl_handler;
	#define AUDIO_CL_VOLUME (0)
	#define AUDIO_CL_MUTE   (1)
		struct v4l2_ctrl *audio_cluster[2];
		...
	};

	state->audio_cluster[AUDIO_CL_VOLUME] =
		v4l2_ctrl_new_std(&state->ctrl_handler, ...);
	state->audio_cluster[AUDIO_CL_MUTE] =
		v4l2_ctrl_new_std(&state->ctrl_handler, ...);
	v4l2_ctrl_cluster(ARRAY_SIZE(state->audio_cluster), state->audio_cluster);

From now on whenever one or more of the controls belonging to the same
cluster is set (or 'gotten', or 'tried'), only the control ops of the first
control ('volume' in this example) is called. You effectively create a new
composite control. Similar to how a 'struct' works in C.

So when s_ctrl is called with V4L2_CID_AUDIO_VOLUME as argument, you should set
all two controls belonging to the audio_cluster:

.. code-block:: c

	static int foo_s_ctrl(struct v4l2_ctrl *ctrl)
	{
		struct foo *state = container_of(ctrl->handler, struct foo, ctrl_handler);

		switch (ctrl->id) {
		case V4L2_CID_AUDIO_VOLUME: {
			struct v4l2_ctrl *mute = ctrl->cluster[AUDIO_CL_MUTE];

			write_reg(0x123, mute->val ? 0 : ctrl->val);
			break;
		}
		case V4L2_CID_CONTRAST:
			write_reg(0x456, ctrl->val);
			break;
		}
		return 0;
	}

In the example above the following are equivalent for the VOLUME case:

.. code-block:: c

	ctrl == ctrl->cluster[AUDIO_CL_VOLUME] == state->audio_cluster[AUDIO_CL_VOLUME]
	ctrl->cluster[AUDIO_CL_MUTE] == state->audio_cluster[AUDIO_CL_MUTE]

In practice using cluster arrays like this becomes very tiresome. So instead
the following equivalent method is used:

.. code-block:: c

	struct {
		/* audio cluster */
		struct v4l2_ctrl *volume;
		struct v4l2_ctrl *mute;
	};

The anonymous struct is used to clearly 'cluster' these two control pointers,
but it serves no other purpose. The effect is the same as creating an
array with two control pointers. So you can just do:

.. code-block:: c

	state->volume = v4l2_ctrl_new_std(&state->ctrl_handler, ...);
	state->mute = v4l2_ctrl_new_std(&state->ctrl_handler, ...);
	v4l2_ctrl_cluster(2, &state->volume);

And in foo_s_ctrl you can use these pointers directly: state->mute->val.

Note that controls in a cluster may be NULL. For example, if for some
reason mute was never added (because the hardware doesn't support that
particular feature), then mute will be NULL. So in that case we have a
cluster of 2 controls, of which only 1 is actually instantiated. The
only restriction is that the first control of the cluster must always be
present, since that is the 'master' control of the cluster. The master
control is the one that identifies the cluster and that provides the
pointer to the v4l2_ctrl_ops struct that is used for that cluster.

Obviously, all controls in the cluster array must be initialized to either
a valid control or to NULL.

In rare cases you might want to know which controls of a cluster actually
were set explicitly by the user. For this you can check the 'is_new' flag of
each control. For example, in the case of a volume/mute cluster the 'is_new'
flag of the mute control would be set if the user called VIDIOC_S_CTRL for
mute only. If the user would call VIDIOC_S_EXT_CTRLS for both mute and volume
controls, then the 'is_new' flag would be 1 for both controls.

The 'is_new' flag is always 1 when called from v4l2_ctrl_handler_setup().

Handling autogain/gain-type Controls with Auto Clusters
-------------------------------------------------------

A common type of control cluster is one that handles 'auto-foo/foo'-type
controls. Typical examples are autogain/gain, autoexposure/exposure,
autowhitebalance/red balance/blue balance. In all cases you have one control
that determines whether another control is handled automatically by the hardware,
or whether it is under manual control from the user.

If the cluster is in automatic mode, then the manual controls should be
marked inactive and volatile. When the volatile controls are read the
g_volatile_ctrl operation should return the value that the hardware's automatic
mode set up automatically.

If the cluster is put in manual mode, then the manual controls should become
active again and the volatile flag is cleared (so g_volatile_ctrl is no longer
called while in manual mode). In addition just before switching to manual mode
the current values as determined by the auto mode are copied as the new manual
values.

Finally the V4L2_CTRL_FLAG_UPDATE should be set for the auto control since
changing that control affects the control flags of the manual controls.

In order to simplify this a special variation of v4l2_ctrl_cluster was
introduced:

.. code-block:: c

	void v4l2_ctrl_auto_cluster(unsigned ncontrols, struct v4l2_ctrl **controls,
				    u8 manual_val, bool set_volatile);

The first two arguments are identical to v4l2_ctrl_cluster. The third argument
tells the framework which value switches the cluster into manual mode. The
last argument will optionally set V4L2_CTRL_FLAG_VOLATILE for the non-auto controls.
If it is false, then the manual controls are never volatile. You would typically
use that if the hardware does not give you the option to read back to values as
determined by the auto mode (e.g. if autogain is on, the hardware doesn't allow
you to obtain the current gain value).

The first control of the cluster is assumed to be the 'auto' control.

Using this function will ensure that you don't need to handle all the complex
flag and volatile handling.

VIDIOC_LOG_STATUS Support
-------------------------

This ioctl allow you to dump the current status of a driver to the kernel log.
The v4l2_ctrl_handler_log_status(ctrl_handler, prefix) can be used to dump the
value of the controls owned by the given handler to the log. You can supply a
prefix as well. If the prefix didn't end with a space, then ': ' will be added
for you.

Different Handlers for Different Video Nodes
--------------------------------------------

Usually the V4L2 driver has just one control handler that is global for
all video nodes. But you can also specify different control handlers for
different video nodes. You can do that by manually setting the ctrl_handler
field of struct video_device.

That is no problem if there are no subdevs involved but if there are, then
you need to block the automatic merging of subdev controls to the global
control handler. You do that by simply setting the ctrl_handler field in
struct v4l2_device to NULL. Now v4l2_device_register_subdev() will no longer
merge subdev controls.

After each subdev was added, you will then have to call v4l2_ctrl_add_handler
manually to add the subdev's control handler (sd->ctrl_handler) to the desired
control handler. This control handler may be specific to the video_device or
for a subset of video_device's. For example: the radio device nodes only have
audio controls, while the video and vbi device nodes share the same control
handler for the audio and video controls.

If you want to have one handler (e.g. for a radio device node) have a subset
of another handler (e.g. for a video device node), then you should first add
the controls to the first handler, add the other controls to the second
handler and finally add the first handler to the second. For example:

.. code-block:: c

	v4l2_ctrl_new_std(&radio_ctrl_handler, &radio_ops, V4L2_CID_AUDIO_VOLUME, ...);
	v4l2_ctrl_new_std(&radio_ctrl_handler, &radio_ops, V4L2_CID_AUDIO_MUTE, ...);
	v4l2_ctrl_new_std(&video_ctrl_handler, &video_ops, V4L2_CID_BRIGHTNESS, ...);
	v4l2_ctrl_new_std(&video_ctrl_handler, &video_ops, V4L2_CID_CONTRAST, ...);
	v4l2_ctrl_add_handler(&video_ctrl_handler, &radio_ctrl_handler, NULL);

The last argument to v4l2_ctrl_add_handler() is a filter function that allows
you to filter which controls will be added. Set it to NULL if you want to add
all controls.

Or you can add specific controls to a handler:

.. code-block:: c

	volume = v4l2_ctrl_new_std(&video_ctrl_handler, &ops, V4L2_CID_AUDIO_VOLUME, ...);
	v4l2_ctrl_new_std(&video_ctrl_handler, &ops, V4L2_CID_BRIGHTNESS, ...);
	v4l2_ctrl_new_std(&video_ctrl_handler, &ops, V4L2_CID_CONTRAST, ...);

What you should not do is make two identical controls for two handlers.
For example:

.. code-block:: c

	v4l2_ctrl_new_std(&radio_ctrl_handler, &radio_ops, V4L2_CID_AUDIO_MUTE, ...);
	v4l2_ctrl_new_std(&video_ctrl_handler, &video_ops, V4L2_CID_AUDIO_MUTE, ...);

This would be bad since muting the radio would not change the video mute
control. The rule is to have one control for each hardware 'knob' that you
can twiddle.

Finding Controls
----------------

Normally you have created the controls yourself and you can store the struct
v4l2_ctrl pointer into your own struct.

But sometimes you need to find a control from another handler that you do
not own. For example, if you have to find a volume control from a subdev.

You can do that by calling v4l2_ctrl_find:

.. code-block:: c

	struct v4l2_ctrl *volume;

	volume = v4l2_ctrl_find(sd->ctrl_handler, V4L2_CID_AUDIO_VOLUME);

Since v4l2_ctrl_find will lock the handler you have to be careful where you
use it. For example, this is not a good idea:

.. code-block:: c

	struct v4l2_ctrl_handler ctrl_handler;

	v4l2_ctrl_new_std(&ctrl_handler, &video_ops, V4L2_CID_BRIGHTNESS, ...);
	v4l2_ctrl_new_std(&ctrl_handler, &video_ops, V4L2_CID_CONTRAST, ...);

...and in video_ops.s_ctrl:

.. code-block:: c

	case V4L2_CID_BRIGHTNESS:
		contrast = v4l2_find_ctrl(&ctrl_handler, V4L2_CID_CONTRAST);
		...

When s_ctrl is called by the framework the ctrl_handler.lock is already taken, so
attempting to find another control from the same handler will deadlock.

It is recommended not to use this function from inside the control ops.

Preventing Controls inheritance
-------------------------------

When one control handler is added to another using v4l2_ctrl_add_handler, then
by default all controls from one are merged to the other. But a subdev might
have low-level controls that make sense for some advanced embedded system, but
not when it is used in consumer-level hardware. In that case you want to keep
those low-level controls local to the subdev. You can do this by simply
setting the 'is_private' flag of the control to 1:

.. code-block:: c

	static const struct v4l2_ctrl_config ctrl_private = {
		.ops = &ctrl_custom_ops,
		.id = V4L2_CID_...,
		.name = "Some Private Control",
		.type = V4L2_CTRL_TYPE_INTEGER,
		.max = 15,
		.step = 1,
		.is_private = 1,
	};

	ctrl = v4l2_ctrl_new_custom(&foo->ctrl_handler, &ctrl_private, NULL);

These controls will now be skipped when v4l2_ctrl_add_handler is called.

V4L2_CTRL_TYPE_CTRL_CLASS Controls
----------------------------------

Controls of this type can be used by GUIs to get the name of the control class.
A fully featured GUI can make a dialog with multiple tabs with each tab
containing the controls belonging to a particular control class. The name of
each tab can be found by querying a special control with ID <control class | 1>.

Drivers do not have to care about this. The framework will automatically add
a control of this type whenever the first control belonging to a new control
class is added.

Adding Notify Callbacks
-----------------------

Sometimes the platform or bridge driver needs to be notified when a control
from a sub-device driver changes. You can set a notify callback by calling
this function:

.. code-block:: c

	void v4l2_ctrl_notify(struct v4l2_ctrl *ctrl,
		void (*notify)(struct v4l2_ctrl *ctrl, void *priv), void *priv);

Whenever the give control changes value the notify callback will be called
with a pointer to the control and the priv pointer that was passed with
v4l2_ctrl_notify. Note that the control's handler lock is held when the
notify function is called.

There can be only one notify function per control handler. Any attempt
to set another notify function will cause a WARN_ON.

v4l2_ctrl functions and data structures
---------------------------------------

.. kernel-doc:: include/media/v4l2-ctrls.h


code:
.. SPDX-License-Identifier: GPL-2.0

Video4Linux devices
-------------------

.. toctree::
    :maxdepth: 1

    v4l2-intro
    v4l2-dev
    v4l2-device
    v4l2-fh
    v4l2-subdev
    v4l2-event
    v4l2-controls
    v4l2-videobuf
    v4l2-videobuf2
    v4l2-dv-timings
    v4l2-flash-led-class
    v4l2-mc
    v4l2-mediabus
    v4l2-mem2mem
    v4l2-async
    v4l2-fwnode
    v4l2-rect
    v4l2-tuner
    v4l2-common
    v4l2-tveeprom


code:
.. SPDX-License-Identifier: GPL-2.0

Video device' s internal representation
=======================================

The actual device nodes in the ``/dev`` directory are created using the
:c:type:`video_device` struct (``v4l2-dev.h``). This struct can either be
allocated dynamically or embedded in a larger struct.

To allocate it dynamically use :c:func:`video_device_alloc`:

.. code-block:: c

	struct video_device *vdev = video_device_alloc();

	if (vdev == NULL)
		return -ENOMEM;

	vdev->release = video_device_release;

If you embed it in a larger struct, then you must set the ``release()``
callback to your own function:

.. code-block:: c

	struct video_device *vdev = &my_vdev->vdev;

	vdev->release = my_vdev_release;

The ``release()`` callback must be set and it is called when the last user
of the video device exits.

The default :c:func:`video_device_release` callback currently
just calls ``kfree`` to free the allocated memory.

There is also a :c:func:`video_device_release_empty` function that does
nothing (is empty) and should be used if the struct is embedded and there
is nothing to do when it is released.

You should also set these fields of :c:type:`video_device`:

- :c:type:`video_device`->v4l2_dev: must be set to the :c:type:`v4l2_device`
  parent device.

- :c:type:`video_device`->name: set to something descriptive and unique.

- :c:type:`video_device`->vfl_dir: set this to ``VFL_DIR_RX`` for capture
  devices (``VFL_DIR_RX`` has value 0, so this is normally already the
  default), set to ``VFL_DIR_TX`` for output devices and ``VFL_DIR_M2M`` for mem2mem (codec) devices.

- :c:type:`video_device`->fops: set to the :c:type:`v4l2_file_operations`
  struct.

- :c:type:`video_device`->ioctl_ops: if you use the :c:type:`v4l2_ioctl_ops`
  to simplify ioctl maintenance (highly recommended to use this and it might
  become compulsory in the future!), then set this to your
  :c:type:`v4l2_ioctl_ops` struct. The :c:type:`video_device`->vfl_type and
  :c:type:`video_device`->vfl_dir fields are used to disable ops that do not
  match the type/dir combination. E.g. VBI ops are disabled for non-VBI nodes,
  and output ops  are disabled for a capture device. This makes it possible to
  provide just one :c:type:`v4l2_ioctl_ops` struct for both vbi and
  video nodes.

- :c:type:`video_device`->lock: leave to ``NULL`` if you want to do all the
  locking  in the driver. Otherwise you give it a pointer to a struct
  ``mutex_lock`` and before the :c:type:`video_device`->unlocked_ioctl
  file operation is called this lock will be taken by the core and released
  afterwards. See the next section for more details.

- :c:type:`video_device`->queue: a pointer to the struct vb2_queue
  associated with this device node.
  If queue is not ``NULL``, and queue->lock is not ``NULL``, then queue->lock
  is used for the queuing ioctls (``VIDIOC_REQBUFS``, ``CREATE_BUFS``,
  ``QBUF``, ``DQBUF``,  ``QUERYBUF``, ``PREPARE_BUF``, ``STREAMON`` and
  ``STREAMOFF``) instead of the lock above.
  That way the :ref:`vb2 <vb2_framework>` queuing framework does not have
  to wait for other ioctls.   This queue pointer is also used by the
  :ref:`vb2 <vb2_framework>` helper functions to check for
  queuing ownership (i.e. is the filehandle calling it allowed to do the
  operation).

- :c:type:`video_device`->prio: keeps track of the priorities. Used to
  implement ``VIDIOC_G_PRIORITY`` and ``VIDIOC_S_PRIORITY``.
  If left to ``NULL``, then it will use the struct v4l2_prio_state
  in :c:type:`v4l2_device`. If you want to have a separate priority state per
  (group of) device node(s),   then you can point it to your own struct
  :c:type:`v4l2_prio_state`.

- :c:type:`video_device`->dev_parent: you only set this if v4l2_device was
  registered with ``NULL`` as the parent ``device`` struct. This only happens
  in cases where one hardware device has multiple PCI devices that all share
  the same :c:type:`v4l2_device` core.

  The cx88 driver is an example of this: one core :c:type:`v4l2_device` struct,
  but   it is used by both a raw video PCI device (cx8800) and a MPEG PCI device
  (cx8802). Since the :c:type:`v4l2_device` cannot be associated with two PCI
  devices at the same time it is setup without a parent device. But when the
  struct video_device is initialized you **do** know which parent
  PCI device to use and so you set ``dev_device`` to the correct PCI device.

If you use :c:type:`v4l2_ioctl_ops`, then you should set
:c:type:`video_device`->unlocked_ioctl to :c:func:`video_ioctl2` in your
:c:type:`v4l2_file_operations` struct.

In some cases you want to tell the core that a function you had specified in
your :c:type:`v4l2_ioctl_ops` should be ignored. You can mark such ioctls by
calling this function before :c:func:`video_register_device` is called:

	:c:func:`v4l2_disable_ioctl <v4l2_disable_ioctl>`
	(:c:type:`vdev <video_device>`, cmd).

This tends to be needed if based on external factors (e.g. which card is
being used) you want to turns off certain features in :c:type:`v4l2_ioctl_ops`
without having to make a new struct.

The :c:type:`v4l2_file_operations` struct is a subset of file_operations.
The main difference is that the inode argument is omitted since it is never
used.

If integration with the media framework is needed, you must initialize the
:c:type:`media_entity` struct embedded in the :c:type:`video_device` struct
(entity field) by calling :c:func:`media_entity_pads_init`:

.. code-block:: c

	struct media_pad *pad = &my_vdev->pad;
	int err;

	err = media_entity_pads_init(&vdev->entity, 1, pad);

The pads array must have been previously initialized. There is no need to
manually set the struct media_entity type and name fields.

A reference to the entity will be automatically acquired/released when the
video device is opened/closed.

ioctls and locking
------------------

The V4L core provides optional locking services. The main service is the
lock field in struct video_device, which is a pointer to a mutex.
If you set this pointer, then that will be used by unlocked_ioctl to
serialize all ioctls.

If you are using the :ref:`videobuf2 framework <vb2_framework>`, then there
is a second lock that you can set: :c:type:`video_device`->queue->lock. If
set, then this lock will be used instead of :c:type:`video_device`->lock
to serialize all queuing ioctls (see the previous section
for the full list of those ioctls).

The advantage of using a different lock for the queuing ioctls is that for some
drivers (particularly USB drivers) certain commands such as setting controls
can take a long time, so you want to use a separate lock for the buffer queuing
ioctls. That way your ``VIDIOC_DQBUF`` doesn't stall because the driver is busy
changing the e.g. exposure of the webcam.

Of course, you can always do all the locking yourself by leaving both lock
pointers at ``NULL``.

If you use the old :ref:`videobuf framework <vb_framework>` then you must
pass the :c:type:`video_device`->lock to the videobuf queue initialize
function: if videobuf has to wait for a frame to arrive, then it will
temporarily unlock the lock and relock it afterwards. If your driver also
waits in the code, then you should do the same to allow other
processes to access the device node while the first process is waiting for
something.

In the case of :ref:`videobuf2 <vb2_framework>` you will need to implement the
``wait_prepare()`` and ``wait_finish()`` callbacks to unlock/lock if applicable.
If you use the ``queue->lock`` pointer, then you can use the helper functions
:c:func:`vb2_ops_wait_prepare` and :c:func:`vb2_ops_wait_finish`.

The implementation of a hotplug disconnect should also take the lock from
:c:type:`video_device` before calling v4l2_device_disconnect. If you are also
using :c:type:`video_device`->queue->lock, then you have to first lock
:c:type:`video_device`->queue->lock followed by :c:type:`video_device`->lock.
That way you can be sure no ioctl is running when you call
:c:func:`v4l2_device_disconnect`.

Video device registration
-------------------------

Next you register the video device with :c:func:`video_register_device`.
This will create the character device for you.

.. code-block:: c

	err = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
	if (err) {
		video_device_release(vdev); /* or kfree(my_vdev); */
		return err;
	}

If the :c:type:`v4l2_device` parent device has a not ``NULL`` mdev field,
the video device entity will be automatically registered with the media
device.

Which device is registered depends on the type argument. The following
types exist:

========================== ====================	 ==============================
:c:type:`vfl_devnode_type` Device name		 Usage
========================== ====================	 ==============================
``VFL_TYPE_VIDEO``         ``/dev/videoX``       for video input/output devices
``VFL_TYPE_VBI``           ``/dev/vbiX``         for vertical blank data (i.e.
						 closed captions, teletext)
``VFL_TYPE_RADIO``         ``/dev/radioX``       for radio tuners
``VFL_TYPE_SUBDEV``        ``/dev/v4l-subdevX``  for V4L2 subdevices
``VFL_TYPE_SDR``           ``/dev/swradioX``     for Software Defined Radio
						 (SDR) tuners
``VFL_TYPE_TOUCH``         ``/dev/v4l-touchX``   for touch sensors
========================== ====================	 ==============================

The last argument gives you a certain amount of control over the device
node number used (i.e. the X in ``videoX``). Normally you will pass -1
to let the v4l2 framework pick the first free number. But sometimes users
want to select a specific node number. It is common that drivers allow
the user to select a specific device node number through a driver module
option. That number is then passed to this function and video_register_device
will attempt to select that device node number. If that number was already
in use, then the next free device node number will be selected and it
will send a warning to the kernel log.

Another use-case is if a driver creates many devices. In that case it can
be useful to place different video devices in separate ranges. For example,
video capture devices start at 0, video output devices start at 16.
So you can use the last argument to specify a minimum device node number
and the v4l2 framework will try to pick the first free number that is equal
or higher to what you passed. If that fails, then it will just pick the
first free number.

Since in this case you do not care about a warning about not being able
to select the specified device node number, you can call the function
:c:func:`video_register_device_no_warn` instead.

Whenever a device node is created some attributes are also created for you.
If you look in ``/sys/class/video4linux`` you see the devices. Go into e.g.
``video0`` and you will see 'name', 'dev_debug' and 'index' attributes. The
'name' attribute is the 'name' field of the video_device struct. The
'dev_debug' attribute can be used to enable core debugging. See the next
section for more detailed information on this.

The 'index' attribute is the index of the device node: for each call to
:c:func:`video_register_device()` the index is just increased by 1. The
first video device node you register always starts with index 0.

Users can setup udev rules that utilize the index attribute to make fancy
device names (e.g. '``mpegX``' for MPEG video capture device nodes).

After the device was successfully registered, then you can use these fields:

- :c:type:`video_device`->vfl_type: the device type passed to
  :c:func:`video_register_device`.
- :c:type:`video_device`->minor: the assigned device minor number.
- :c:type:`video_device`->num: the device node number (i.e. the X in
  ``videoX``).
- :c:type:`video_device`->index: the device index number.

If the registration failed, then you need to call
:c:func:`video_device_release` to free the allocated :c:type:`video_device`
struct, or free your own struct if the :c:type:`video_device` was embedded in
it. The ``vdev->release()`` callback will never be called if the registration
failed, nor should you ever attempt to unregister the device if the
registration failed.

video device debugging
----------------------

The 'dev_debug' attribute that is created for each video, vbi, radio or swradio
device in ``/sys/class/video4linux/<devX>/`` allows you to enable logging of
file operations.

It is a bitmask and the following bits can be set:

.. tabularcolumns:: |p{5ex}|L|

===== ================================================================
Mask  Description
===== ================================================================
0x01  Log the ioctl name and error code. VIDIOC_(D)QBUF ioctls are
      only logged if bit 0x08 is also set.
0x02  Log the ioctl name arguments and error code. VIDIOC_(D)QBUF
      ioctls are
      only logged if bit 0x08 is also set.
0x04  Log the file operations open, release, read, write, mmap and
      get_unmapped_area. The read and write operations are only
      logged if bit 0x08 is also set.
0x08  Log the read and write file operations and the VIDIOC_QBUF and
      VIDIOC_DQBUF ioctls.
0x10  Log the poll file operation.
0x20  Log error and messages in the control operations.
===== ================================================================

Video device cleanup
--------------------

When the video device nodes have to be removed, either during the unload
of the driver or because the USB device was disconnected, then you should
unregister them with:

	:c:func:`video_unregister_device`
	(:c:type:`vdev <video_device>`);

This will remove the device nodes from sysfs (causing udev to remove them
from ``/dev``).

After :c:func:`video_unregister_device` returns no new opens can be done.
However, in the case of USB devices some application might still have one of
these device nodes open. So after the unregister all file operations (except
release, of course) will return an error as well.

When the last user of the video device node exits, then the ``vdev->release()``
callback is called and you can do the final cleanup there.

Don't forget to cleanup the media entity associated with the video device if
it has been initialized:

	:c:func:`media_entity_cleanup <media_entity_cleanup>`
	(&vdev->entity);

This can be done from the release callback.

helper functions
----------------

There are a few useful helper functions:

- file and :c:type:`video_device` private data

You can set/get driver private data in the video_device struct using:

	:c:func:`video_get_drvdata <video_get_drvdata>`
	(:c:type:`vdev <video_device>`);

	:c:func:`video_set_drvdata <video_set_drvdata>`
	(:c:type:`vdev <video_device>`);

Note that you can safely call :c:func:`video_set_drvdata` before calling
:c:func:`video_register_device`.

And this function:

	:c:func:`video_devdata <video_devdata>`
	(struct file \*file);

returns the video_device belonging to the file struct.

The :c:func:`video_devdata` function combines :c:func:`video_get_drvdata`
with :c:func:`video_devdata`:

	:c:func:`video_drvdata <video_drvdata>`
	(struct file \*file);

You can go from a :c:type:`video_device` struct to the v4l2_device struct using:

.. code-block:: c

	struct v4l2_device *v4l2_dev = vdev->v4l2_dev;

- Device node name

The :c:type:`video_device` node kernel name can be retrieved using:

	:c:func:`video_device_node_name <video_device_node_name>`
	(:c:type:`vdev <video_device>`);

The name is used as a hint by userspace tools such as udev. The function
should be used where possible instead of accessing the video_device::num and
video_device::minor fields.

video_device functions and data structures
------------------------------------------

.. kernel-doc:: include/media/v4l2-dev.h


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 device instance
--------------------

Each device instance is represented by a struct v4l2_device.
Very simple devices can just allocate this struct, but most of the time you
would embed this struct inside a larger struct.

You must register the device instance by calling:

	:c:func:`v4l2_device_register <v4l2_device_register>`
	(dev, :c:type:`v4l2_dev <v4l2_device>`).

Registration will initialize the :c:type:`v4l2_device` struct. If the
dev->driver_data field is ``NULL``, it will be linked to
:c:type:`v4l2_dev <v4l2_device>` argument.

Drivers that want integration with the media device framework need to set
dev->driver_data manually to point to the driver-specific device structure
that embed the struct v4l2_device instance. This is achieved by a
``dev_set_drvdata()`` call before registering the V4L2 device instance.
They must also set the struct v4l2_device mdev field to point to a
properly initialized and registered :c:type:`media_device` instance.

If :c:type:`v4l2_dev <v4l2_device>`\ ->name is empty then it will be set to a
value derived from dev (driver name followed by the bus_id, to be precise).
If you set it up before  calling :c:func:`v4l2_device_register` then it will
be untouched. If dev is ``NULL``, then you **must** setup
:c:type:`v4l2_dev <v4l2_device>`\ ->name before calling
:c:func:`v4l2_device_register`.

You can use :c:func:`v4l2_device_set_name` to set the name based on a driver
name and a driver-global atomic_t instance. This will generate names like
``ivtv0``, ``ivtv1``, etc. If the name ends with a digit, then it will insert
a dash: ``cx18-0``, ``cx18-1``, etc. This function returns the instance number.

The first ``dev`` argument is normally the ``struct device`` pointer of a
``pci_dev``, ``usb_interface`` or ``platform_device``. It is rare for dev to
be ``NULL``, but it happens with ISA devices or when one device creates
multiple PCI devices, thus making it impossible to associate
:c:type:`v4l2_dev <v4l2_device>` with a particular parent.

You can also supply a ``notify()`` callback that can be called by sub-devices
to notify you of events. Whether you need to set this depends on the
sub-device. Any notifications a sub-device supports must be defined in a header
in ``include/media/subdevice.h``.

V4L2 devices are unregistered by calling:

	:c:func:`v4l2_device_unregister`
	(:c:type:`v4l2_dev <v4l2_device>`).

If the dev->driver_data field points to :c:type:`v4l2_dev <v4l2_device>`,
it will be reset to ``NULL``. Unregistering will also automatically unregister
all subdevs from the device.

If you have a hotpluggable device (e.g. a USB device), then when a disconnect
happens the parent device becomes invalid. Since :c:type:`v4l2_device` has a
pointer to that parent device it has to be cleared as well to mark that the
parent is gone. To do this call:

	:c:func:`v4l2_device_disconnect`
	(:c:type:`v4l2_dev <v4l2_device>`).

This does *not* unregister the subdevs, so you still need to call the
:c:func:`v4l2_device_unregister` function for that. If your driver is not
hotpluggable, then there is no need to call :c:func:`v4l2_device_disconnect`.

Sometimes you need to iterate over all devices registered by a specific
driver. This is usually the case if multiple device drivers use the same
hardware. E.g. the ivtvfb driver is a framebuffer driver that uses the ivtv
hardware. The same is true for alsa drivers for example.

You can iterate over all registered devices as follows:

.. code-block:: c

	static int callback(struct device *dev, void *p)
	{
		struct v4l2_device *v4l2_dev = dev_get_drvdata(dev);

		/* test if this device was inited */
		if (v4l2_dev == NULL)
			return 0;
		...
		return 0;
	}

	int iterate(void *p)
	{
		struct device_driver *drv;
		int err;

		/* Find driver 'ivtv' on the PCI bus.
		pci_bus_type is a global. For USB buses use usb_bus_type. */
		drv = driver_find("ivtv", &pci_bus_type);
		/* iterate over all ivtv device instances */
		err = driver_for_each_device(drv, NULL, p, callback);
		put_driver(drv);
		return err;
	}

Sometimes you need to keep a running counter of the device instance. This is
commonly used to map a device instance to an index of a module option array.

The recommended approach is as follows:

.. code-block:: c

	static atomic_t drv_instance = ATOMIC_INIT(0);

	static int drv_probe(struct pci_dev *pdev, const struct pci_device_id *pci_id)
	{
		...
		state->instance = atomic_inc_return(&drv_instance) - 1;
	}

If you have multiple device nodes then it can be difficult to know when it is
safe to unregister :c:type:`v4l2_device` for hotpluggable devices. For this
purpose :c:type:`v4l2_device` has refcounting support. The refcount is
increased whenever :c:func:`video_register_device` is called and it is
decreased whenever that device node is released. When the refcount reaches
zero, then the :c:type:`v4l2_device` release() callback is called. You can
do your final cleanup there.

If other device nodes (e.g. ALSA) are created, then you can increase and
decrease the refcount manually as well by calling:

	:c:func:`v4l2_device_get`
	(:c:type:`v4l2_dev <v4l2_device>`).

or:

	:c:func:`v4l2_device_put`
	(:c:type:`v4l2_dev <v4l2_device>`).

Since the initial refcount is 1 you also need to call
:c:func:`v4l2_device_put` in the ``disconnect()`` callback (for USB devices)
or in the ``remove()`` callback (for e.g. PCI devices), otherwise the refcount
will never reach 0.

v4l2_device functions and data structures
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/v4l2-device.h


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 DV Timings functions
^^^^^^^^^^^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/v4l2-dv-timings.h


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 events
-----------

The V4L2 events provide a generic way to pass events to user space.
The driver must use :c:type:`v4l2_fh` to be able to support V4L2 events.

Events are subscribed per-filehandle. An event specification consists of a
``type`` and is optionally associated with an object identified through the
``id`` field. If unused, then the ``id`` is 0. So an event is uniquely
identified by the ``(type, id)`` tuple.

The :c:type:`v4l2_fh` struct has a list of subscribed events on its
``subscribed`` field.

When the user subscribes to an event, a :c:type:`v4l2_subscribed_event`
struct is added to :c:type:`v4l2_fh`\ ``.subscribed``, one for every
subscribed event.

Each :c:type:`v4l2_subscribed_event` struct ends with a
:c:type:`v4l2_kevent` ringbuffer, with the size given by the caller
of :c:func:`v4l2_event_subscribe`. This ringbuffer is used to store any events
raised by the driver.

So every ``(type, ID)`` event tuple will have its own
:c:type:`v4l2_kevent` ringbuffer. This guarantees that if a driver is
generating lots of events of one type in a short time, then that will
not overwrite events of another type.

But if you get more events of one type than the size of the
:c:type:`v4l2_kevent` ringbuffer, then the oldest event will be dropped
and the new one added.

The :c:type:`v4l2_kevent` struct links into the ``available``
list of the :c:type:`v4l2_fh` struct so :ref:`VIDIOC_DQEVENT` will
know which event to dequeue first.

Finally, if the event subscription is associated with a particular object
such as a V4L2 control, then that object needs to know about that as well
so that an event can be raised by that object. So the ``node`` field can
be used to link the :c:type:`v4l2_subscribed_event` struct into a list of
such objects.

So to summarize:

- struct v4l2_fh has two lists: one of the ``subscribed`` events,
  and one of the ``available`` events.

- struct v4l2_subscribed_event has a ringbuffer of raised
  (pending) events of that particular type.

- If struct v4l2_subscribed_event is associated with a specific
  object, then that object will have an internal list of
  struct v4l2_subscribed_event so it knows who subscribed an
  event to that object.

Furthermore, the internal struct v4l2_subscribed_event has
``merge()`` and ``replace()`` callbacks which drivers can set. These
callbacks are called when a new event is raised and there is no more room.

The ``replace()`` callback allows you to replace the payload of the old event
with that of the new event, merging any relevant data from the old payload
into the new payload that replaces it. It is called when this event type has
a ringbuffer with size is one, i.e. only one event can be stored in the
ringbuffer.

The ``merge()`` callback allows you to merge the oldest event payload into
that of the second-oldest event payload. It is called when
the ringbuffer has size is greater than one.

This way no status information is lost, just the intermediate steps leading
up to that state.

A good example of these ``replace``/``merge`` callbacks is in v4l2-event.c:
``ctrls_replace()`` and ``ctrls_merge()`` callbacks for the control event.

.. note::
	these callbacks can be called from interrupt context, so they must
	be fast.

In order to queue events to video device, drivers should call:

	:c:func:`v4l2_event_queue <v4l2_event_queue>`
	(:c:type:`vdev <video_device>`, :c:type:`ev <v4l2_event>`)

The driver's only responsibility is to fill in the type and the data fields.
The other fields will be filled in by V4L2.

Event subscription
~~~~~~~~~~~~~~~~~~

Subscribing to an event is via:

	:c:func:`v4l2_event_subscribe <v4l2_event_subscribe>`
	(:c:type:`fh <v4l2_fh>`, :c:type:`sub <v4l2_event_subscription>` ,
	elems, :c:type:`ops <v4l2_subscribed_event_ops>`)

This function is used to implement :c:type:`video_device`->
:c:type:`ioctl_ops <v4l2_ioctl_ops>`-> ``vidioc_subscribe_event``,
but the driver must check first if the driver is able to produce events
with specified event id, and then should call
:c:func:`v4l2_event_subscribe` to subscribe the event.

The elems argument is the size of the event queue for this event. If it is 0,
then the framework will fill in a default value (this depends on the event
type).

The ops argument allows the driver to specify a number of callbacks:

.. tabularcolumns:: |p{1.5cm}|p{16.0cm}|

======== ==============================================================
Callback Description
======== ==============================================================
add      called when a new listener gets added (subscribing to the same
	 event twice will only cause this callback to get called once)
del      called when a listener stops listening
replace  replace event 'old' with event 'new'.
merge    merge event 'old' into event 'new'.
======== ==============================================================

All 4 callbacks are optional, if you don't want to specify any callbacks
the ops argument itself maybe ``NULL``.

Unsubscribing an event
~~~~~~~~~~~~~~~~~~~~~~

Unsubscribing to an event is via:

	:c:func:`v4l2_event_unsubscribe <v4l2_event_unsubscribe>`
	(:c:type:`fh <v4l2_fh>`, :c:type:`sub <v4l2_event_subscription>`)

This function is used to implement :c:type:`video_device`->
:c:type:`ioctl_ops <v4l2_ioctl_ops>`-> ``vidioc_unsubscribe_event``.
A driver may call :c:func:`v4l2_event_unsubscribe` directly unless it
wants to be involved in unsubscription process.

The special type ``V4L2_EVENT_ALL`` may be used to unsubscribe all events. The
drivers may want to handle this in a special way.

Check if there's a pending event
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Checking if there's a pending event is via:

	:c:func:`v4l2_event_pending <v4l2_event_pending>`
	(:c:type:`fh <v4l2_fh>`)

This function returns the number of pending events. Useful when implementing
poll.

How events work
~~~~~~~~~~~~~~~

Events are delivered to user space through the poll system call. The driver
can use :c:type:`v4l2_fh`->wait (a wait_queue_head_t) as the argument for
``poll_wait()``.

There are standard and private events. New standard events must use the
smallest available event type. The drivers must allocate their events from
their own class starting from class base. Class base is
``V4L2_EVENT_PRIVATE_START`` + n * 1000 where n is the lowest available number.
The first event type in the class is reserved for future use, so the first
available event type is 'class base + 1'.

An example on how the V4L2 events may be used can be found in the OMAP
3 ISP driver (``drivers/media/platform/ti/omap3isp``).

A subdev can directly send an event to the :c:type:`v4l2_device` notify
function with ``V4L2_DEVICE_NOTIFY_EVENT``. This allows the bridge to map
the subdev that sends the event to the video node(s) associated with the
subdev that need to be informed about such an event.

V4L2 event functions and data structures
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/v4l2-event.h


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 File handlers
------------------

struct v4l2_fh provides a way to easily keep file handle specific
data that is used by the V4L2 framework.

.. attention::
	New drivers must use struct v4l2_fh
	since it is also used to implement priority handling
	(:ref:`VIDIOC_G_PRIORITY`).

The users of :c:type:`v4l2_fh` (in the V4L2 framework, not the driver) know
whether a driver uses :c:type:`v4l2_fh` as its ``file->private_data`` pointer
by testing the ``V4L2_FL_USES_V4L2_FH`` bit in :c:type:`video_device`->flags.
This bit is set whenever :c:func:`v4l2_fh_init` is called.

struct v4l2_fh is allocated as a part of the driver's own file handle
structure and ``file->private_data`` is set to it in the driver's ``open()``
function by the driver.

In many cases the struct v4l2_fh will be embedded in a larger
structure. In that case you should call:

#) :c:func:`v4l2_fh_init` and :c:func:`v4l2_fh_add` in ``open()``
#) :c:func:`v4l2_fh_del` and :c:func:`v4l2_fh_exit` in ``release()``

Drivers can extract their own file handle structure by using the container_of
macro.

Example:

.. code-block:: c

	struct my_fh {
		int blah;
		struct v4l2_fh fh;
	};

	...

	int my_open(struct file *file)
	{
		struct my_fh *my_fh;
		struct video_device *vfd;
		int ret;

		...

		my_fh = kzalloc(sizeof(*my_fh), GFP_KERNEL);

		...

		v4l2_fh_init(&my_fh->fh, vfd);

		...

		file->private_data = &my_fh->fh;
		v4l2_fh_add(&my_fh->fh);
		return 0;
	}

	int my_release(struct file *file)
	{
		struct v4l2_fh *fh = file->private_data;
		struct my_fh *my_fh = container_of(fh, struct my_fh, fh);

		...
		v4l2_fh_del(&my_fh->fh);
		v4l2_fh_exit(&my_fh->fh);
		kfree(my_fh);
		return 0;
	}

Below is a short description of the :c:type:`v4l2_fh` functions used:

:c:func:`v4l2_fh_init <v4l2_fh_init>`
(:c:type:`fh <v4l2_fh>`, :c:type:`vdev <video_device>`)

- Initialise the file handle. This **MUST** be performed in the driver's
  :c:type:`v4l2_file_operations`->open() handler.

:c:func:`v4l2_fh_add <v4l2_fh_add>`
(:c:type:`fh <v4l2_fh>`)

- Add a :c:type:`v4l2_fh` to :c:type:`video_device` file handle list.
  Must be called once the file handle is completely initialized.

:c:func:`v4l2_fh_del <v4l2_fh_del>`
(:c:type:`fh <v4l2_fh>`)

- Unassociate the file handle from :c:type:`video_device`. The file handle
  exit function may now be called.

:c:func:`v4l2_fh_exit <v4l2_fh_exit>`
(:c:type:`fh <v4l2_fh>`)

- Uninitialise the file handle. After uninitialisation the :c:type:`v4l2_fh`
  memory can be freed.

If struct v4l2_fh is not embedded, then you can use these helper functions:

:c:func:`v4l2_fh_open <v4l2_fh_open>`
(struct file \*filp)

- This allocates a struct v4l2_fh, initializes it and adds it to
  the struct video_device associated with the file struct.

:c:func:`v4l2_fh_release <v4l2_fh_release>`
(struct file \*filp)

- This deletes it from the struct video_device associated with the
  file struct, uninitialised the :c:type:`v4l2_fh` and frees it.

These two functions can be plugged into the v4l2_file_operation's ``open()``
and ``release()`` ops.

Several drivers need to do something when the first file handle is opened and
when the last file handle closes. Two helper functions were added to check
whether the :c:type:`v4l2_fh` struct is the only open filehandle of the
associated device node:

:c:func:`v4l2_fh_is_singular <v4l2_fh_is_singular>`
(:c:type:`fh <v4l2_fh>`)

-  Returns 1 if the file handle is the only open file handle, else 0.

:c:func:`v4l2_fh_is_singular_file <v4l2_fh_is_singular_file>`
(struct file \*filp)

- Same, but it calls v4l2_fh_is_singular with filp->private_data.

V4L2 fh functions and data structures
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/v4l2-fh.h


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 flash functions and data structures
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/v4l2-flash-led-class.h


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 fwnode kAPI
^^^^^^^^^^^^^^^^
.. kernel-doc:: include/media/v4l2-fwnode.h


code:
.. SPDX-License-Identifier: GPL-2.0

Introduction
------------

The V4L2 drivers tend to be very complex due to the complexity of the
hardware: most devices have multiple ICs, export multiple device nodes in
/dev, and create also non-V4L2 devices such as DVB, ALSA, FB, I2C and input
(IR) devices.

Especially the fact that V4L2 drivers have to setup supporting ICs to
do audio/video muxing/encoding/decoding makes it more complex than most.
Usually these ICs are connected to the main bridge driver through one or
more I2C buses, but other buses can also be used. Such devices are
called 'sub-devices'.

For a long time the framework was limited to the video_device struct for
creating V4L device nodes and video_buf for handling the video buffers
(note that this document does not discuss the video_buf framework).

This meant that all drivers had to do the setup of device instances and
connecting to sub-devices themselves. Some of this is quite complicated
to do right and many drivers never did do it correctly.

There is also a lot of common code that could never be refactored due to
the lack of a framework.

So this framework sets up the basic building blocks that all drivers
need and this same framework should make it much easier to refactor
common code into utility functions shared by all drivers.

A good example to look at as a reference is the v4l2-pci-skeleton.c
source that is available in samples/v4l/. It is a skeleton driver for
a PCI capture card, and demonstrates how to use the V4L2 driver
framework. It can be used as a template for real PCI video capture driver.

Structure of a V4L driver
-------------------------

All drivers have the following structure:

1) A struct for each device instance containing the device state.

2) A way of initializing and commanding sub-devices (if any).

3) Creating V4L2 device nodes (/dev/videoX, /dev/vbiX and /dev/radioX)
   and keeping track of device-node specific data.

4) Filehandle-specific structs containing per-filehandle data;

5) video buffer handling.

This is a rough schematic of how it all relates:

.. code-block:: none

    device instances
      |
      +-sub-device instances
      |
      \-V4L2 device nodes
	  |
	  \-filehandle instances

Structure of the V4L2 framework
-------------------------------

The framework closely resembles the driver structure: it has a v4l2_device
struct for the device instance data, a v4l2_subdev struct to refer to
sub-device instances, the video_device struct stores V4L2 device node data
and the v4l2_fh struct keeps track of filehandle instances.

The V4L2 framework also optionally integrates with the media framework. If a
driver sets the struct v4l2_device mdev field, sub-devices and video nodes
will automatically appear in the media framework as entities.


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 Media Controller functions and data structures
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/v4l2-mc.h


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 Media Bus functions and data structures
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/v4l2-mediabus.h


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 Memory to Memory functions and data structures
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/v4l2-mem2mem.h


code:
.. SPDX-License-Identifier: GPL-2.0

V4L2 rect helper functions
^^^^^^^^^^^^^^^^^^^^^^^^^^

.. kernel-doc:: include/media/v4l2-rect.h