code:
.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>

.. _driverapi_pm_devices:

==============================
Device Power Management Basics
==============================

:Copyright: |copy| 2010-2011 Rafael J. Wysocki <rjw@sisk.pl>, Novell Inc.
:Copyright: |copy| 2010 Alan Stern <stern@rowland.harvard.edu>
:Copyright: |copy| 2016 Intel Corporation

:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Most of the code in Linux is device drivers, so most of the Linux power
management (PM) code is also driver-specific.  Most drivers will do very
little; others, especially for platforms with small batteries (like cell
phones), will do a lot.

This writeup gives an overview of how drivers interact with system-wide
power management goals, emphasizing the models and interfaces that are
shared by everything that hooks up to the driver model core.  Read it as
background for the domain-specific work you'd do with any specific driver.

Two Models for Device Power Management
======================================

Drivers will use one or both of these models to put devices into low-power
states:

    System Sleep model:

	Drivers can enter low-power states as part of entering system-wide
	low-power states like "suspend" (also known as "suspend-to-RAM"), or
	(mostly for systems with disks) "hibernation" (also known as
	"suspend-to-disk").

	This is something that device, bus, and class drivers collaborate on
	by implementing various role-specific suspend and resume methods to
	cleanly power down hardware and software subsystems, then reactivate
	them without loss of data.

	Some drivers can manage hardware wakeup events, which make the system
	leave the low-power state.  This feature may be enabled or disabled
	using the relevant :file:`/sys/devices/.../power/wakeup` file (for
	Ethernet drivers the ioctl interface used by ethtool may also be used
	for this purpose); enabling it may cost some power usage, but let the
	whole system enter low-power states more often.

    Runtime Power Management model:

	Devices may also be put into low-power states while the system is
	running, independently of other power management activity in principle.
	However, devices are not generally independent of each other (for
	example, a parent device cannot be suspended unless all of its child
	devices have been suspended).  Moreover, depending on the bus type the
	device is on, it may be necessary to carry out some bus-specific
	operations on the device for this purpose.  Devices put into low power
	states at run time may require special handling during system-wide power
	transitions (suspend or hibernation).

	For these reasons not only the device driver itself, but also the
	appropriate subsystem (bus type, device type or device class) driver and
	the PM core are involved in runtime power management.  As in the system
	sleep power management case, they need to collaborate by implementing
	various role-specific suspend and resume methods, so that the hardware
	is cleanly powered down and reactivated without data or service loss.

There's not a lot to be said about those low-power states except that they are
very system-specific, and often device-specific.  Also, that if enough devices
have been put into low-power states (at runtime), the effect may be very similar
to entering some system-wide low-power state (system sleep) ... and that
synergies exist, so that several drivers using runtime PM might put the system
into a state where even deeper power saving options are available.

Most suspended devices will have quiesced all I/O: no more DMA or IRQs (except
for wakeup events), no more data read or written, and requests from upstream
drivers are no longer accepted.  A given bus or platform may have different
requirements though.

Examples of hardware wakeup events include an alarm from a real time clock,
network wake-on-LAN packets, keyboard or mouse activity, and media insertion
or removal (for PCMCIA, MMC/SD, USB, and so on).

Interfaces for Entering System Sleep States
===========================================

There are programming interfaces provided for subsystems (bus type, device type,
device class) and device drivers to allow them to participate in the power
management of devices they are concerned with.  These interfaces cover both
system sleep and runtime power management.

Device Power Management Operations
----------------------------------

Device power management operations, at the subsystem level as well as at the
device driver level, are implemented by defining and populating objects of type
struct dev_pm_ops defined in :file:`include/linux/pm.h`.  The roles of the
methods included in it will be explained in what follows.  For now, it should be
sufficient to remember that the last three methods are specific to runtime power
management while the remaining ones are used during system-wide power
transitions.

There also is a deprecated "old" or "legacy" interface for power management
operations available at least for some subsystems.  This approach does not use
struct dev_pm_ops objects and it is suitable only for implementing system
sleep power management methods in a limited way.  Therefore it is not described
in this document, so please refer directly to the source code for more
information about it.

Subsystem-Level Methods
-----------------------

The core methods to suspend and resume devices reside in
struct dev_pm_ops pointed to by the :c:member:`ops` member of
struct dev_pm_domain, or by the :c:member:`pm` member of struct bus_type,
struct device_type and struct class.  They are mostly of interest to the
people writing infrastructure for platforms and buses, like PCI or USB, or
device type and device class drivers.  They also are relevant to the writers of
device drivers whose subsystems (PM domains, device types, device classes and
bus types) don't provide all power management methods.

Bus drivers implement these methods as appropriate for the hardware and the
drivers using it; PCI works differently from USB, and so on.  Not many people
write subsystem-level drivers; most driver code is a "device driver" that builds
on top of bus-specific framework code.

For more information on these driver calls, see the description later;
they are called in phases for every device, respecting the parent-child
sequencing in the driver model tree.

:file:`/sys/devices/.../power/wakeup` files
-------------------------------------------

All device objects in the driver model contain fields that control the handling
of system wakeup events (hardware signals that can force the system out of a
sleep state).  These fields are initialized by bus or device driver code using
:c:func:`device_set_wakeup_capable()` and :c:func:`device_set_wakeup_enable()`,
defined in :file:`include/linux/pm_wakeup.h`.

The :c:member:`power.can_wakeup` flag just records whether the device (and its
driver) can physically support wakeup events.  The
:c:func:`device_set_wakeup_capable()` routine affects this flag.  The
:c:member:`power.wakeup` field is a pointer to an object of type
struct wakeup_source used for controlling whether or not the device should use
its system wakeup mechanism and for notifying the PM core of system wakeup
events signaled by the device.  This object is only present for wakeup-capable
devices (i.e. devices whose :c:member:`can_wakeup` flags are set) and is created
(or removed) by :c:func:`device_set_wakeup_capable()`.

Whether or not a device is capable of issuing wakeup events is a hardware
matter, and the kernel is responsible for keeping track of it.  By contrast,
whether or not a wakeup-capable device should issue wakeup events is a policy
decision, and it is managed by user space through a sysfs attribute: the
:file:`power/wakeup` file.  User space can write the "enabled" or "disabled"
strings to it to indicate whether or not, respectively, the device is supposed
to signal system wakeup.  This file is only present if the
:c:member:`power.wakeup` object exists for the given device and is created (or
removed) along with that object, by :c:func:`device_set_wakeup_capable()`.
Reads from the file will return the corresponding string.

The initial value in the :file:`power/wakeup` file is "disabled" for the
majority of devices; the major exceptions are power buttons, keyboards, and
Ethernet adapters whose WoL (wake-on-LAN) feature has been set up with ethtool.
It should also default to "enabled" for devices that don't generate wakeup
requests on their own but merely forward wakeup requests from one bus to another
(like PCI Express ports).

The :c:func:`device_may_wakeup()` routine returns true only if the
:c:member:`power.wakeup` object exists and the corresponding :file:`power/wakeup`
file contains the "enabled" string.  This information is used by subsystems,
like the PCI bus type code, to see whether or not to enable the devices' wakeup
mechanisms.  If device wakeup mechanisms are enabled or disabled directly by
drivers, they also should use :c:func:`device_may_wakeup()` to decide what to do
during a system sleep transition.  Device drivers, however, are not expected to
call :c:func:`device_set_wakeup_enable()` directly in any case.

It ought to be noted that system wakeup is conceptually different from "remote
wakeup" used by runtime power management, although it may be supported by the
same physical mechanism.  Remote wakeup is a feature allowing devices in
low-power states to trigger specific interrupts to signal conditions in which
they should be put into the full-power state.  Those interrupts may or may not
be used to signal system wakeup events, depending on the hardware design.  On
some systems it is impossible to trigger them from system sleep states.  In any
case, remote wakeup should always be enabled for runtime power management for
all devices and drivers that support it.

:file:`/sys/devices/.../power/control` files
--------------------------------------------

Each device in the driver model has a flag to control whether it is subject to
runtime power management.  This flag, :c:member:`runtime_auto`, is initialized
by the bus type (or generally subsystem) code using :c:func:`pm_runtime_allow()`
or :c:func:`pm_runtime_forbid()`; the default is to allow runtime power
management.

The setting can be adjusted by user space by writing either "on" or "auto" to
the device's :file:`power/control` sysfs file.  Writing "auto" calls
:c:func:`pm_runtime_allow()`, setting the flag and allowing the device to be
runtime power-managed by its driver.  Writing "on" calls
:c:func:`pm_runtime_forbid()`, clearing the flag, returning the device to full
power if it was in a low-power state, and preventing the
device from being runtime power-managed.  User space can check the current value
of the :c:member:`runtime_auto` flag by reading that file.

The device's :c:member:`runtime_auto` flag has no effect on the handling of
system-wide power transitions.  In particular, the device can (and in the
majority of cases should and will) be put into a low-power state during a
system-wide transition to a sleep state even though its :c:member:`runtime_auto`
flag is clear.

For more information about the runtime power management framework, refer to
Documentation/power/runtime_pm.rst.

Calling Drivers to Enter and Leave System Sleep States
======================================================

When the system goes into a sleep state, each device's driver is asked to
suspend the device by putting it into a state compatible with the target
system state.  That's usually some version of "off", but the details are
system-specific.  Also, wakeup-enabled devices will usually stay partly
functional in order to wake the system.

When the system leaves that low-power state, the device's driver is asked to
resume it by returning it to full power.  The suspend and resume operations
always go together, and both are multi-phase operations.

For simple drivers, suspend might quiesce the device using class code
and then turn its hardware as "off" as possible during suspend_noirq.  The
matching resume calls would then completely reinitialize the hardware
before reactivating its class I/O queues.

More power-aware drivers might prepare the devices for triggering system wakeup
events.

Call Sequence Guarantees
------------------------

To ensure that bridges and similar links needing to talk to a device are
available when the device is suspended or resumed, the device hierarchy is
walked in a bottom-up order to suspend devices.  A top-down order is
used to resume those devices.

The ordering of the device hierarchy is defined by the order in which devices
get registered:  a child can never be registered, probed or resumed before
its parent; and can't be removed or suspended after that parent.

The policy is that the device hierarchy should match hardware bus topology.
[Or at least the control bus, for devices which use multiple busses.]
In particular, this means that a device registration may fail if the parent of
the device is suspending (i.e. has been chosen by the PM core as the next
device to suspend) or has already suspended, as well as after all of the other
devices have been suspended.  Device drivers must be prepared to cope with such
situations.

System Power Management Phases
------------------------------

Suspending or resuming the system is done in several phases.  Different phases
are used for suspend-to-idle, shallow (standby), and deep ("suspend-to-RAM")
sleep states and the hibernation state ("suspend-to-disk").  Each phase involves
executing callbacks for every device before the next phase begins.  Not all
buses or classes support all these callbacks and not all drivers use all the
callbacks.  The various phases always run after tasks have been frozen and
before they are unfrozen.  Furthermore, the ``*_noirq`` phases run at a time
when IRQ handlers have been disabled (except for those marked with the
IRQF_NO_SUSPEND flag).

All phases use PM domain, bus, type, class or driver callbacks (that is, methods
defined in ``dev->pm_domain->ops``, ``dev->bus->pm``, ``dev->type->pm``,
``dev->class->pm`` or ``dev->driver->pm``).  These callbacks are regarded by the
PM core as mutually exclusive.  Moreover, PM domain callbacks always take
precedence over all of the other callbacks and, for example, type callbacks take
precedence over bus, class and driver callbacks.  To be precise, the following
rules are used to determine which callback to execute in the given phase:

    1.	If ``dev->pm_domain`` is present, the PM core will choose the callback
	provided by ``dev->pm_domain->ops`` for execution.

    2.	Otherwise, if both ``dev->type`` and ``dev->type->pm`` are present, the
	callback provided by ``dev->type->pm`` will be chosen for execution.

    3.	Otherwise, if both ``dev->class`` and ``dev->class->pm`` are present,
	the callback provided by ``dev->class->pm`` will be chosen for
	execution.

    4.	Otherwise, if both ``dev->bus`` and ``dev->bus->pm`` are present, the
	callback provided by ``dev->bus->pm`` will be chosen for execution.

This allows PM domains and device types to override callbacks provided by bus
types or device classes if necessary.

The PM domain, type, class and bus callbacks may in turn invoke device- or
driver-specific methods stored in ``dev->driver->pm``, but they don't have to do
that.

If the subsystem callback chosen for execution is not present, the PM core will
execute the corresponding method from the ``dev->driver->pm`` set instead if
there is one.

Entering System Suspend
-----------------------

When the system goes into the freeze, standby or memory sleep state,
the phases are: ``prepare``, ``suspend``, ``suspend_late``, ``suspend_noirq``.

    1.	The ``prepare`` phase is meant to prevent races by preventing new
	devices from being registered; the PM core would never know that all the
	children of a device had been suspended if new children could be
	registered at will.  [By contrast, from the PM core's perspective,
	devices may be unregistered at any time.]  Unlike the other
	suspend-related phases, during the ``prepare`` phase the device
	hierarchy is traversed top-down.

	After the ``->prepare`` callback method returns, no new children may be
	registered below the device.  The method may also prepare the device or
	driver in some way for the upcoming system power transition, but it
	should not put the device into a low-power state.  Moreover, if the
	device supports runtime power management, the ``->prepare`` callback
	method must not update its state in case it is necessary to resume it
	from runtime suspend later on.

	For devices supporting runtime power management, the return value of the
	prepare callback can be used to indicate to the PM core that it may
	safely leave the device in runtime suspend (if runtime-suspended
	already), provided that all of the device's descendants are also left in
	runtime suspend.  Namely, if the prepare callback returns a positive
	number and that happens for all of the descendants of the device too,
	and all of them (including the device itself) are runtime-suspended, the
	PM core will skip the ``suspend``, ``suspend_late`` and
	``suspend_noirq`` phases as well as all of the corresponding phases of
	the subsequent device resume for all of these devices.	In that case,
	the ``->complete`` callback will be the next one invoked after the
	``->prepare`` callback and is entirely responsible for putting the
	device into a consistent state as appropriate.

	Note that this direct-complete procedure applies even if the device is
	disabled for runtime PM; only the runtime-PM status matters.  It follows
	that if a device has system-sleep callbacks but does not support runtime
	PM, then its prepare callback must never return a positive value.  This
	is because all such devices are initially set to runtime-suspended with
	runtime PM disabled.

	This feature also can be controlled by device drivers by using the
	``DPM_FLAG_NO_DIRECT_COMPLETE`` and ``DPM_FLAG_SMART_PREPARE`` driver
	power management flags.  [Typically, they are set at the time the driver
	is probed against the device in question by passing them to the
	:c:func:`dev_pm_set_driver_flags` helper function.]  If the first of
	these flags is set, the PM core will not apply the direct-complete
	procedure described above to the given device and, consequenty, to any
	of its ancestors.  The second flag, when set, informs the middle layer
	code (bus types, device types, PM domains, classes) that it should take
	the return value of the ``->prepare`` callback provided by the driver
	into account and it may only return a positive value from its own
	``->prepare`` callback if the driver's one also has returned a positive
	value.

    2.	The ``->suspend`` methods should quiesce the device to stop it from
	performing I/O.  They also may save the device registers and put it into
	the appropriate low-power state, depending on the bus type the device is
	on, and they may enable wakeup events.

	However, for devices supporting runtime power management, the
	``->suspend`` methods provided by subsystems (bus types and PM domains
	in particular) must follow an additional rule regarding what can be done
	to the devices before their drivers' ``->suspend`` methods are called.
	Namely, they may resume the devices from runtime suspend by
	calling :c:func:`pm_runtime_resume` for them, if that is necessary, but
	they must not update the state of the devices in any other way at that
	time (in case the drivers need to resume the devices from runtime
	suspend in their ``->suspend`` methods).  In fact, the PM core prevents
	subsystems or drivers from putting devices into runtime suspend at
	these times by calling :c:func:`pm_runtime_get_noresume` before issuing
	the ``->prepare`` callback (and calling :c:func:`pm_runtime_put` after
	issuing the ``->complete`` callback).

    3.	For a number of devices it is convenient to split suspend into the
	"quiesce device" and "save device state" phases, in which cases
	``suspend_late`` is meant to do the latter.  It is always executed after
	runtime power management has been disabled for the device in question.

    4.	The ``suspend_noirq`` phase occurs after IRQ handlers have been disabled,
	which means that the driver's interrupt handler will not be called while
	the callback method is running.  The ``->suspend_noirq`` methods should
	save the values of the device's registers that weren't saved previously
	and finally put the device into the appropriate low-power state.

	The majority of subsystems and device drivers need not implement this
	callback.  However, bus types allowing devices to share interrupt
	vectors, like PCI, generally need it; otherwise a driver might encounter
	an error during the suspend phase by fielding a shared interrupt
	generated by some other device after its own device had been set to low
	power.

At the end of these phases, drivers should have stopped all I/O transactions
(DMA, IRQs), saved enough state that they can re-initialize or restore previous
state (as needed by the hardware), and placed the device into a low-power state.
On many platforms they will gate off one or more clock sources; sometimes they
will also switch off power supplies or reduce voltages.  [Drivers supporting
runtime PM may already have performed some or all of these steps.]

If :c:func:`device_may_wakeup()` returns ``true``, the device should be
prepared for generating hardware wakeup signals to trigger a system wakeup event
when the system is in the sleep state.  For example, :c:func:`enable_irq_wake()`
might identify GPIO signals hooked up to a switch or other external hardware,
and :c:func:`pci_enable_wake()` does something similar for the PCI PME signal.

If any of these callbacks returns an error, the system won't enter the desired
low-power state.  Instead, the PM core will unwind its actions by resuming all
the devices that were suspended.

Leaving System Suspend
----------------------

When resuming from freeze, standby or memory sleep, the phases are:
``resume_noirq``, ``resume_early``, ``resume``, ``complete``.

    1.	The ``->resume_noirq`` callback methods should perform any actions
	needed before the driver's interrupt handlers are invoked.  This
	generally means undoing the actions of the ``suspend_noirq`` phase.  If
	the bus type permits devices to share interrupt vectors, like PCI, the
	method should bring the device and its driver into a state in which the
	driver can recognize if the device is the source of incoming interrupts,
	if any, and handle them correctly.

	For example, the PCI bus type's ``->pm.resume_noirq()`` puts the device
	into the full-power state (D0 in the PCI terminology) and restores the
	standard configuration registers of the device.  Then it calls the
	device driver's ``->pm.resume_noirq()`` method to perform device-specific
	actions.

    2.	The ``->resume_early`` methods should prepare devices for the execution
	of the resume methods.  This generally involves undoing the actions of
	the preceding ``suspend_late`` phase.

    3.	The ``->resume`` methods should bring the device back to its operating
	state, so that it can perform normal I/O.  This generally involves
	undoing the actions of the ``suspend`` phase.

    4.	The ``complete`` phase should undo the actions of the ``prepare`` phase.
        For this reason, unlike the other resume-related phases, during the
        ``complete`` phase the device hierarchy is traversed bottom-up.

	Note, however, that new children may be registered below the device as
	soon as the ``->resume`` callbacks occur; it's not necessary to wait
	until the ``complete`` phase runs.

	Moreover, if the preceding ``->prepare`` callback returned a positive
	number, the device may have been left in runtime suspend throughout the
	whole system suspend and resume (its ``->suspend``, ``->suspend_late``,
	``->suspend_noirq``, ``->resume_noirq``,
	``->resume_early``, and ``->resume`` callbacks may have been
	skipped).  In that case, the ``->complete`` callback is entirely
	responsible for putting the device into a consistent state after system
	suspend if necessary.  [For example, it may need to queue up a runtime
	resume request for the device for this purpose.]  To check if that is
	the case, the ``->complete`` callback can consult the device's
	``power.direct_complete`` flag.  If that flag is set when the
	``->complete`` callback is being run then the direct-complete mechanism
	was used, and special actions may be required to make the device work
	correctly afterward.

At the end of these phases, drivers should be as functional as they were before
suspending: I/O can be performed using DMA and IRQs, and the relevant clocks are
gated on.

However, the details here may again be platform-specific.  For example,
some systems support multiple "run" states, and the mode in effect at
the end of resume might not be the one which preceded suspension.
That means availability of certain clocks or power supplies changed,
which could easily affect how a driver works.

Drivers need to be able to handle hardware which has been reset since all of the
suspend methods were called, for example by complete reinitialization.
This may be the hardest part, and the one most protected by NDA'd documents
and chip errata.  It's simplest if the hardware state hasn't changed since
the suspend was carried out, but that can only be guaranteed if the target
system sleep entered was suspend-to-idle.  For the other system sleep states
that may not be the case (and usually isn't for ACPI-defined system sleep
states, like S3).

Drivers must also be prepared to notice that the device has been removed
while the system was powered down, whenever that's physically possible.
PCMCIA, MMC, USB, Firewire, SCSI, and even IDE are common examples of busses
where common Linux platforms will see such removal.  Details of how drivers
will notice and handle such removals are currently bus-specific, and often
involve a separate thread.

These callbacks may return an error value, but the PM core will ignore such
errors since there's nothing it can do about them other than printing them in
the system log.

Entering Hibernation
--------------------

Hibernating the system is more complicated than putting it into sleep states,
because it involves creating and saving a system image.  Therefore there are
more phases for hibernation, with a different set of callbacks.  These phases
always run after tasks have been frozen and enough memory has been freed.

The general procedure for hibernation is to quiesce all devices ("freeze"),
create an image of the system memory while everything is stable, reactivate all
devices ("thaw"), write the image to permanent storage, and finally shut down
the system ("power off").  The phases used to accomplish this are: ``prepare``,
``freeze``, ``freeze_late``, ``freeze_noirq``, ``thaw_noirq``, ``thaw_early``,
``thaw``, ``complete``, ``prepare``, ``poweroff``, ``poweroff_late``,
``poweroff_noirq``.

    1.	The ``prepare`` phase is discussed in the "Entering System Suspend"
	section above.

    2.	The ``->freeze`` methods should quiesce the device so that it doesn't
	generate IRQs or DMA, and they may need to save the values of device
	registers.  However the device does not have to be put in a low-power
	state, and to save time it's best not to do so.  Also, the device should
	not be prepared to generate wakeup events.

    3.	The ``freeze_late`` phase is analogous to the ``suspend_late`` phase
	described earlier, except that the device should not be put into a
	low-power state and should not be allowed to generate wakeup events.

    4.	The ``freeze_noirq`` phase is analogous to the ``suspend_noirq`` phase
	discussed earlier, except again that the device should not be put into
	a low-power state and should not be allowed to generate wakeup events.

At this point the system image is created.  All devices should be inactive and
the contents of memory should remain undisturbed while this happens, so that the
image forms an atomic snapshot of the system state.

    5.	The ``thaw_noirq`` phase is analogous to the ``resume_noirq`` phase
	discussed earlier.  The main difference is that its methods can assume
	the device is in the same state as at the end of the ``freeze_noirq``
	phase.

    6.	The ``thaw_early`` phase is analogous to the ``resume_early`` phase
	described above.  Its methods should undo the actions of the preceding
	``freeze_late``, if necessary.

    7.	The ``thaw`` phase is analogous to the ``resume`` phase discussed
	earlier.  Its methods should bring the device back to an operating
	state, so that it can be used for saving the image if necessary.

    8.	The ``complete`` phase is discussed in the "Leaving System Suspend"
	section above.

At this point the system image is saved, and the devices then need to be
prepared for the upcoming system shutdown.  This is much like suspending them
before putting the system into the suspend-to-idle, shallow or deep sleep state,
and the phases are similar.

    9.	The ``prepare`` phase is discussed above.

    10.	The ``poweroff`` phase is analogous to the ``suspend`` phase.

    11.	The ``poweroff_late`` phase is analogous to the ``suspend_late`` phase.

    12.	The ``poweroff_noirq`` phase is analogous to the ``suspend_noirq`` phase.

The ``->poweroff``, ``->poweroff_late`` and ``->poweroff_noirq`` callbacks
should do essentially the same things as the ``->suspend``, ``->suspend_late``
and ``->suspend_noirq`` callbacks, respectively.  A notable difference is
that they need not store the device register values, because the registers
should already have been stored during the ``freeze``, ``freeze_late`` or
``freeze_noirq`` phases.  Also, on many machines the firmware will power-down
the entire system, so it is not necessary for the callback to put the device in
a low-power state.

Leaving Hibernation
-------------------

Resuming from hibernation is, again, more complicated than resuming from a sleep
state in which the contents of main memory are preserved, because it requires
a system image to be loaded into memory and the pre-hibernation memory contents
to be restored before control can be passed back to the image kernel.

Although in principle the image might be loaded into memory and the
pre-hibernation memory contents restored by the boot loader, in practice this
can't be done because boot loaders aren't smart enough and there is no
established protocol for passing the necessary information.  So instead, the
boot loader loads a fresh instance of the kernel, called "the restore kernel",
into memory and passes control to it in the usual way.  Then the restore kernel
reads the system image, restores the pre-hibernation memory contents, and passes
control to the image kernel.  Thus two different kernel instances are involved
in resuming from hibernation.  In fact, the restore kernel may be completely
different from the image kernel: a different configuration and even a different
version.  This has important consequences for device drivers and their
subsystems.

To be able to load the system image into memory, the restore kernel needs to
include at least a subset of device drivers allowing it to access the storage
medium containing the image, although it doesn't need to include all of the
drivers present in the image kernel.  After the image has been loaded, the
devices managed by the boot kernel need to be prepared for passing control back
to the image kernel.  This is very similar to the initial steps involved in
creating a system image, and it is accomplished in the same way, using
``prepare``, ``freeze``, and ``freeze_noirq`` phases.  However, the devices
affected by these phases are only those having drivers in the restore kernel;
other devices will still be in whatever state the boot loader left them.

Should the restoration of the pre-hibernation memory contents fail, the restore
kernel would go through the "thawing" procedure described above, using the
``thaw_noirq``, ``thaw_early``, ``thaw``, and ``complete`` phases, and then
continue running normally.  This happens only rarely.  Most often the
pre-hibernation memory contents are restored successfully and control is passed
to the image kernel, which then becomes responsible for bringing the system back
to the working state.

To achieve this, the image kernel must restore the devices' pre-hibernation
functionality.  The operation is much like waking up from a sleep state (with
the memory contents preserved), although it involves different phases:
``restore_noirq``, ``restore_early``, ``restore``, ``complete``.

    1.	The ``restore_noirq`` phase is analogous to the ``resume_noirq`` phase.

    2.	The ``restore_early`` phase is analogous to the ``resume_early`` phase.

    3.	The ``restore`` phase is analogous to the ``resume`` phase.

    4.	The ``complete`` phase is discussed above.

The main difference from ``resume[_early|_noirq]`` is that
``restore[_early|_noirq]`` must assume the device has been accessed and
reconfigured by the boot loader or the restore kernel.  Consequently, the state
of the device may be different from the state remembered from the ``freeze``,
``freeze_late`` and ``freeze_noirq`` phases.  The device may even need to be
reset and completely re-initialized.  In many cases this difference doesn't
matter, so the ``->resume[_early|_noirq]`` and ``->restore[_early|_norq]``
method pointers can be set to the same routines.  Nevertheless, different
callback pointers are used in case there is a situation where it actually does
matter.

Power Management Notifiers
==========================

There are some operations that cannot be carried out by the power management
callbacks discussed above, because the callbacks occur too late or too early.
To handle these cases, subsystems and device drivers may register power
management notifiers that are called before tasks are frozen and after they have
been thawed.  Generally speaking, the PM notifiers are suitable for performing
actions that either require user space to be available, or at least won't
interfere with user space.

For details refer to Documentation/driver-api/pm/notifiers.rst.

Device Low-Power (suspend) States
=================================

Device low-power states aren't standard.  One device might only handle
"on" and "off", while another might support a dozen different versions of
"on" (how many engines are active?), plus a state that gets back to "on"
faster than from a full "off".

Some buses define rules about what different suspend states mean.  PCI
gives one example: after the suspend sequence completes, a non-legacy
PCI device may not perform DMA or issue IRQs, and any wakeup events it
issues would be issued through the PME# bus signal.  Plus, there are
several PCI-standard device states, some of which are optional.

In contrast, integrated system-on-chip processors often use IRQs as the
wakeup event sources (so drivers would call :c:func:`enable_irq_wake`) and
might be able to treat DMA completion as a wakeup event (sometimes DMA can stay
active too, it'd only be the CPU and some peripherals that sleep).

Some details here may be platform-specific.  Systems may have devices that
can be fully active in certain sleep states, such as an LCD display that's
refreshed using DMA while most of the system is sleeping lightly ... and
its frame buffer might even be updated by a DSP or other non-Linux CPU while
the Linux control processor stays idle.

Moreover, the specific actions taken may depend on the target system state.
One target system state might allow a given device to be very operational;
another might require a hard shut down with re-initialization on resume.
And two different target systems might use the same device in different
ways; the aforementioned LCD might be active in one product's "standby",
but a different product using the same SOC might work differently.

Device Power Management Domains
===============================

Sometimes devices share reference clocks or other power resources.  In those
cases it generally is not possible to put devices into low-power states
individually.  Instead, a set of devices sharing a power resource can be put
into a low-power state together at the same time by turning off the shared
power resource.  Of course, they also need to be put into the full-power state
together, by turning the shared power resource on.  A set of devices with this
property is often referred to as a power domain. A power domain may also be
nested inside another power domain. The nested domain is referred to as the
sub-domain of the parent domain.

Support for power domains is provided through the :c:member:`pm_domain` field of
struct device.  This field is a pointer to an object of type
struct dev_pm_domain, defined in :file:`include/linux/pm.h`, providing a set
of power management callbacks analogous to the subsystem-level and device driver
callbacks that are executed for the given device during all power transitions,
instead of the respective subsystem-level callbacks.  Specifically, if a
device's :c:member:`pm_domain` pointer is not NULL, the ``->suspend()`` callback
from the object pointed to by it will be executed instead of its subsystem's
(e.g. bus type's) ``->suspend()`` callback and analogously for all of the
remaining callbacks.  In other words, power management domain callbacks, if
defined for the given device, always take precedence over the callbacks provided
by the device's subsystem (e.g. bus type).

The support for device power management domains is only relevant to platforms
needing to use the same device driver power management callbacks in many
different power domain configurations and wanting to avoid incorporating the
support for power domains into subsystem-level callbacks, for example by
modifying the platform bus type.  Other platforms need not implement it or take
it into account in any way.

Devices may be defined as IRQ-safe which indicates to the PM core that their
runtime PM callbacks may be invoked with disabled interrupts (see
Documentation/power/runtime_pm.rst for more information).  If an
IRQ-safe device belongs to a PM domain, the runtime PM of the domain will be
disallowed, unless the domain itself is defined as IRQ-safe. However, it
makes sense to define a PM domain as IRQ-safe only if all the devices in it
are IRQ-safe. Moreover, if an IRQ-safe domain has a parent domain, the runtime
PM of the parent is only allowed if the parent itself is IRQ-safe too with the
additional restriction that all child domains of an IRQ-safe parent must also
be IRQ-safe.

Runtime Power Management
========================

Many devices are able to dynamically power down while the system is still
running. This feature is useful for devices that are not being used, and
can offer significant power savings on a running system.  These devices
often support a range of runtime power states, which might use names such
as "off", "sleep", "idle", "active", and so on.  Those states will in some
cases (like PCI) be partially constrained by the bus the device uses, and will
usually include hardware states that are also used in system sleep states.

A system-wide power transition can be started while some devices are in low
power states due to runtime power management.  The system sleep PM callbacks
should recognize such situations and react to them appropriately, but the
necessary actions are subsystem-specific.

In some cases the decision may be made at the subsystem level while in other
cases the device driver may be left to decide.  In some cases it may be
desirable to leave a suspended device in that state during a system-wide power
transition, but in other cases the device must be put back into the full-power
state temporarily, for example so that its system wakeup capability can be
disabled.  This all depends on the hardware and the design of the subsystem and
device driver in question.

If it is necessary to resume a device from runtime suspend during a system-wide
transition into a sleep state, that can be done by calling
:c:func:`pm_runtime_resume` from the ``->suspend`` callback (or the ``->freeze``
or ``->poweroff`` callback for transitions related to hibernation) of either the
device's driver or its subsystem (for example, a bus type or a PM domain).
However, subsystems must not otherwise change the runtime status of devices
from their ``->prepare`` and ``->suspend`` callbacks (or equivalent) *before*
invoking device drivers' ``->suspend`` callbacks (or equivalent).

.. _smart_suspend_flag:

The ``DPM_FLAG_SMART_SUSPEND`` Driver Flag
------------------------------------------

Some bus types and PM domains have a policy to resume all devices from runtime
suspend upfront in their ``->suspend`` callbacks, but that may not be really
necessary if the device's driver can cope with runtime-suspended devices.
The driver can indicate this by setting ``DPM_FLAG_SMART_SUSPEND`` in
:c:member:`power.driver_flags` at probe time, with the assistance of the
:c:func:`dev_pm_set_driver_flags` helper routine.

Setting that flag causes the PM core and middle-layer code
(bus types, PM domains etc.) to skip the ``->suspend_late`` and
``->suspend_noirq`` callbacks provided by the driver if the device remains in
runtime suspend throughout those phases of the system-wide suspend (and
similarly for the "freeze" and "poweroff" parts of system hibernation).
[Otherwise the same driver
callback might be executed twice in a row for the same device, which would not
be valid in general.]  If the middle-layer system-wide PM callbacks are present
for the device then they are responsible for skipping these driver callbacks;
if not then the PM core skips them.  The subsystem callback routines can
determine whether they need to skip the driver callbacks by testing the return
value from the :c:func:`dev_pm_skip_suspend` helper function.

In addition, with ``DPM_FLAG_SMART_SUSPEND`` set, the driver's ``->thaw_noirq``
and ``->thaw_early`` callbacks are skipped in hibernation if the device remained
in runtime suspend throughout the preceding "freeze" transition.  Again, if the
middle-layer callbacks are present for the device, they are responsible for
doing this, otherwise the PM core takes care of it.

The ``DPM_FLAG_MAY_SKIP_RESUME`` Driver Flag
--------------------------------------------

During system-wide resume from a sleep state it's easiest to put devices into
the full-power state, as explained in Documentation/power/runtime_pm.rst.
[Refer to that document for more information regarding this particular issue as
well as for information on the device runtime power management framework in
general.]  However, it often is desirable to leave devices in suspend after
system transitions to the working state, especially if those devices had been in
runtime suspend before the preceding system-wide suspend (or analogous)
transition.

To that end, device drivers can use the ``DPM_FLAG_MAY_SKIP_RESUME`` flag to
indicate to the PM core and middle-layer code that they allow their "noirq" and
"early" resume callbacks to be skipped if the device can be left in suspend
after system-wide PM transitions to the working state.  Whether or not that is
the case generally depends on the state of the device before the given system
suspend-resume cycle and on the type of the system transition under way.
In particular, the "thaw" and "restore" transitions related to hibernation are
not affected by ``DPM_FLAG_MAY_SKIP_RESUME`` at all.  [All callbacks are
issued during the "restore" transition regardless of the flag settings,
and whether or not any driver callbacks
are skipped during the "thaw" transition depends whether or not the
``DPM_FLAG_SMART_SUSPEND`` flag is set (see `above <smart_suspend_flag_>`_).
In addition, a device is not allowed to remain in runtime suspend if any of its
children will be returned to full power.]

The ``DPM_FLAG_MAY_SKIP_RESUME`` flag is taken into account in combination with
the :c:member:`power.may_skip_resume` status bit set by the PM core during the
"suspend" phase of suspend-type transitions.  If the driver or the middle layer
has a reason to prevent the driver's "noirq" and "early" resume callbacks from
being skipped during the subsequent system resume transition, it should
clear :c:member:`power.may_skip_resume` in its ``->suspend``, ``->suspend_late``
or ``->suspend_noirq`` callback.  [Note that the drivers setting
``DPM_FLAG_SMART_SUSPEND`` need to clear :c:member:`power.may_skip_resume` in
their ``->suspend`` callback in case the other two are skipped.]

Setting the :c:member:`power.may_skip_resume` status bit along with the
``DPM_FLAG_MAY_SKIP_RESUME`` flag is necessary, but generally not sufficient,
for the driver's "noirq" and "early" resume callbacks to be skipped.  Whether or
not they should be skipped can be determined by evaluating the
:c:func:`dev_pm_skip_resume` helper function.

If that function returns ``true``, the driver's "noirq" and "early" resume
callbacks should be skipped and the device's runtime PM status will be set to
"suspended" by the PM core.  Otherwise, if the device was runtime-suspended
during the preceding system-wide suspend transition and its
``DPM_FLAG_SMART_SUSPEND`` is set, its runtime PM status will be set to
"active" by the PM core.  [Hence, the drivers that do not set
``DPM_FLAG_SMART_SUSPEND`` should not expect the runtime PM status of their
devices to be changed from "suspended" to "active" by the PM core during
system-wide resume-type transitions.]

If the ``DPM_FLAG_MAY_SKIP_RESUME`` flag is not set for a device, but
``DPM_FLAG_SMART_SUSPEND`` is set and the driver's "late" and "noirq" suspend
callbacks are skipped, its system-wide "noirq" and "early" resume callbacks, if
present, are invoked as usual and the device's runtime PM status is set to
"active" by the PM core before enabling runtime PM for it.  In that case, the
driver must be prepared to cope with the invocation of its system-wide resume
callbacks back-to-back with its ``->runtime_suspend`` one (without the
intervening ``->runtime_resume`` and system-wide suspend callbacks) and the
final state of the device must reflect the "active" runtime PM status in that
case.  [Note that this is not a problem at all if the driver's
``->suspend_late`` callback pointer points to the same function as its
``->runtime_suspend`` one and its ``->resume_early`` callback pointer points to
the same function as the ``->runtime_resume`` one, while none of the other
system-wide suspend-resume callbacks of the driver are present, for example.]

Likewise, if ``DPM_FLAG_MAY_SKIP_RESUME`` is set for a device, its driver's
system-wide "noirq" and "early" resume callbacks may be skipped while its "late"
and "noirq" suspend callbacks may have been executed (in principle, regardless
of whether or not ``DPM_FLAG_SMART_SUSPEND`` is set).  In that case, the driver
needs to be able to cope with the invocation of its ``->runtime_resume``
callback back-to-back with its "late" and "noirq" suspend ones.  [For instance,
that is not a concern if the driver sets both ``DPM_FLAG_SMART_SUSPEND`` and
``DPM_FLAG_MAY_SKIP_RESUME`` and uses the same pair of suspend/resume callback
functions for runtime PM and system-wide suspend/resume.]


code:
.. SPDX-License-Identifier: GPL-2.0

===============================
CPU and Device Power Management
===============================

.. toctree::

   cpuidle
   devices
   notifiers
   types

.. only::  subproject and html

   Indices
   =======

   * :ref:`genindex`

repo_name: angery/linux


code:
.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>

=============================
Suspend/Hibernation Notifiers
=============================

:Copyright: |copy| 2016 Intel Corporation

:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

There are some operations that subsystems or drivers may want to carry out
before hibernation/suspend or after restore/resume, but they require the system
to be fully functional, so the drivers' and subsystems' ``->suspend()`` and
``->resume()`` or even ``->prepare()`` and ``->complete()`` callbacks are not
suitable for this purpose.

For example, device drivers may want to upload firmware to their devices after
resume/restore, but they cannot do it by calling :c:func:`request_firmware()`
from their ``->resume()`` or ``->complete()`` callback routines (user land
processes are frozen at these points).  The solution may be to load the firmware
into memory before processes are frozen and upload it from there in the
``->resume()`` routine.  A suspend/hibernation notifier may be used for that.

Subsystems or drivers having such needs can register suspend notifiers that
will be called upon the following events by the PM core:

``PM_HIBERNATION_PREPARE``
	The system is going to hibernate, tasks will be frozen immediately. This
	is different from ``PM_SUSPEND_PREPARE`` below,	because in this case
	additional work is done between the notifiers and the invocation of PM
	callbacks for the "freeze" transition.

``PM_POST_HIBERNATION``
	The system memory state has been restored from a hibernation image or an
	error occurred during hibernation.  Device restore callbacks have been
	executed and tasks have been thawed.

``PM_RESTORE_PREPARE``
	The system is going to restore a hibernation image.  If all goes well,
	the restored image kernel will issue a ``PM_POST_HIBERNATION``
	notification.

``PM_POST_RESTORE``
	An error occurred during restore from hibernation.  Device restore
	callbacks have been executed and tasks have been thawed.

``PM_SUSPEND_PREPARE``
	The system is preparing for suspend.

``PM_POST_SUSPEND``
	The system has just resumed or an error occurred during suspend.  Device
	resume callbacks have been executed and tasks have been thawed.

It is generally assumed that whatever the notifiers do for
``PM_HIBERNATION_PREPARE``, should be undone for ``PM_POST_HIBERNATION``.
Analogously, operations carried out for ``PM_SUSPEND_PREPARE`` should be
reversed for ``PM_POST_SUSPEND``.

Moreover, if one of the notifiers fails for the ``PM_HIBERNATION_PREPARE`` or
``PM_SUSPEND_PREPARE`` event, the notifiers that have already succeeded for that
event will be called for ``PM_POST_HIBERNATION`` or ``PM_POST_SUSPEND``,
respectively.

The hibernation and suspend notifiers are called with :c:data:`pm_mutex` held.
They are defined in the usual way, but their last argument is meaningless (it is
always NULL).

To register and/or unregister a suspend notifier use
:c:func:`register_pm_notifier()` and :c:func:`unregister_pm_notifier()`,
respectively (both defined in :file:`include/linux/suspend.h`).  If you don't
need to unregister the notifier, you can also use the :c:func:`pm_notifier()`
macro defined in :file:`include/linux/suspend.h`.


code:
.. SPDX-License-Identifier: GPL-2.0

======================
PPS - Pulse Per Second
======================

Copyright (C) 2007 Rodolfo Giometti <giometti@enneenne.com>

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

Overview
--------

LinuxPPS provides a programming interface (API) to define in the
system several PPS sources.

PPS means "pulse per second" and a PPS source is just a device which
provides a high precision signal each second so that an application
can use it to adjust system clock time.

A PPS source can be connected to a serial port (usually to the Data
Carrier Detect pin) or to a parallel port (ACK-pin) or to a special
CPU's GPIOs (this is the common case in embedded systems) but in each
case when a new pulse arrives the system must apply to it a timestamp
and record it for userland.

Common use is the combination of the NTPD as userland program, with a
GPS receiver as PPS source, to obtain a wallclock-time with
sub-millisecond synchronisation to UTC.

RFC considerations
------------------

While implementing a PPS API as RFC 2783 defines and using an embedded
CPU GPIO-Pin as physical link to the signal, I encountered a deeper
problem:

   At startup it needs a file descriptor as argument for the function
   time_pps_create().

This implies that the source has a /dev/... entry. This assumption is
OK for the serial and parallel port, where you can do something
useful besides(!) the gathering of timestamps as it is the central
task for a PPS API. But this assumption does not work for a single
purpose GPIO line. In this case even basic file-related functionality
(like read() and write()) makes no sense at all and should not be a
precondition for the use of a PPS API.

The problem can be simply solved if you consider that a PPS source is
not always connected with a GPS data source.

So your programs should check if the GPS data source (the serial port
for instance) is a PPS source too, and if not they should provide the
possibility to open another device as PPS source.

In LinuxPPS the PPS sources are simply char devices usually mapped
into files /dev/pps0, /dev/pps1, etc.

PPS with USB to serial devices
------------------------------

It is possible to grab the PPS from an USB to serial device. However,
you should take into account the latencies and jitter introduced by
the USB stack. Users have reported clock instability around +-1ms when
synchronized with PPS through USB. With USB 2.0, jitter may decrease
down to the order of 125 microseconds.

This may be suitable for time server synchronization with NTP because
of its undersampling and algorithms.

If your device doesn't report PPS, you can check that the feature is
supported by its driver. Most of the time, you only need to add a call
to usb_serial_handle_dcd_change after checking the DCD status (see
ch341 and pl2303 examples).

Coding example
--------------

To register a PPS source into the kernel you should define a struct
pps_source_info as follows::

    static struct pps_source_info pps_ktimer_info = {
	    .name         = "ktimer",
	    .path         = "",
	    .mode         = PPS_CAPTUREASSERT | PPS_OFFSETASSERT |
			    PPS_ECHOASSERT |
			    PPS_CANWAIT | PPS_TSFMT_TSPEC,
	    .echo         = pps_ktimer_echo,
	    .owner        = THIS_MODULE,
    };

and then calling the function pps_register_source() in your
initialization routine as follows::

    source = pps_register_source(&pps_ktimer_info,
			PPS_CAPTUREASSERT | PPS_OFFSETASSERT);

The pps_register_source() prototype is::

  int pps_register_source(struct pps_source_info *info, int default_params)

where "info" is a pointer to a structure that describes a particular
PPS source, "default_params" tells the system what the initial default
parameters for the device should be (it is obvious that these parameters
must be a subset of ones defined in the struct
pps_source_info which describe the capabilities of the driver).

Once you have registered a new PPS source into the system you can
signal an assert event (for example in the interrupt handler routine)
just using::

    pps_event(source, &ts, PPS_CAPTUREASSERT, ptr)

where "ts" is the event's timestamp.

The same function may also run the defined echo function
(pps_ktimer_echo(), passing to it the "ptr" pointer) if the user
asked for that... etc..

Please see the file drivers/pps/clients/pps-ktimer.c for example code.

SYSFS support
-------------

If the SYSFS filesystem is enabled in the kernel it provides a new class::

   $ ls /sys/class/pps/
   pps0/  pps1/  pps2/

Every directory is the ID of a PPS sources defined in the system and
inside you find several files::

   $ ls -F /sys/class/pps/pps0/
   assert     dev        mode       path       subsystem@
   clear      echo       name       power/     uevent

Inside each "assert" and "clear" file you can find the timestamp and a
sequence number::

   $ cat /sys/class/pps/pps0/assert
   1170026870.983207967#8

Where before the "#" is the timestamp in seconds; after it is the
sequence number. Other files are:

 * echo: reports if the PPS source has an echo function or not;

 * mode: reports available PPS functioning modes;

 * name: reports the PPS source's name;

 * path: reports the PPS source's device path, that is the device the
   PPS source is connected to (if it exists).

Testing the PPS support
-----------------------

In order to test the PPS support even without specific hardware you can use
the pps-ktimer driver (see the client subsection in the PPS configuration menu)
and the userland tools available in your distribution's pps-tools package,
http://linuxpps.org , or https://github.com/redlab-i/pps-tools.

Once you have enabled the compilation of pps-ktimer just modprobe it (if
not statically compiled)::

   # modprobe pps-ktimer

and the run ppstest as follow::

   $ ./ppstest /dev/pps1
   trying PPS source "/dev/pps1"
   found PPS source "/dev/pps1"
   ok, found 1 source(s), now start fetching data...
   source 0 - assert 1186592699.388832443, sequence: 364 - clear  0.000000000, sequence: 0
   source 0 - assert 1186592700.388931295, sequence: 365 - clear  0.000000000, sequence: 0
   source 0 - assert 1186592701.389032765, sequence: 366 - clear  0.000000000, sequence: 0

Please note that to compile userland programs, you need the file timepps.h.
This is available in the pps-tools repository mentioned above.

Generators
----------

Sometimes one needs to be able not only to catch PPS signals but to produce
them also. For example, running a distributed simulation, which requires
computers' clock to be synchronized very tightly. One way to do this is to
invent some complicated hardware solutions but it may be neither necessary
nor affordable. The cheap way is to load a PPS generator on one of the
computers (master) and PPS clients on others (slaves), and use very simple
cables to deliver signals using parallel ports, for example.

Parallel port cable pinout::

	pin	name	master      slave
	1	STROBE	  *------     *
	2	D0	  *     |     *
	3	D1	  *     |     *
	4	D2	  *     |     *
	5	D3	  *     |     *
	6	D4	  *     |     *
	7	D5	  *     |     *
	8	D6	  *     |     *
	9	D7	  *     |     *
	10	ACK	  *     ------*
	11	BUSY	  *           *
	12	PE	  *           *
	13	SEL	  *           *
	14	AUTOFD	  *           *
	15	ERROR	  *           *
	16	INIT	  *           *
	17	SELIN	  *           *
	18-25	GND	  *-----------*

Please note that parallel port interrupt occurs only on high->low transition,
so it is used for PPS assert edge. PPS clear edge can be determined only
using polling in the interrupt handler which actually can be done way more
precisely because interrupt handling delays can be quite big and random. So
current parport PPS generator implementation (pps_gen_parport module) is
geared towards using the clear edge for time synchronization.

Clear edge polling is done with disabled interrupts so it's better to select
delay between assert and clear edge as small as possible to reduce system
latencies. But if it is too small slave won't be able to capture clear edge
transition. The default of 30us should be good enough in most situations.
The delay can be selected using 'delay' pps_gen_parport module parameter.


code:
.. SPDX-License-Identifier: GPL-2.0

===========================================
PTP hardware clock infrastructure for Linux
===========================================

  This patch set introduces support for IEEE 1588 PTP clocks in
  Linux. Together with the SO_TIMESTAMPING socket options, this
  presents a standardized method for developing PTP user space
  programs, synchronizing Linux with external clocks, and using the
  ancillary features of PTP hardware clocks.

  A new class driver exports a kernel interface for specific clock
  drivers and a user space interface. The infrastructure supports a
  complete set of PTP hardware clock functionality.

  + Basic clock operations
    - Set time
    - Get time
    - Shift the clock by a given offset atomically
    - Adjust clock frequency

  + Ancillary clock features
    - Time stamp external events
    - Period output signals configurable from user space
    - Low Pass Filter (LPF) access from user space
    - Synchronization of the Linux system time via the PPS subsystem

PTP hardware clock kernel API
=============================

   A PTP clock driver registers itself with the class driver. The
   class driver handles all of the dealings with user space. The
   author of a clock driver need only implement the details of
   programming the clock hardware. The clock driver notifies the class
   driver of asynchronous events (alarms and external time stamps) via
   a simple message passing interface.

   The class driver supports multiple PTP clock drivers. In normal use
   cases, only one PTP clock is needed. However, for testing and
   development, it can be useful to have more than one clock in a
   single system, in order to allow performance comparisons.

PTP hardware clock user space API
=================================

   The class driver also creates a character device for each
   registered clock. User space can use an open file descriptor from
   the character device as a POSIX clock id and may call
   clock_gettime, clock_settime, and clock_adjtime.  These calls
   implement the basic clock operations.

   User space programs may control the clock using standardized
   ioctls. A program may query, enable, configure, and disable the
   ancillary clock features. User space can receive time stamped
   events via blocking read() and poll().

Writing clock drivers
=====================

   Clock drivers include include/linux/ptp_clock_kernel.h and register
   themselves by presenting a 'struct ptp_clock_info' to the
   registration method. Clock drivers must implement all of the
   functions in the interface. If a clock does not offer a particular
   ancillary feature, then the driver should just return -EOPNOTSUPP
   from those functions.

   Drivers must ensure that all of the methods in interface are
   reentrant. Since most hardware implementations treat the time value
   as a 64 bit integer accessed as two 32 bit registers, drivers
   should use spin_lock_irqsave/spin_unlock_irqrestore to protect
   against concurrent access. This locking cannot be accomplished in
   class driver, since the lock may also be needed by the clock
   driver's interrupt service routine.

Supported hardware
==================

   * Freescale eTSEC gianfar

     - 2 Time stamp external triggers, programmable polarity (opt. interrupt)
     - 2 Alarm registers (optional interrupt)
     - 3 Periodic signals (optional interrupt)

   * National DP83640

     - 6 GPIOs programmable as inputs or outputs
     - 6 GPIOs with dedicated functions (LED/JTAG/clock) can also be
       used as general inputs or outputs
     - GPIO inputs can time stamp external triggers
     - GPIO outputs can produce periodic signals
     - 1 interrupt pin

   * Intel IXP465

     - Auxiliary Slave/Master Mode Snapshot (optional interrupt)
     - Target Time (optional interrupt)

   * Renesas (IDT) ClockMatrix™

     - Up to 4 independent PHC channels
     - Integrated low pass filter (LPF), access via .adjPhase (compliant to ITU-T G.8273.2)
     - Programmable output periodic signals
     - Programmable inputs can time stamp external triggers
     - Driver and/or hardware configuration through firmware (idtcm.bin)
          - LPF settings (bandwidth, phase limiting, automatic holdover, physical layer assist (per ITU-T G.8273.2))
          - Programmable output PTP clocks, any frequency up to 1GHz (to other PHY/MAC time stampers, refclk to ASSPs/SoCs/FPGAs)
          - Lock to GNSS input, automatic switching between GNSS and user-space PHC control (optional)


code:
======================================
Pulse Width Modulation (PWM) interface
======================================

This provides an overview about the Linux PWM interface

PWMs are commonly used for controlling LEDs, fans or vibrators in
cell phones. PWMs with a fixed purpose have no need implementing
the Linux PWM API (although they could). However, PWMs are often
found as discrete devices on SoCs which have no fixed purpose. It's
up to the board designer to connect them to LEDs or fans. To provide
this kind of flexibility the generic PWM API exists.

Identifying PWMs
----------------

Users of the legacy PWM API use unique IDs to refer to PWM devices.

Instead of referring to a PWM device via its unique ID, board setup code
should instead register a static mapping that can be used to match PWM
consumers to providers, as given in the following example::

	static struct pwm_lookup board_pwm_lookup[] = {
		PWM_LOOKUP("tegra-pwm", 0, "pwm-backlight", NULL,
			   50000, PWM_POLARITY_NORMAL),
	};

	static void __init board_init(void)
	{
		...
		pwm_add_table(board_pwm_lookup, ARRAY_SIZE(board_pwm_lookup));
		...
	}

Using PWMs
----------

Legacy users can request a PWM device using pwm_request() and free it
after usage with pwm_free().

New users should use the pwm_get() function and pass to it the consumer
device or a consumer name. pwm_put() is used to free the PWM device. Managed
variants of the getter, devm_pwm_get() and devm_fwnode_pwm_get(), also exist.

After being requested, a PWM has to be configured using::

	int pwm_apply_state(struct pwm_device *pwm, struct pwm_state *state);

This API controls both the PWM period/duty_cycle config and the
enable/disable state.

As a consumer, don't rely on the output's state for a disabled PWM. If it's
easily possible, drivers are supposed to emit the inactive state, but some
drivers cannot. If you rely on getting the inactive state, use .duty_cycle=0,
.enabled=true.

There is also a usage_power setting: If set, the PWM driver is only required to
maintain the power output but has more freedom regarding signal form.
If supported by the driver, the signal can be optimized, for example to improve
EMI by phase shifting the individual channels of a chip.

The pwm_config(), pwm_enable() and pwm_disable() functions are just wrappers
around pwm_apply_state() and should not be used if the user wants to change
several parameter at once. For example, if you see pwm_config() and
pwm_{enable,disable}() calls in the same function, this probably means you
should switch to pwm_apply_state().

The PWM user API also allows one to query the PWM state that was passed to the
last invocation of pwm_apply_state() using pwm_get_state(). Note this is
different to what the driver has actually implemented if the request cannot be
satisfied exactly with the hardware in use. There is currently no way for
consumers to get the actually implemented settings.

In addition to the PWM state, the PWM API also exposes PWM arguments, which
are the reference PWM config one should use on this PWM.
PWM arguments are usually platform-specific and allows the PWM user to only
care about dutycycle relatively to the full period (like, duty = 50% of the
period). struct pwm_args contains 2 fields (period and polarity) and should
be used to set the initial PWM config (usually done in the probe function
of the PWM user). PWM arguments are retrieved with pwm_get_args().

All consumers should really be reconfiguring the PWM upon resume as
appropriate. This is the only way to ensure that everything is resumed in
the proper order.

Using PWMs with the sysfs interface
-----------------------------------

If CONFIG_SYSFS is enabled in your kernel configuration a simple sysfs
interface is provided to use the PWMs from userspace. It is exposed at
/sys/class/pwm/. Each probed PWM controller/chip will be exported as
pwmchipN, where N is the base of the PWM chip. Inside the directory you
will find:

  npwm
    The number of PWM channels this chip supports (read-only).

  export
    Exports a PWM channel for use with sysfs (write-only).

  unexport
   Unexports a PWM channel from sysfs (write-only).

The PWM channels are numbered using a per-chip index from 0 to npwm-1.

When a PWM channel is exported a pwmX directory will be created in the
pwmchipN directory it is associated with, where X is the number of the
channel that was exported. The following properties will then be available:

  period
    The total period of the PWM signal (read/write).
    Value is in nanoseconds and is the sum of the active and inactive
    time of the PWM.

  duty_cycle
    The active time of the PWM signal (read/write).
    Value is in nanoseconds and must be less than the period.

  polarity
    Changes the polarity of the PWM signal (read/write).
    Writes to this property only work if the PWM chip supports changing
    the polarity. The polarity can only be changed if the PWM is not
    enabled. Value is the string "normal" or "inversed".

  enable
    Enable/disable the PWM signal (read/write).

	- 0 - disabled
	- 1 - enabled

Implementing a PWM driver
-------------------------

Currently there are two ways to implement pwm drivers. Traditionally
there only has been the barebone API meaning that each driver has
to implement the pwm_*() functions itself. This means that it's impossible
to have multiple PWM drivers in the system. For this reason it's mandatory
for new drivers to use the generic PWM framework.

A new PWM controller/chip can be added using pwmchip_add() and removed
again with pwmchip_remove(). pwmchip_add() takes a filled in struct
pwm_chip as argument which provides a description of the PWM chip, the
number of PWM devices provided by the chip and the chip-specific
implementation of the supported PWM operations to the framework.

When implementing polarity support in a PWM driver, make sure to respect the
signal conventions in the PWM framework. By definition, normal polarity
characterizes a signal starts high for the duration of the duty cycle and
goes low for the remainder of the period. Conversely, a signal with inversed
polarity starts low for the duration of the duty cycle and goes high for the
remainder of the period.

Drivers are encouraged to implement ->apply() instead of the legacy
->enable(), ->disable() and ->config() methods. Doing that should provide
atomicity in the PWM config workflow, which is required when the PWM controls
a critical device (like a regulator).

The implementation of ->get_state() (a method used to retrieve initial PWM
state) is also encouraged for the same reason: letting the PWM user know
about the current PWM state would allow him to avoid glitches.

Drivers should not implement any power management. In other words,
consumers should implement it as described in the "Using PWMs" section.

Locking
-------

The PWM core list manipulations are protected by a mutex, so pwm_request()
and pwm_free() may not be called from an atomic context. Currently the
PWM core does not enforce any locking to pwm_enable(), pwm_disable() and
pwm_config(), so the calling context is currently driver specific. This
is an issue derived from the former barebone API and should be fixed soon.

Helpers
-------

Currently a PWM can only be configured with period_ns and duty_ns. For several
use cases freq_hz and duty_percent might be better. Instead of calculating
this in your driver please consider adding appropriate helpers to the framework.


code:
.. SPDX-License-Identifier: GPL-2.0

===========================
The Linux RapidIO Subsystem
===========================

.. toctree::
   :maxdepth: 1

   rapidio
   sysfs

   tsi721
   mport_cdev
   rio_cm


code:
==================================================================
RapidIO subsystem mport character device driver (rio_mport_cdev.c)
==================================================================

1. Overview
===========

This device driver is the result of collaboration within the RapidIO.org
Software Task Group (STG) between Texas Instruments, Freescale,
Prodrive Technologies, Nokia Networks, BAE and IDT.  Additional input was
received from other members of RapidIO.org. The objective was to create a
character mode driver interface which exposes the capabilities of RapidIO
devices directly to applications, in a manner that allows the numerous and
varied RapidIO implementations to interoperate.

This driver (MPORT_CDEV) provides access to basic RapidIO subsystem operations
for user-space applications. Most of RapidIO operations are supported through
'ioctl' system calls.

When loaded this device driver creates filesystem nodes named rio_mportX in /dev
directory for each registered RapidIO mport device. 'X' in the node name matches
to unique port ID assigned to each local mport device.

Using available set of ioctl commands user-space applications can perform
following RapidIO bus and subsystem operations:

- Reads and writes from/to configuration registers of mport devices
  (RIO_MPORT_MAINT_READ_LOCAL/RIO_MPORT_MAINT_WRITE_LOCAL)
- Reads and writes from/to configuration registers of remote RapidIO devices.
  This operations are defined as RapidIO Maintenance reads/writes in RIO spec.
  (RIO_MPORT_MAINT_READ_REMOTE/RIO_MPORT_MAINT_WRITE_REMOTE)
- Set RapidIO Destination ID for mport devices (RIO_MPORT_MAINT_HDID_SET)
- Set RapidIO Component Tag for mport devices (RIO_MPORT_MAINT_COMPTAG_SET)
- Query logical index of mport devices (RIO_MPORT_MAINT_PORT_IDX_GET)
- Query capabilities and RapidIO link configuration of mport devices
  (RIO_MPORT_GET_PROPERTIES)
- Enable/Disable reporting of RapidIO doorbell events to user-space applications
  (RIO_ENABLE_DOORBELL_RANGE/RIO_DISABLE_DOORBELL_RANGE)
- Enable/Disable reporting of RIO port-write events to user-space applications
  (RIO_ENABLE_PORTWRITE_RANGE/RIO_DISABLE_PORTWRITE_RANGE)
- Query/Control type of events reported through this driver: doorbells,
  port-writes or both (RIO_SET_EVENT_MASK/RIO_GET_EVENT_MASK)
- Configure/Map mport's outbound requests window(s) for specific size,
  RapidIO destination ID, hopcount and request type
  (RIO_MAP_OUTBOUND/RIO_UNMAP_OUTBOUND)
- Configure/Map mport's inbound requests window(s) for specific size,
  RapidIO base address and local memory base address
  (RIO_MAP_INBOUND/RIO_UNMAP_INBOUND)
- Allocate/Free contiguous DMA coherent memory buffer for DMA data transfers
  to/from remote RapidIO devices (RIO_ALLOC_DMA/RIO_FREE_DMA)
- Initiate DMA data transfers to/from remote RapidIO devices (RIO_TRANSFER).
  Supports blocking, asynchronous and posted (a.k.a 'fire-and-forget') data
  transfer modes.
- Check/Wait for completion of asynchronous DMA data transfer
  (RIO_WAIT_FOR_ASYNC)
- Manage device objects supported by RapidIO subsystem (RIO_DEV_ADD/RIO_DEV_DEL).
  This allows implementation of various RapidIO fabric enumeration algorithms
  as user-space applications while using remaining functionality provided by
  kernel RapidIO subsystem.

2. Hardware Compatibility
=========================

This device driver uses standard interfaces defined by kernel RapidIO subsystem
and therefore it can be used with any mport device driver registered by RapidIO
subsystem with limitations set by available mport implementation.

At this moment the most common limitation is availability of RapidIO-specific
DMA engine framework for specific mport device. Users should verify available
functionality of their platform when planning to use this driver:

- IDT Tsi721 PCIe-to-RapidIO bridge device and its mport device driver are fully
  compatible with this driver.
- Freescale SoCs 'fsl_rio' mport driver does not have implementation for RapidIO
  specific DMA engine support and therefore DMA data transfers mport_cdev driver
  are not available.

3. Module parameters
====================

- 'dma_timeout'
      - DMA transfer completion timeout (in msec, default value 3000).
        This parameter set a maximum completion wait time for SYNC mode DMA
        transfer requests and for RIO_WAIT_FOR_ASYNC ioctl requests.

- 'dbg_level'
      - This parameter allows to control amount of debug information
        generated by this device driver. This parameter is formed by set of
        bit masks that correspond to the specific functional blocks.
        For mask definitions see 'drivers/rapidio/devices/rio_mport_cdev.c'
        This parameter can be changed dynamically.
        Use CONFIG_RAPIDIO_DEBUG=y to enable debug output at the top level.

4. Known problems
=================

  None.

5. User-space Applications and API
==================================

API library and applications that use this device driver are available from
RapidIO.org.

6. TODO List
============

- Add support for sending/receiving "raw" RapidIO messaging packets.
- Add memory mapped DMA data transfers as an option when RapidIO-specific DMA
  is not available.


code:
============
Introduction
============

The RapidIO standard is a packet-based fabric interconnect standard designed for
use in embedded systems. Development of the RapidIO standard is directed by the
RapidIO Trade Association (RTA). The current version of the RapidIO specification
is publicly available for download from the RTA web-site [1].

This document describes the basics of the Linux RapidIO subsystem and provides
information on its major components.

1 Overview
==========

Because the RapidIO subsystem follows the Linux device model it is integrated
into the kernel similarly to other buses by defining RapidIO-specific device and
bus types and registering them within the device model.

The Linux RapidIO subsystem is architecture independent and therefore defines
architecture-specific interfaces that provide support for common RapidIO
subsystem operations.

2. Core Components
==================

A typical RapidIO network is a combination of endpoints and switches.
Each of these components is represented in the subsystem by an associated data
structure. The core logical components of the RapidIO subsystem are defined
in include/linux/rio.h file.

2.1 Master Port
---------------

A master port (or mport) is a RapidIO interface controller that is local to the
processor executing the Linux code. A master port generates and receives RapidIO
packets (transactions). In the RapidIO subsystem each master port is represented
by a rio_mport data structure. This structure contains master port specific
resources such as mailboxes and doorbells. The rio_mport also includes a unique
host device ID that is valid when a master port is configured as an enumerating
host.

RapidIO master ports are serviced by subsystem specific mport device drivers
that provide functionality defined for this subsystem. To provide a hardware
independent interface for RapidIO subsystem operations, rio_mport structure
includes rio_ops data structure which contains pointers to hardware specific
implementations of RapidIO functions.

2.2 Device
----------

A RapidIO device is any endpoint (other than mport) or switch in the network.
All devices are presented in the RapidIO subsystem by corresponding rio_dev data
structure. Devices form one global device list and per-network device lists
(depending on number of available mports and networks).

2.3 Switch
----------

A RapidIO switch is a special class of device that routes packets between its
ports towards their final destination. The packet destination port within a
switch is defined by an internal routing table. A switch is presented in the
RapidIO subsystem by rio_dev data structure expanded by additional rio_switch
data structure, which contains switch specific information such as copy of the
routing table and pointers to switch specific functions.

The RapidIO subsystem defines the format and initialization method for subsystem
specific switch drivers that are designed to provide hardware-specific
implementation of common switch management routines.

2.4 Network
-----------

A RapidIO network is a combination of interconnected endpoint and switch devices.
Each RapidIO network known to the system is represented by corresponding rio_net
data structure. This structure includes lists of all devices and local master
ports that form the same network. It also contains a pointer to the default
master port that is used to communicate with devices within the network.

2.5 Device Drivers
------------------

RapidIO device-specific drivers follow Linux Kernel Driver Model and are
intended to support specific RapidIO devices attached to the RapidIO network.

2.6 Subsystem Interfaces
------------------------

RapidIO interconnect specification defines features that may be used to provide
one or more common service layers for all participating RapidIO devices. These
common services may act separately from device-specific drivers or be used by
device-specific drivers. Example of such service provider is the RIONET driver
which implements Ethernet-over-RapidIO interface. Because only one driver can be
registered for a device, all common RapidIO services have to be registered as
subsystem interfaces. This allows to have multiple common services attached to
the same device without blocking attachment of a device-specific driver.

3. Subsystem Initialization
===========================

In order to initialize the RapidIO subsystem, a platform must initialize and
register at least one master port within the RapidIO network. To register mport
within the subsystem controller driver's initialization code calls function
rio_register_mport() for each available master port.

After all active master ports are registered with a RapidIO subsystem,
an enumeration and/or discovery routine may be called automatically or
by user-space command.

RapidIO subsystem can be configured to be built as a statically linked or
modular component of the kernel (see details below).

4. Enumeration and Discovery
============================

4.1 Overview
------------

RapidIO subsystem configuration options allow users to build enumeration and
discovery methods as statically linked components or loadable modules.
An enumeration/discovery method implementation and available input parameters
define how any given method can be attached to available RapidIO mports:
simply to all available mports OR individually to the specified mport device.

Depending on selected enumeration/discovery build configuration, there are
several methods to initiate an enumeration and/or discovery process:

  (a) Statically linked enumeration and discovery process can be started
  automatically during kernel initialization time using corresponding module
  parameters. This was the original method used since introduction of RapidIO
  subsystem. Now this method relies on enumerator module parameter which is
  'rio-scan.scan' for existing basic enumeration/discovery method.
  When automatic start of enumeration/discovery is used a user has to ensure
  that all discovering endpoints are started before the enumerating endpoint
  and are waiting for enumeration to be completed.
  Configuration option CONFIG_RAPIDIO_DISC_TIMEOUT defines time that discovering
  endpoint waits for enumeration to be completed. If the specified timeout
  expires the discovery process is terminated without obtaining RapidIO network
  information. NOTE: a timed out discovery process may be restarted later using
  a user-space command as it is described below (if the given endpoint was
  enumerated successfully).

  (b) Statically linked enumeration and discovery process can be started by
  a command from user space. This initiation method provides more flexibility
  for a system startup compared to the option (a) above. After all participating
  endpoints have been successfully booted, an enumeration process shall be
  started first by issuing a user-space command, after an enumeration is
  completed a discovery process can be started on all remaining endpoints.

  (c) Modular enumeration and discovery process can be started by a command from
  user space. After an enumeration/discovery module is loaded, a network scan
  process can be started by issuing a user-space command.
  Similar to the option (b) above, an enumerator has to be started first.

  (d) Modular enumeration and discovery process can be started by a module
  initialization routine. In this case an enumerating module shall be loaded
  first.

When a network scan process is started it calls an enumeration or discovery
routine depending on the configured role of a master port: host or agent.

Enumeration is performed by a master port if it is configured as a host port by
assigning a host destination ID greater than or equal to zero. The host
destination ID can be assigned to a master port using various methods depending
on RapidIO subsystem build configuration:

  (a) For a statically linked RapidIO subsystem core use command line parameter
  "rapidio.hdid=" with a list of destination ID assignments in order of mport
  device registration. For example, in a system with two RapidIO controllers
  the command line parameter "rapidio.hdid=-1,7" will result in assignment of
  the host destination ID=7 to the second RapidIO controller, while the first
  one will be assigned destination ID=-1.

  (b) If the RapidIO subsystem core is built as a loadable module, in addition
  to the method shown above, the host destination ID(s) can be specified using
  traditional methods of passing module parameter "hdid=" during its loading:

  - from command line: "modprobe rapidio hdid=-1,7", or
  - from modprobe configuration file using configuration command "options",
    like in this example: "options rapidio hdid=-1,7". An example of modprobe
    configuration file is provided in the section below.

NOTES:
  (i) if "hdid=" parameter is omitted all available mport will be assigned
  destination ID = -1;

  (ii) the "hdid=" parameter in systems with multiple mports can have
  destination ID assignments omitted from the end of list (default = -1).

If the host device ID for a specific master port is set to -1, the discovery
process will be performed for it.

The enumeration and discovery routines use RapidIO maintenance transactions
to access the configuration space of devices.

NOTE: If RapidIO switch-specific device drivers are built as loadable modules
they must be loaded before enumeration/discovery process starts.
This requirement is cased by the fact that enumeration/discovery methods invoke
vendor-specific callbacks on early stages.

4.2 Automatic Start of Enumeration and Discovery
------------------------------------------------

Automatic enumeration/discovery start method is applicable only to built-in
enumeration/discovery RapidIO configuration selection. To enable automatic
enumeration/discovery start by existing basic enumerator method set use boot
command line parameter "rio-scan.scan=1".

This configuration requires synchronized start of all RapidIO endpoints that
form a network which will be enumerated/discovered. Discovering endpoints have
to be started before an enumeration starts to ensure that all RapidIO
controllers have been initialized and are ready to be discovered. Configuration
parameter CONFIG_RAPIDIO_DISC_TIMEOUT defines time (in seconds) which
a discovering endpoint will wait for enumeration to be completed.

When automatic enumeration/discovery start is selected, basic method's
initialization routine calls rio_init_mports() to perform enumeration or
discovery for all known mport devices.

Depending on RapidIO network size and configuration this automatic
enumeration/discovery start method may be difficult to use due to the
requirement for synchronized start of all endpoints.

4.3 User-space Start of Enumeration and Discovery
-------------------------------------------------

User-space start of enumeration and discovery can be used with built-in and
modular build configurations. For user-space controlled start RapidIO subsystem
creates the sysfs write-only attribute file '/sys/bus/rapidio/scan'. To initiate
an enumeration or discovery process on specific mport device, a user needs to
write mport_ID (not RapidIO destination ID) into that file. The mport_ID is a
sequential number (0 ... RIO_MAX_MPORTS) assigned during mport device
registration. For example for machine with single RapidIO controller, mport_ID
for that controller always will be 0.

To initiate RapidIO enumeration/discovery on all available mports a user may
write '-1' (or RIO_MPORT_ANY) into the scan attribute file.

4.4 Basic Enumeration Method
----------------------------

This is an original enumeration/discovery method which is available since
first release of RapidIO subsystem code. The enumeration process is
implemented according to the enumeration algorithm outlined in the RapidIO
Interconnect Specification: Annex I [1].

This method can be configured as statically linked or loadable module.
The method's single parameter "scan" allows to trigger the enumeration/discovery
process from module initialization routine.

This enumeration/discovery method can be started only once and does not support
unloading if it is built as a module.

The enumeration process traverses the network using a recursive depth-first
algorithm. When a new device is found, the enumerator takes ownership of that
device by writing into the Host Device ID Lock CSR. It does this to ensure that
the enumerator has exclusive right to enumerate the device. If device ownership
is successfully acquired, the enumerator allocates a new rio_dev structure and
initializes it according to device capabilities.

If the device is an endpoint, a unique device ID is assigned to it and its value
is written into the device's Base Device ID CSR.

If the device is a switch, the enumerator allocates an additional rio_switch
structure to store switch specific information. Then the switch's vendor ID and
device ID are queried against a table of known RapidIO switches. Each switch
table entry contains a pointer to a switch-specific initialization routine that
initializes pointers to the rest of switch specific operations, and performs
hardware initialization if necessary. A RapidIO switch does not have a unique
device ID; it relies on hopcount and routing for device ID of an attached
endpoint if access to its configuration registers is required. If a switch (or
chain of switches) does not have any endpoint (except enumerator) attached to
it, a fake device ID will be assigned to configure a route to that switch.
In the case of a chain of switches without endpoint, one fake device ID is used
to configure a route through the entire chain and switches are differentiated by
their hopcount value.

For both endpoints and switches the enumerator writes a unique component tag
into device's Component Tag CSR. That unique value is used by the error
management notification mechanism to identify a device that is reporting an
error management event.

Enumeration beyond a switch is completed by iterating over each active egress
port of that switch. For each active link, a route to a default device ID
(0xFF for 8-bit systems and 0xFFFF for 16-bit systems) is temporarily written
into the routing table. The algorithm recurs by calling itself with hopcount + 1
and the default device ID in order to access the device on the active port.

After the host has completed enumeration of the entire network it releases
devices by clearing device ID locks (calls rio_clear_locks()). For each endpoint
in the system, it sets the Discovered bit in the Port General Control CSR
to indicate that enumeration is completed and agents are allowed to execute
passive discovery of the network.

The discovery process is performed by agents and is similar to the enumeration
process that is described above. However, the discovery process is performed
without changes to the existing routing because agents only gather information
about RapidIO network structure and are building an internal map of discovered
devices. This way each Linux-based component of the RapidIO subsystem has
a complete view of the network. The discovery process can be performed
simultaneously by several agents. After initializing its RapidIO master port
each agent waits for enumeration completion by the host for the configured wait
time period. If this wait time period expires before enumeration is completed,
an agent skips RapidIO discovery and continues with remaining kernel
initialization.

4.5 Adding New Enumeration/Discovery Method
-------------------------------------------

RapidIO subsystem code organization allows addition of new enumeration/discovery
methods as new configuration options without significant impact to the core
RapidIO code.

A new enumeration/discovery method has to be attached to one or more mport
devices before an enumeration/discovery process can be started. Normally,
method's module initialization routine calls rio_register_scan() to attach
an enumerator to a specified mport device (or devices). The basic enumerator
implementation demonstrates this process.

4.6 Using Loadable RapidIO Switch Drivers
-----------------------------------------

In the case when RapidIO switch drivers are built as loadable modules a user
must ensure that they are loaded before the enumeration/discovery starts.
This process can be automated by specifying pre- or post- dependencies in the
RapidIO-specific modprobe configuration file as shown in the example below.

File /etc/modprobe.d/rapidio.conf::

  # Configure RapidIO subsystem modules

  # Set enumerator host destination ID (overrides kernel command line option)
  options rapidio hdid=-1,2

  # Load RapidIO switch drivers immediately after rapidio core module was loaded
  softdep rapidio post: idt_gen2 idtcps tsi57x

  # OR :

  # Load RapidIO switch drivers just before rio-scan enumerator module is loaded
  softdep rio-scan pre: idt_gen2 idtcps tsi57x

  --------------------------

NOTE:
  In the example above, one of "softdep" commands must be removed or
  commented out to keep required module loading sequence.

5. References
=============

[1] RapidIO Trade Association. RapidIO Interconnect Specifications.
    http://www.rapidio.org.

[2] Rapidio TA. Technology Comparisons.
    http://www.rapidio.org/education/technology_comparisons/

[3] RapidIO support for Linux.
    https://lwn.net/Articles/139118/

[4] Matt Porter. RapidIO for Linux. Ottawa Linux Symposium, 2005
    https://www.kernel.org/doc/ols/2005/ols2005v2-pages-43-56.pdf