code:
.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>

===============================================
``intel_pstate`` CPU Performance Scaling Driver
===============================================

:Copyright: |copy| 2017 Intel Corporation

:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

General Information
===================

``intel_pstate`` is a part of the
:doc:`CPU performance scaling subsystem <cpufreq>` in the Linux kernel
(``CPUFreq``).  It is a scaling driver for the Sandy Bridge and later
generations of Intel processors.  Note, however, that some of those processors
may not be supported.  [To understand ``intel_pstate`` it is necessary to know
how ``CPUFreq`` works in general, so this is the time to read
Documentation/admin-guide/pm/cpufreq.rst if you have not done that yet.]

For the processors supported by ``intel_pstate``, the P-state concept is broader
than just an operating frequency or an operating performance point (see the
LinuxCon Europe 2015 presentation by Kristen Accardi [1]_ for more
information about that).  For this reason, the representation of P-states used
by ``intel_pstate`` internally follows the hardware specification (for details
refer to Intel Software Developer’s Manual [2]_).  However, the ``CPUFreq`` core
uses frequencies for identifying operating performance points of CPUs and
frequencies are involved in the user space interface exposed by it, so
``intel_pstate`` maps its internal representation of P-states to frequencies too
(fortunately, that mapping is unambiguous).  At the same time, it would not be
practical for ``intel_pstate`` to supply the ``CPUFreq`` core with a table of
available frequencies due to the possible size of it, so the driver does not do
that.  Some functionality of the core is limited by that.

Since the hardware P-state selection interface used by ``intel_pstate`` is
available at the logical CPU level, the driver always works with individual
CPUs.  Consequently, if ``intel_pstate`` is in use, every ``CPUFreq`` policy
object corresponds to one logical CPU and ``CPUFreq`` policies are effectively
equivalent to CPUs.  In particular, this means that they become "inactive" every
time the corresponding CPU is taken offline and need to be re-initialized when
it goes back online.

``intel_pstate`` is not modular, so it cannot be unloaded, which means that the
only way to pass early-configuration-time parameters to it is via the kernel
command line.  However, its configuration can be adjusted via ``sysfs`` to a
great extent.  In some configurations it even is possible to unregister it via
``sysfs`` which allows another ``CPUFreq`` scaling driver to be loaded and
registered (see `below <status_attr_>`_).

Operation Modes
===============

``intel_pstate`` can operate in two different modes, active or passive.  In the
active mode, it uses its own internal performance scaling governor algorithm or
allows the hardware to do performance scaling by itself, while in the passive
mode it responds to requests made by a generic ``CPUFreq`` governor implementing
a certain performance scaling algorithm.  Which of them will be in effect
depends on what kernel command line options are used and on the capabilities of
the processor.

Active Mode
-----------

This is the default operation mode of ``intel_pstate`` for processors with
hardware-managed P-states (HWP) support.  If it works in this mode, the
``scaling_driver`` policy attribute in ``sysfs`` for all ``CPUFreq`` policies
contains the string "intel_pstate".

In this mode the driver bypasses the scaling governors layer of ``CPUFreq`` and
provides its own scaling algorithms for P-state selection.  Those algorithms
can be applied to ``CPUFreq`` policies in the same way as generic scaling
governors (that is, through the ``scaling_governor`` policy attribute in
``sysfs``).  [Note that different P-state selection algorithms may be chosen for
different policies, but that is not recommended.]

They are not generic scaling governors, but their names are the same as the
names of some of those governors.  Moreover, confusingly enough, they generally
do not work in the same way as the generic governors they share the names with.
For example, the ``powersave`` P-state selection algorithm provided by
``intel_pstate`` is not a counterpart of the generic ``powersave`` governor
(roughly, it corresponds to the ``schedutil`` and ``ondemand`` governors).

There are two P-state selection algorithms provided by ``intel_pstate`` in the
active mode: ``powersave`` and ``performance``.  The way they both operate
depends on whether or not the hardware-managed P-states (HWP) feature has been
enabled in the processor and possibly on the processor model.

Which of the P-state selection algorithms is used by default depends on the
:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option.
Namely, if that option is set, the ``performance`` algorithm will be used by
default, and the other one will be used by default if it is not set.

Active Mode With HWP
~~~~~~~~~~~~~~~~~~~~

If the processor supports the HWP feature, it will be enabled during the
processor initialization and cannot be disabled after that.  It is possible
to avoid enabling it by passing the ``intel_pstate=no_hwp`` argument to the
kernel in the command line.

If the HWP feature has been enabled, ``intel_pstate`` relies on the processor to
select P-states by itself, but still it can give hints to the processor's
internal P-state selection logic.  What those hints are depends on which P-state
selection algorithm has been applied to the given policy (or to the CPU it
corresponds to).

Even though the P-state selection is carried out by the processor automatically,
``intel_pstate`` registers utilization update callbacks with the CPU scheduler
in this mode.  However, they are not used for running a P-state selection
algorithm, but for periodic updates of the current CPU frequency information to
be made available from the ``scaling_cur_freq`` policy attribute in ``sysfs``.

HWP + ``performance``
.....................

In this configuration ``intel_pstate`` will write 0 to the processor's
Energy-Performance Preference (EPP) knob (if supported) or its
Energy-Performance Bias (EPB) knob (otherwise), which means that the processor's
internal P-state selection logic is expected to focus entirely on performance.

This will override the EPP/EPB setting coming from the ``sysfs`` interface
(see `Energy vs Performance Hints`_ below).  Moreover, any attempts to change
the EPP/EPB to a value different from 0 ("performance") via ``sysfs`` in this
configuration will be rejected.

Also, in this configuration the range of P-states available to the processor's
internal P-state selection logic is always restricted to the upper boundary
(that is, the maximum P-state that the driver is allowed to use).

HWP + ``powersave``
...................

In this configuration ``intel_pstate`` will set the processor's
Energy-Performance Preference (EPP) knob (if supported) or its
Energy-Performance Bias (EPB) knob (otherwise) to whatever value it was
previously set to via ``sysfs`` (or whatever default value it was
set to by the platform firmware).  This usually causes the processor's
internal P-state selection logic to be less performance-focused.

Active Mode Without HWP
~~~~~~~~~~~~~~~~~~~~~~~

This operation mode is optional for processors that do not support the HWP
feature or when the ``intel_pstate=no_hwp`` argument is passed to the kernel in
the command line.  The active mode is used in those cases if the
``intel_pstate=active`` argument is passed to the kernel in the command line.
In this mode ``intel_pstate`` may refuse to work with processors that are not
recognized by it.  [Note that ``intel_pstate`` will never refuse to work with
any processor with the HWP feature enabled.]

In this mode ``intel_pstate`` registers utilization update callbacks with the
CPU scheduler in order to run a P-state selection algorithm, either
``powersave`` or ``performance``, depending on the ``scaling_governor`` policy
setting in ``sysfs``.  The current CPU frequency information to be made
available from the ``scaling_cur_freq`` policy attribute in ``sysfs`` is
periodically updated by those utilization update callbacks too.

``performance``
...............

Without HWP, this P-state selection algorithm is always the same regardless of
the processor model and platform configuration.

It selects the maximum P-state it is allowed to use, subject to limits set via
``sysfs``, every time the driver configuration for the given CPU is updated
(e.g. via ``sysfs``).

This is the default P-state selection algorithm if the
:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option
is set.

``powersave``
.............

Without HWP, this P-state selection algorithm is similar to the algorithm
implemented by the generic ``schedutil`` scaling governor except that the
utilization metric used by it is based on numbers coming from feedback
registers of the CPU.  It generally selects P-states proportional to the
current CPU utilization.

This algorithm is run by the driver's utilization update callback for the
given CPU when it is invoked by the CPU scheduler, but not more often than
every 10 ms.  Like in the ``performance`` case, the hardware configuration
is not touched if the new P-state turns out to be the same as the current
one.

This is the default P-state selection algorithm if the
:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option
is not set.

Passive Mode
------------

This is the default operation mode of ``intel_pstate`` for processors without
hardware-managed P-states (HWP) support.  It is always used if the
``intel_pstate=passive`` argument is passed to the kernel in the command line
regardless of whether or not the given processor supports HWP.  [Note that the
``intel_pstate=no_hwp`` setting causes the driver to start in the passive mode
if it is not combined with ``intel_pstate=active``.]  Like in the active mode
without HWP support, in this mode ``intel_pstate`` may refuse to work with
processors that are not recognized by it if HWP is prevented from being enabled
through the kernel command line.

If the driver works in this mode, the ``scaling_driver`` policy attribute in
``sysfs`` for all ``CPUFreq`` policies contains the string "intel_cpufreq".
Then, the driver behaves like a regular ``CPUFreq`` scaling driver.  That is,
it is invoked by generic scaling governors when necessary to talk to the
hardware in order to change the P-state of a CPU (in particular, the
``schedutil`` governor can invoke it directly from scheduler context).

While in this mode, ``intel_pstate`` can be used with all of the (generic)
scaling governors listed by the ``scaling_available_governors`` policy attribute
in ``sysfs`` (and the P-state selection algorithms described above are not
used).  Then, it is responsible for the configuration of policy objects
corresponding to CPUs and provides the ``CPUFreq`` core (and the scaling
governors attached to the policy objects) with accurate information on the
maximum and minimum operating frequencies supported by the hardware (including
the so-called "turbo" frequency ranges).  In other words, in the passive mode
the entire range of available P-states is exposed by ``intel_pstate`` to the
``CPUFreq`` core.  However, in this mode the driver does not register
utilization update callbacks with the CPU scheduler and the ``scaling_cur_freq``
information comes from the ``CPUFreq`` core (and is the last frequency selected
by the current scaling governor for the given policy).

.. _turbo:

Turbo P-states Support
======================

In the majority of cases, the entire range of P-states available to
``intel_pstate`` can be divided into two sub-ranges that correspond to
different types of processor behavior, above and below a boundary that
will be referred to as the "turbo threshold" in what follows.

The P-states above the turbo threshold are referred to as "turbo P-states" and
the whole sub-range of P-states they belong to is referred to as the "turbo
range".  These names are related to the Turbo Boost technology allowing a
multicore processor to opportunistically increase the P-state of one or more
cores if there is enough power to do that and if that is not going to cause the
thermal envelope of the processor package to be exceeded.

Specifically, if software sets the P-state of a CPU core within the turbo range
(that is, above the turbo threshold), the processor is permitted to take over
performance scaling control for that core and put it into turbo P-states of its
choice going forward.  However, that permission is interpreted differently by
different processor generations.  Namely, the Sandy Bridge generation of
processors will never use any P-states above the last one set by software for
the given core, even if it is within the turbo range, whereas all of the later
processor generations will take it as a license to use any P-states from the
turbo range, even above the one set by software.  In other words, on those
processors setting any P-state from the turbo range will enable the processor
to put the given core into all turbo P-states up to and including the maximum
supported one as it sees fit.

One important property of turbo P-states is that they are not sustainable.  More
precisely, there is no guarantee that any CPUs will be able to stay in any of
those states indefinitely, because the power distribution within the processor
package may change over time  or the thermal envelope it was designed for might
be exceeded if a turbo P-state was used for too long.

In turn, the P-states below the turbo threshold generally are sustainable.  In
fact, if one of them is set by software, the processor is not expected to change
it to a lower one unless in a thermal stress or a power limit violation
situation (a higher P-state may still be used if it is set for another CPU in
the same package at the same time, for example).

Some processors allow multiple cores to be in turbo P-states at the same time,
but the maximum P-state that can be set for them generally depends on the number
of cores running concurrently.  The maximum turbo P-state that can be set for 3
cores at the same time usually is lower than the analogous maximum P-state for
2 cores, which in turn usually is lower than the maximum turbo P-state that can
be set for 1 core.  The one-core maximum turbo P-state is thus the maximum
supported one overall.

The maximum supported turbo P-state, the turbo threshold (the maximum supported
non-turbo P-state) and the minimum supported P-state are specific to the
processor model and can be determined by reading the processor's model-specific
registers (MSRs).  Moreover, some processors support the Configurable TDP
(Thermal Design Power) feature and, when that feature is enabled, the turbo
threshold effectively becomes a configurable value that can be set by the
platform firmware.

Unlike ``_PSS`` objects in the ACPI tables, ``intel_pstate`` always exposes
the entire range of available P-states, including the whole turbo range, to the
``CPUFreq`` core and (in the passive mode) to generic scaling governors.  This
generally causes turbo P-states to be set more often when ``intel_pstate`` is
used relative to ACPI-based CPU performance scaling (see `below <acpi-cpufreq_>`_
for more information).

Moreover, since ``intel_pstate`` always knows what the real turbo threshold is
(even if the Configurable TDP feature is enabled in the processor), its
``no_turbo`` attribute in ``sysfs`` (described `below <no_turbo_attr_>`_) should
work as expected in all cases (that is, if set to disable turbo P-states, it
always should prevent ``intel_pstate`` from using them).

Processor Support
=================

To handle a given processor ``intel_pstate`` requires a number of different
pieces of information on it to be known, including:

 * The minimum supported P-state.

 * The maximum supported `non-turbo P-state <turbo_>`_.

 * Whether or not turbo P-states are supported at all.

 * The maximum supported `one-core turbo P-state <turbo_>`_ (if turbo P-states
   are supported).

 * The scaling formula to translate the driver's internal representation
   of P-states into frequencies and the other way around.

Generally, ways to obtain that information are specific to the processor model
or family.  Although it often is possible to obtain all of it from the processor
itself (using model-specific registers), there are cases in which hardware
manuals need to be consulted to get to it too.

For this reason, there is a list of supported processors in ``intel_pstate`` and
the driver initialization will fail if the detected processor is not in that
list, unless it supports the HWP feature.  [The interface to obtain all of the
information listed above is the same for all of the processors supporting the
HWP feature, which is why ``intel_pstate`` works with all of them.]

User Space Interface in ``sysfs``
=================================

Global Attributes
-----------------

``intel_pstate`` exposes several global attributes (files) in ``sysfs`` to
control its functionality at the system level.  They are located in the
``/sys/devices/system/cpu/intel_pstate/`` directory and affect all CPUs.

Some of them are not present if the ``intel_pstate=per_cpu_perf_limits``
argument is passed to the kernel in the command line.

``max_perf_pct``
	Maximum P-state the driver is allowed to set in percent of the
	maximum supported performance level (the highest supported `turbo
	P-state <turbo_>`_).

	This attribute will not be exposed if the
	``intel_pstate=per_cpu_perf_limits`` argument is present in the kernel
	command line.

``min_perf_pct``
	Minimum P-state the driver is allowed to set in percent of the
	maximum supported performance level (the highest supported `turbo
	P-state <turbo_>`_).

	This attribute will not be exposed if the
	``intel_pstate=per_cpu_perf_limits`` argument is present in the kernel
	command line.

``num_pstates``
	Number of P-states supported by the processor (between 0 and 255
	inclusive) including both turbo and non-turbo P-states (see
	`Turbo P-states Support`_).

	This attribute is present only if the value exposed by it is the same
	for all of the CPUs in the system.

	The value of this attribute is not affected by the ``no_turbo``
	setting described `below <no_turbo_attr_>`_.

	This attribute is read-only.

``turbo_pct``
	Ratio of the `turbo range <turbo_>`_ size to the size of the entire
	range of supported P-states, in percent.

	This attribute is present only if the value exposed by it is the same
	for all of the CPUs in the system.

	This attribute is read-only.

.. _no_turbo_attr:

``no_turbo``
	If set (equal to 1), the driver is not allowed to set any turbo P-states
	(see `Turbo P-states Support`_).  If unset (equal to 0, which is the
	default), turbo P-states can be set by the driver.
	[Note that ``intel_pstate`` does not support the general ``boost``
	attribute (supported by some other scaling drivers) which is replaced
	by this one.]

	This attribute does not affect the maximum supported frequency value
	supplied to the ``CPUFreq`` core and exposed via the policy interface,
	but it affects the maximum possible value of per-policy P-state	limits
	(see `Interpretation of Policy Attributes`_ below for details).

``hwp_dynamic_boost``
	This attribute is only present if ``intel_pstate`` works in the
	`active mode with the HWP feature enabled <Active Mode With HWP_>`_ in
	the processor.  If set (equal to 1), it causes the minimum P-state limit
	to be increased dynamically for a short time whenever a task previously
	waiting on I/O is selected to run on a given logical CPU (the purpose
	of this mechanism is to improve performance).

	This setting has no effect on logical CPUs whose minimum P-state limit
	is directly set to the highest non-turbo P-state or above it.

.. _status_attr:

``status``
	Operation mode of the driver: "active", "passive" or "off".

	"active"
		The driver is functional and in the `active mode
		<Active Mode_>`_.

	"passive"
		The driver is functional and in the `passive mode
		<Passive Mode_>`_.

	"off"
		The driver is not functional (it is not registered as a scaling
		driver with the ``CPUFreq`` core).

	This attribute can be written to in order to change the driver's
	operation mode or to unregister it.  The string written to it must be
	one of the possible values of it and, if successful, the write will
	cause the driver to switch over to the operation mode represented by
	that string - or to be unregistered in the "off" case.  [Actually,
	switching over from the active mode to the passive mode or the other
	way around causes the driver to be unregistered and registered again
	with a different set of callbacks, so all of its settings (the global
	as well as the per-policy ones) are then reset to their default
	values, possibly depending on the target operation mode.]

``energy_efficiency``
	This attribute is only present on platforms with CPUs matching the Kaby
	Lake or Coffee Lake desktop CPU model. By default, energy-efficiency
	optimizations are disabled on these CPU models if HWP is enabled.
	Enabling energy-efficiency optimizations may limit maximum operating
	frequency with or without the HWP feature.  With HWP enabled, the
	optimizations are done only in the turbo frequency range.  Without it,
	they are done in the entire available frequency range.  Setting this
	attribute to "1" enables the energy-efficiency optimizations and setting
	to "0" disables them.

Interpretation of Policy Attributes
-----------------------------------

The interpretation of some ``CPUFreq`` policy attributes described in
Documentation/admin-guide/pm/cpufreq.rst is special with ``intel_pstate``
as the current scaling driver and it generally depends on the driver's
`operation mode <Operation Modes_>`_.

First of all, the values of the ``cpuinfo_max_freq``, ``cpuinfo_min_freq`` and
``scaling_cur_freq`` attributes are produced by applying a processor-specific
multiplier to the internal P-state representation used by ``intel_pstate``.
Also, the values of the ``scaling_max_freq`` and ``scaling_min_freq``
attributes are capped by the frequency corresponding to the maximum P-state that
the driver is allowed to set.

If the ``no_turbo`` `global attribute <no_turbo_attr_>`_ is set, the driver is
not allowed to use turbo P-states, so the maximum value of ``scaling_max_freq``
and ``scaling_min_freq`` is limited to the maximum non-turbo P-state frequency.
Accordingly, setting ``no_turbo`` causes ``scaling_max_freq`` and
``scaling_min_freq`` to go down to that value if they were above it before.
However, the old values of ``scaling_max_freq`` and ``scaling_min_freq`` will be
restored after unsetting ``no_turbo``, unless these attributes have been written
to after ``no_turbo`` was set.

If ``no_turbo`` is not set, the maximum possible value of ``scaling_max_freq``
and ``scaling_min_freq`` corresponds to the maximum supported turbo P-state,
which also is the value of ``cpuinfo_max_freq`` in either case.

Next, the following policy attributes have special meaning if
``intel_pstate`` works in the `active mode <Active Mode_>`_:

``scaling_available_governors``
	List of P-state selection algorithms provided by ``intel_pstate``.

``scaling_governor``
	P-state selection algorithm provided by ``intel_pstate`` currently in
	use with the given policy.

``scaling_cur_freq``
	Frequency of the average P-state of the CPU represented by the given
	policy for the time interval between the last two invocations of the
	driver's utilization update callback by the CPU scheduler for that CPU.

One more policy attribute is present if the HWP feature is enabled in the
processor:

``base_frequency``
	Shows the base frequency of the CPU. Any frequency above this will be
	in the turbo frequency range.

The meaning of these attributes in the `passive mode <Passive Mode_>`_ is the
same as for other scaling drivers.

Additionally, the value of the ``scaling_driver`` attribute for ``intel_pstate``
depends on the operation mode of the driver.  Namely, it is either
"intel_pstate" (in the `active mode <Active Mode_>`_) or "intel_cpufreq" (in the
`passive mode <Passive Mode_>`_).

Coordination of P-State Limits
------------------------------

``intel_pstate`` allows P-state limits to be set in two ways: with the help of
the ``max_perf_pct`` and ``min_perf_pct`` `global attributes
<Global Attributes_>`_ or via the ``scaling_max_freq`` and ``scaling_min_freq``
``CPUFreq`` policy attributes.  The coordination between those limits is based
on the following rules, regardless of the current operation mode of the driver:

 1. All CPUs are affected by the global limits (that is, none of them can be
    requested to run faster than the global maximum and none of them can be
    requested to run slower than the global minimum).

 2. Each individual CPU is affected by its own per-policy limits (that is, it
    cannot be requested to run faster than its own per-policy maximum and it
    cannot be requested to run slower than its own per-policy minimum). The
    effective performance depends on whether the platform supports per core
    P-states, hyper-threading is enabled and on current performance requests
    from other CPUs. When platform doesn't support per core P-states, the
    effective performance can be more than the policy limits set on a CPU, if
    other CPUs are requesting higher performance at that moment. Even with per
    core P-states support, when hyper-threading is enabled, if the sibling CPU
    is requesting higher performance, the other siblings will get higher
    performance than their policy limits.

 3. The global and per-policy limits can be set independently.

In the `active mode with the HWP feature enabled <Active Mode With HWP_>`_, the
resulting effective values are written into hardware registers whenever the
limits change in order to request its internal P-state selection logic to always
set P-states within these limits.  Otherwise, the limits are taken into account
by scaling governors (in the `passive mode <Passive Mode_>`_) and by the driver
every time before setting a new P-state for a CPU.

Additionally, if the ``intel_pstate=per_cpu_perf_limits`` command line argument
is passed to the kernel, ``max_perf_pct`` and ``min_perf_pct`` are not exposed
at all and the only way to set the limits is by using the policy attributes.

Energy vs Performance Hints
---------------------------

If the hardware-managed P-states (HWP) is enabled in the processor, additional
attributes, intended to allow user space to help ``intel_pstate`` to adjust the
processor's internal P-state selection logic by focusing it on performance or on
energy-efficiency, or somewhere between the two extremes, are present in every
``CPUFreq`` policy directory in ``sysfs``.  They are :

``energy_performance_preference``
	Current value of the energy vs performance hint for the given policy
	(or the CPU represented by it).

	The hint can be changed by writing to this attribute.

``energy_performance_available_preferences``
	List of strings that can be written to the
	``energy_performance_preference`` attribute.

	They represent different energy vs performance hints and should be
	self-explanatory, except that ``default`` represents whatever hint
	value was set by the platform firmware.

Strings written to the ``energy_performance_preference`` attribute are
internally translated to integer values written to the processor's
Energy-Performance Preference (EPP) knob (if supported) or its
Energy-Performance Bias (EPB) knob. It is also possible to write a positive
integer value between 0 to 255, if the EPP feature is present. If the EPP
feature is not present, writing integer value to this attribute is not
supported. In this case, user can use the
"/sys/devices/system/cpu/cpu*/power/energy_perf_bias" interface.

[Note that tasks may by migrated from one CPU to another by the scheduler's
load-balancing algorithm and if different energy vs performance hints are
set for those CPUs, that may lead to undesirable outcomes.  To avoid such
issues it is better to set the same energy vs performance hint for all CPUs
or to pin every task potentially sensitive to them to a specific CPU.]

.. _acpi-cpufreq:

``intel_pstate`` vs ``acpi-cpufreq``
====================================

On the majority of systems supported by ``intel_pstate``, the ACPI tables
provided by the platform firmware contain ``_PSS`` objects returning information
that can be used for CPU performance scaling (refer to the ACPI specification
[3]_ for details on the ``_PSS`` objects and the format of the information
returned by them).

The information returned by the ACPI ``_PSS`` objects is used by the
``acpi-cpufreq`` scaling driver.  On systems supported by ``intel_pstate``
the ``acpi-cpufreq`` driver uses the same hardware CPU performance scaling
interface, but the set of P-states it can use is limited by the ``_PSS``
output.

On those systems each ``_PSS`` object returns a list of P-states supported by
the corresponding CPU which basically is a subset of the P-states range that can
be used by ``intel_pstate`` on the same system, with one exception: the whole
`turbo range <turbo_>`_ is represented by one item in it (the topmost one).  By
convention, the frequency returned by ``_PSS`` for that item is greater by 1 MHz
than the frequency of the highest non-turbo P-state listed by it, but the
corresponding P-state representation (following the hardware specification)
returned for it matches the maximum supported turbo P-state (or is the
special value 255 meaning essentially "go as high as you can get").

The list of P-states returned by ``_PSS`` is reflected by the table of
available frequencies supplied by ``acpi-cpufreq`` to the ``CPUFreq`` core and
scaling governors and the minimum and maximum supported frequencies reported by
it come from that list as well.  In particular, given the special representation
of the turbo range described above, this means that the maximum supported
frequency reported by ``acpi-cpufreq`` is higher by 1 MHz than the frequency
of the highest supported non-turbo P-state listed by ``_PSS`` which, of course,
affects decisions made by the scaling governors, except for ``powersave`` and
``performance``.

For example, if a given governor attempts to select a frequency proportional to
estimated CPU load and maps the load of 100% to the maximum supported frequency
(possibly multiplied by a constant), then it will tend to choose P-states below
the turbo threshold if ``acpi-cpufreq`` is used as the scaling driver, because
in that case the turbo range corresponds to a small fraction of the frequency
band it can use (1 MHz vs 1 GHz or more).  In consequence, it will only go to
the turbo range for the highest loads and the other loads above 50% that might
benefit from running at turbo frequencies will be given non-turbo P-states
instead.

One more issue related to that may appear on systems supporting the
`Configurable TDP feature <turbo_>`_ allowing the platform firmware to set the
turbo threshold.  Namely, if that is not coordinated with the lists of P-states
returned by ``_PSS`` properly, there may be more than one item corresponding to
a turbo P-state in those lists and there may be a problem with avoiding the
turbo range (if desirable or necessary).  Usually, to avoid using turbo
P-states overall, ``acpi-cpufreq`` simply avoids using the topmost state listed
by ``_PSS``, but that is not sufficient when there are other turbo P-states in
the list returned by it.

Apart from the above, ``acpi-cpufreq`` works like ``intel_pstate`` in the
`passive mode <Passive Mode_>`_, except that the number of P-states it can set
is limited to the ones listed by the ACPI ``_PSS`` objects.

Kernel Command Line Options for ``intel_pstate``
================================================

Several kernel command line options can be used to pass early-configuration-time
parameters to ``intel_pstate`` in order to enforce specific behavior of it.  All
of them have to be prepended with the ``intel_pstate=`` prefix.

``disable``
	Do not register ``intel_pstate`` as the scaling driver even if the
	processor is supported by it.

``active``
	Register ``intel_pstate`` in the `active mode <Active Mode_>`_ to start
	with.

``passive``
	Register ``intel_pstate`` in the `passive mode <Passive Mode_>`_ to
	start with.

``force``
	Register ``intel_pstate`` as the scaling driver instead of
	``acpi-cpufreq`` even if the latter is preferred on the given system.

	This may prevent some platform features (such as thermal controls and
	power capping) that rely on the availability of ACPI P-states
	information from functioning as expected, so it should be used with
	caution.

	This option does not work with processors that are not supported by
	``intel_pstate`` and on platforms where the ``pcc-cpufreq`` scaling
	driver is used instead of ``acpi-cpufreq``.

``no_hwp``
	Do not enable the hardware-managed P-states (HWP) feature even if it is
	supported by the processor.

``hwp_only``
	Register ``intel_pstate`` as the scaling driver only if the
	hardware-managed P-states (HWP) feature is supported by the processor.

``support_acpi_ppc``
	Take ACPI ``_PPC`` performance limits into account.

	If the preferred power management profile in the FADT (Fixed ACPI
	Description Table) is set to "Enterprise Server" or "Performance
	Server", the ACPI ``_PPC`` limits are taken into account by default
	and this option has no effect.

``per_cpu_perf_limits``
	Use per-logical-CPU P-State limits (see `Coordination of P-state
	Limits`_ for details).

Diagnostics and Tuning
======================

Trace Events
------------

There are two static trace events that can be used for ``intel_pstate``
diagnostics.  One of them is the ``cpu_frequency`` trace event generally used
by ``CPUFreq``, and the other one is the ``pstate_sample`` trace event specific
to ``intel_pstate``.  Both of them are triggered by ``intel_pstate`` only if
it works in the `active mode <Active Mode_>`_.

The following sequence of shell commands can be used to enable them and see
their output (if the kernel is generally configured to support event tracing)::

 # cd /sys/kernel/debug/tracing/
 # echo 1 > events/power/pstate_sample/enable
 # echo 1 > events/power/cpu_frequency/enable
 # cat trace
 gnome-terminal--4510  [001] ..s.  1177.680733: pstate_sample: core_busy=107 scaled=94 from=26 to=26 mperf=1143818 aperf=1230607 tsc=29838618 freq=2474476
 cat-5235  [002] ..s.  1177.681723: cpu_frequency: state=2900000 cpu_id=2

If ``intel_pstate`` works in the `passive mode <Passive Mode_>`_, the
``cpu_frequency`` trace event will be triggered either by the ``schedutil``
scaling governor (for the policies it is attached to), or by the ``CPUFreq``
core (for the policies with other scaling governors).

``ftrace``
----------

The ``ftrace`` interface can be used for low-level diagnostics of
``intel_pstate``.  For example, to check how often the function to set a
P-state is called, the ``ftrace`` filter can be set to
:c:func:`intel_pstate_set_pstate`::

 # cd /sys/kernel/debug/tracing/
 # cat available_filter_functions | grep -i pstate
 intel_pstate_set_pstate
 intel_pstate_cpu_init
 ...
 # echo intel_pstate_set_pstate > set_ftrace_filter
 # echo function > current_tracer
 # cat trace | head -15
 # tracer: function
 #
 # entries-in-buffer/entries-written: 80/80   #P:4
 #
 #                              _-----=> irqs-off
 #                             / _----=> need-resched
 #                            | / _---=> hardirq/softirq
 #                            || / _--=> preempt-depth
 #                            ||| /     delay
 #           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
 #              | |       |   ||||       |         |
             Xorg-3129  [000] ..s.  2537.644844: intel_pstate_set_pstate <-intel_pstate_timer_func
  gnome-terminal--4510  [002] ..s.  2537.649844: intel_pstate_set_pstate <-intel_pstate_timer_func
      gnome-shell-3409  [001] ..s.  2537.650850: intel_pstate_set_pstate <-intel_pstate_timer_func
           <idle>-0     [000] ..s.  2537.654843: intel_pstate_set_pstate <-intel_pstate_timer_func

References
==========

.. [1] Kristen Accardi, *Balancing Power and Performance in the Linux Kernel*,
       https://events.static.linuxfound.org/sites/events/files/slides/LinuxConEurope_2015.pdf

.. [2] *Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3: System Programming Guide*,
       https://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-software-developer-system-programming-manual-325384.html

.. [3] *Advanced Configuration and Power Interface Specification*,
       https://uefi.org/sites/default/files/resources/ACPI_6_3_final_Jan30.pdf


code:
.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>

==============================
Intel Uncore Frequency Scaling
==============================

:Copyright: |copy| 2022 Intel Corporation

:Author: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>

Introduction
------------

The uncore can consume significant amount of power in Intel's Xeon servers based
on the workload characteristics. To optimize the total power and improve overall
performance, SoCs have internal algorithms for scaling uncore frequency. These
algorithms monitor workload usage of uncore and set a desirable frequency.

It is possible that users have different expectations of uncore performance and
want to have control over it. The objective is similar to allowing users to set
the scaling min/max frequencies via cpufreq sysfs to improve CPU performance.
Users may have some latency sensitive workloads where they do not want any
change to uncore frequency. Also, users may have workloads which require
different core and uncore performance at distinct phases and they may want to
use both cpufreq and the uncore scaling interface to distribute power and
improve overall performance.

Sysfs Interface
---------------

To control uncore frequency, a sysfs interface is provided in the directory:
`/sys/devices/system/cpu/intel_uncore_frequency/`.

There is one directory for each package and die combination as the scope of
uncore scaling control is per die in multiple die/package SoCs or per
package for single die per package SoCs. The name represents the
scope of control. For example: 'package_00_die_00' is for package id 0 and
die 0.

Each package_*_die_* contains the following attributes:

``initial_max_freq_khz``
	Out of reset, this attribute represent the maximum possible frequency.
	This is a read-only attribute. If users adjust max_freq_khz,
	they can always go back to maximum using the value from this attribute.

``initial_min_freq_khz``
	Out of reset, this attribute represent the minimum possible frequency.
	This is a read-only attribute. If users adjust min_freq_khz,
	they can always go back to minimum using the value from this attribute.

``max_freq_khz``
	This attribute is used to set the maximum uncore frequency.

``min_freq_khz``
	This attribute is used to set the minimum uncore frequency.

``current_freq_khz``
	This attribute is used to get the current uncore frequency.

repo_name: angery/linux


code:
.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>

===================
System Sleep States
===================

:Copyright: |copy| 2017 Intel Corporation

:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Sleep states are global low-power states of the entire system in which user
space code cannot be executed and the overall system activity is significantly
reduced.

Sleep States That Can Be Supported
==================================

Depending on its configuration and the capabilities of the platform it runs on,
the Linux kernel can support up to four system sleep states, including
hibernation and up to three variants of system suspend.  The sleep states that
can be supported by the kernel are listed below.

.. _s2idle:

Suspend-to-Idle
---------------

This is a generic, pure software, light-weight variant of system suspend (also
referred to as S2I or S2Idle).  It allows more energy to be saved relative to
runtime idle by freezing user space, suspending the timekeeping and putting all
I/O devices into low-power states (possibly lower-power than available in the
working state), such that the processors can spend time in their deepest idle
states while the system is suspended.

The system is woken up from this state by in-band interrupts, so theoretically
any devices that can cause interrupts to be generated in the working state can
also be set up as wakeup devices for S2Idle.

This state can be used on platforms without support for :ref:`standby <standby>`
or :ref:`suspend-to-RAM <s2ram>`, or it can be used in addition to any of the
deeper system suspend variants to provide reduced resume latency.  It is always
supported if the :c:macro:`CONFIG_SUSPEND` kernel configuration option is set.

.. _standby:

Standby
-------

This state, if supported, offers moderate, but real, energy savings, while
providing a relatively straightforward transition back to the working state.  No
operating state is lost (the system core logic retains power), so the system can
go back to where it left off easily enough.

In addition to freezing user space, suspending the timekeeping and putting all
I/O devices into low-power states, which is done for :ref:`suspend-to-idle
<s2idle>` too, nonboot CPUs are taken offline and all low-level system functions
are suspended during transitions into this state.  For this reason, it should
allow more energy to be saved relative to :ref:`suspend-to-idle <s2idle>`, but
the resume latency will generally be greater than for that state.

The set of devices that can wake up the system from this state usually is
reduced relative to :ref:`suspend-to-idle <s2idle>` and it may be necessary to
rely on the platform for setting up the wakeup functionality as appropriate.

This state is supported if the :c:macro:`CONFIG_SUSPEND` kernel configuration
option is set and the support for it is registered by the platform with the
core system suspend subsystem.  On ACPI-based systems this state is mapped to
the S1 system state defined by ACPI.

.. _s2ram:

Suspend-to-RAM
--------------

This state (also referred to as STR or S2RAM), if supported, offers significant
energy savings as everything in the system is put into a low-power state, except
for memory, which should be placed into the self-refresh mode to retain its
contents.  All of the steps carried out when entering :ref:`standby <standby>`
are also carried out during transitions to S2RAM.  Additional operations may
take place depending on the platform capabilities.  In particular, on ACPI-based
systems the kernel passes control to the platform firmware (BIOS) as the last
step during S2RAM transitions and that usually results in powering down some
more low-level components that are not directly controlled by the kernel.

The state of devices and CPUs is saved and held in memory.  All devices are
suspended and put into low-power states.  In many cases, all peripheral buses
lose power when entering S2RAM, so devices must be able to handle the transition
back to the "on" state.

On ACPI-based systems S2RAM requires some minimal boot-strapping code in the
platform firmware to resume the system from it.  This may be the case on other
platforms too.

The set of devices that can wake up the system from S2RAM usually is reduced
relative to :ref:`suspend-to-idle <s2idle>` and :ref:`standby <standby>` and it
may be necessary to rely on the platform for setting up the wakeup functionality
as appropriate.

S2RAM is supported if the :c:macro:`CONFIG_SUSPEND` kernel configuration option
is set and the support for it is registered by the platform with the core system
suspend subsystem.  On ACPI-based systems it is mapped to the S3 system state
defined by ACPI.

.. _hibernation:

Hibernation
-----------

This state (also referred to as Suspend-to-Disk or STD) offers the greatest
energy savings and can be used even in the absence of low-level platform support
for system suspend.  However, it requires some low-level code for resuming the
system to be present for the underlying CPU architecture.

Hibernation is significantly different from any of the system suspend variants.
It takes three system state changes to put it into hibernation and two system
state changes to resume it.

First, when hibernation is triggered, the kernel stops all system activity and
creates a snapshot image of memory to be written into persistent storage.  Next,
the system goes into a state in which the snapshot image can be saved, the image
is written out and finally the system goes into the target low-power state in
which power is cut from almost all of its hardware components, including memory,
except for a limited set of wakeup devices.

Once the snapshot image has been written out, the system may either enter a
special low-power state (like ACPI S4), or it may simply power down itself.
Powering down means minimum power draw and it allows this mechanism to work on
any system.  However, entering a special low-power state may allow additional
means of system wakeup to be used  (e.g. pressing a key on the keyboard or
opening a laptop lid).

After wakeup, control goes to the platform firmware that runs a boot loader
which boots a fresh instance of the kernel (control may also go directly to
the boot loader, depending on the system configuration, but anyway it causes
a fresh instance of the kernel to be booted).  That new instance of the kernel
(referred to as the ``restore kernel``) looks for a hibernation image in
persistent storage and if one is found, it is loaded into memory.  Next, all
activity in the system is stopped and the restore kernel overwrites itself with
the image contents and jumps into a special trampoline area in the original
kernel stored in the image (referred to as the ``image kernel``), which is where
the special architecture-specific low-level code is needed.  Finally, the
image kernel restores the system to the pre-hibernation state and allows user
space to run again.

Hibernation is supported if the :c:macro:`CONFIG_HIBERNATION` kernel
configuration option is set.  However, this option can only be set if support
for the given CPU architecture includes the low-level code for system resume.

Basic ``sysfs`` Interfaces for System Suspend and Hibernation
=============================================================

The power management subsystem provides userspace with a unified ``sysfs``
interface for system sleep regardless of the underlying system architecture or
platform.  That interface is located in the :file:`/sys/power/` directory
(assuming that ``sysfs`` is mounted at :file:`/sys`) and it consists of the
following attributes (files):

``state``
	This file contains a list of strings representing sleep states supported
	by the kernel.  Writing one of these strings into it causes the kernel
	to start a transition of the system into the sleep state represented by
	that string.

	In particular, the "disk", "freeze" and "standby" strings represent the
	:ref:`hibernation <hibernation>`, :ref:`suspend-to-idle <s2idle>` and
	:ref:`standby <standby>` sleep states, respectively.  The "mem" string
	is interpreted in accordance with the contents of the ``mem_sleep`` file
	described below.

	If the kernel does not support any system sleep states, this file is
	not present.

``mem_sleep``
	This file contains a list of strings representing supported system
	suspend	variants and allows user space to select the variant to be
	associated with the "mem" string in the ``state`` file described above.

	The strings that may be present in this file are "s2idle", "shallow"
	and "deep".  The "s2idle" string always represents :ref:`suspend-to-idle
	<s2idle>` and, by convention, "shallow" and "deep" represent
	:ref:`standby <standby>` and :ref:`suspend-to-RAM <s2ram>`,
	respectively.

	Writing one of the listed strings into this file causes the system
	suspend variant represented by it to be associated with the "mem" string
	in the ``state`` file.  The string representing the suspend variant
	currently associated with the "mem" string in the ``state`` file is
	shown in square brackets.

	If the kernel does not support system suspend, this file is not present.

``disk``
	This file controls the operating mode of hibernation (Suspend-to-Disk).
	Specifically, it tells the kernel what to do after creating a
	hibernation image.

	Reading from it returns a list of supported options encoded as:

	``platform``
		Put the system into a special low-power state (e.g. ACPI S4) to
		make additional wakeup options available and possibly allow the
		platform firmware to take a simplified initialization path after
		wakeup.

		It is only available if the platform provides a special
		mechanism to put the system to sleep after creating a
		hibernation image (platforms with ACPI do that as a rule, for
		example).

	``shutdown``
		Power off the system.

	``reboot``
		Reboot the system (useful for diagnostics mostly).

	``suspend``
		Hybrid system suspend.  Put the system into the suspend sleep
		state selected through the ``mem_sleep`` file described above.
		If the system is successfully woken up from that state, discard
		the hibernation image and continue.  Otherwise, use the image
		to restore the previous state of the system.

		It is available if system suspend is supported.

	``test_resume``
		Diagnostic operation.  Load the image as though the system had
		just woken up from hibernation and the currently running kernel
		instance was a restore kernel and follow up with full system
		resume.

	Writing one of the strings listed above into this file causes the option
	represented by it to be selected.

	The currently selected option is shown in square brackets, which means
	that the operation represented by it will be carried out after creating
	and saving the image when hibernation is triggered by writing ``disk``
	to :file:`/sys/power/state`.

	If the kernel does not support hibernation, this file is not present.

``image_size``
	This file controls the size of hibernation images.

	It can be written a string representing a non-negative integer that will
	be used as a best-effort upper limit of the image size, in bytes.  The
	hibernation core will do its best to ensure that the image size will not
	exceed that number, but if that turns out to be impossible to achieve, a
	hibernation image will still be created and its size will be as small as
	possible.  In particular, writing '0' to this file causes the size of
	hibernation images to be minimum.

	Reading from it returns the current image size limit, which is set to
	around 2/5 of the available RAM size by default.

``pm_trace``
	This file controls the "PM trace" mechanism saving the last suspend
	or resume event point in the RTC memory across reboots.  It helps to
	debug hard lockups or reboots due to device driver failures that occur
	during system suspend or resume (which is more common) more effectively.

	If it contains "1", the fingerprint of each suspend/resume event point
	in turn will be stored in the RTC memory (overwriting the actual RTC
	information), so it will survive a system crash if one occurs right
	after storing it and it can be used later to identify the driver that
	caused the crash to happen.

	It contains "0" by default, which may be changed to "1" by writing a
	string representing a nonzero integer into it.

According to the above, there are two ways to make the system go into the
:ref:`suspend-to-idle <s2idle>` state.  The first one is to write "freeze"
directly to :file:`/sys/power/state`.  The second one is to write "s2idle" to
:file:`/sys/power/mem_sleep` and then to write "mem" to
:file:`/sys/power/state`.  Likewise, there are two ways to make the system go
into the :ref:`standby <standby>` state (the strings to write to the control
files in that case are "standby" or "shallow" and "mem", respectively) if that
state is supported by the platform.  However, there is only one way to make the
system go into the :ref:`suspend-to-RAM <s2ram>` state (write "deep" into
:file:`/sys/power/mem_sleep` and "mem" into :file:`/sys/power/state`).

The default suspend variant (ie. the one to be used without writing anything
into :file:`/sys/power/mem_sleep`) is either "deep" (on the majority of systems
supporting :ref:`suspend-to-RAM <s2ram>`) or "s2idle", but it can be overridden
by the value of the ``mem_sleep_default`` parameter in the kernel command line.
On some systems with ACPI, depending on the information in the ACPI tables, the
default may be "s2idle" even if :ref:`suspend-to-RAM <s2ram>` is supported in
principle.


code:
.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>

===========================
Power Management Strategies
===========================

:Copyright: |copy| 2017 Intel Corporation

:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

The Linux kernel supports two major high-level power management strategies.

One of them is based on using global low-power states of the whole system in
which user space code cannot be executed and the overall system activity is
significantly reduced, referred to as :doc:`sleep states <sleep-states>`.  The
kernel puts the system into one of these states when requested by user space
and the system stays in it until a special signal is received from one of
designated devices, triggering a transition to the ``working state`` in which
user space code can run.  Because sleep states are global and the whole system
is affected by the state changes, this strategy is referred to as the
:doc:`system-wide power management <system-wide>`.

The other strategy, referred to as the :doc:`working-state power management
<working-state>`, is based on adjusting the power states of individual hardware
components of the system, as needed, in the working state.  In consequence, if
this strategy is in use, the working state of the system usually does not
correspond to any particular physical configuration of it, but can be treated as
a metastate covering a range of different power states of the system in which
the individual components of it can be either ``active`` (in use) or
``inactive`` (idle).  If they are active, they have to be in power states
allowing them to process data and to be accessed by software.  In turn, if they
are inactive, ideally, they should be in low-power states in which they may not
be accessible.

If all of the system components are active, the system as a whole is regarded as
"runtime active" and that situation typically corresponds to the maximum power
draw (or maximum energy usage) of it.  If all of them are inactive, the system
as a whole is regarded as "runtime idle" which may be very close to a sleep
state from the physical system configuration and power draw perspective, but
then it takes much less time and effort to start executing user space code than
for the same system in a sleep state.  However, transitions from sleep states
back to the working state can only be started by a limited set of devices, so
typically the system can spend much more time in a sleep state than it can be
runtime idle in one go.  For this reason, systems usually use less energy in
sleep states than when they are runtime idle most of the time.

Moreover, the two power management strategies address different usage scenarios.
Namely, if the user indicates that the system will not be in use going forward,
for example by closing its lid (if the system is a laptop), it probably should
go into a sleep state at that point.  On the other hand, if the user simply goes
away from the laptop keyboard, it probably should stay in the working state and
use the working-state power management in case it becomes idle, because the user
may come back to it at any time and then may want the system to be immediately
accessible.


code:
.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>

=========================
System Suspend Code Flows
=========================

:Copyright: |copy| 2020 Intel Corporation

:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

At least one global system-wide transition needs to be carried out for the
system to get from the working state into one of the supported
:doc:`sleep states <sleep-states>`.  Hibernation requires more than one
transition to occur for this purpose, but the other sleep states, commonly
referred to as *system-wide suspend* (or simply *system suspend*) states, need
only one.

For those sleep states, the transition from the working state of the system into
the target sleep state is referred to as *system suspend* too (in the majority
of cases, whether this means a transition or a sleep state of the system should
be clear from the context) and the transition back from the sleep state into the
working state is referred to as *system resume*.

The kernel code flows associated with the suspend and resume transitions for
different sleep states of the system are quite similar, but there are some
significant differences between the :ref:`suspend-to-idle <s2idle>` code flows
and the code flows related to the :ref:`suspend-to-RAM <s2ram>` and
:ref:`standby <standby>` sleep states.

The :ref:`suspend-to-RAM <s2ram>` and :ref:`standby <standby>` sleep states
cannot be implemented without platform support and the difference between them
boils down to the platform-specific actions carried out by the suspend and
resume hooks that need to be provided by the platform driver to make them
available.  Apart from that, the suspend and resume code flows for these sleep
states are mostly identical, so they both together will be referred to as
*platform-dependent suspend* states in what follows.

.. _s2idle_suspend:

Suspend-to-idle Suspend Code Flow
=================================

The following steps are taken in order to transition the system from the working
state to the :ref:`suspend-to-idle <s2idle>` sleep state:

 1. Invoking system-wide suspend notifiers.

    Kernel subsystems can register callbacks to be invoked when the suspend
    transition is about to occur and when the resume transition has finished.

    That allows them to prepare for the change of the system state and to clean
    up after getting back to the working state.

 2. Freezing tasks.

    Tasks are frozen primarily in order to avoid unchecked hardware accesses
    from user space through MMIO regions or I/O registers exposed directly to
    it and to prevent user space from entering the kernel while the next step
    of the transition is in progress (which might have been problematic for
    various reasons).

    All user space tasks are intercepted as though they were sent a signal and
    put into uninterruptible sleep until the end of the subsequent system resume
    transition.

    The kernel threads that choose to be frozen during system suspend for
    specific reasons are frozen subsequently, but they are not intercepted.
    Instead, they are expected to periodically check whether or not they need
    to be frozen and to put themselves into uninterruptible sleep if so.  [Note,
    however, that kernel threads can use locking and other concurrency controls
    available in kernel space to synchronize themselves with system suspend and
    resume, which can be much more precise than the freezing, so the latter is
    not a recommended option for kernel threads.]

 3. Suspending devices and reconfiguring IRQs.

    Devices are suspended in four phases called *prepare*, *suspend*,
    *late suspend* and *noirq suspend* (see :ref:`driverapi_pm_devices` for more
    information on what exactly happens in each phase).

    Every device is visited in each phase, but typically it is not physically
    accessed in more than two of them.

    The runtime PM API is disabled for every device during the *late* suspend
    phase and high-level ("action") interrupt handlers are prevented from being
    invoked before the *noirq* suspend phase.

    Interrupts are still handled after that, but they are only acknowledged to
    interrupt controllers without performing any device-specific actions that
    would be triggered in the working state of the system (those actions are
    deferred till the subsequent system resume transition as described
    `below <s2idle_resume_>`_).

    IRQs associated with system wakeup devices are "armed" so that the resume
    transition of the system is started when one of them signals an event.

 4. Freezing the scheduler tick and suspending timekeeping.

    When all devices have been suspended, CPUs enter the idle loop and are put
    into the deepest available idle state.  While doing that, each of them
    "freezes" its own scheduler tick so that the timer events associated with
    the tick do not occur until the CPU is woken up by another interrupt source.

    The last CPU to enter the idle state also stops the timekeeping which
    (among other things) prevents high resolution timers from triggering going
    forward until the first CPU that is woken up restarts the timekeeping.
    That allows the CPUs to stay in the deep idle state relatively long in one
    go.

    From this point on, the CPUs can only be woken up by non-timer hardware
    interrupts.  If that happens, they go back to the idle state unless the
    interrupt that woke up one of them comes from an IRQ that has been armed for
    system wakeup, in which case the system resume transition is started.

.. _s2idle_resume:

Suspend-to-idle Resume Code Flow
================================

The following steps are taken in order to transition the system from the
:ref:`suspend-to-idle <s2idle>` sleep state into the working state:

 1. Resuming timekeeping and unfreezing the scheduler tick.

    When one of the CPUs is woken up (by a non-timer hardware interrupt), it
    leaves the idle state entered in the last step of the preceding suspend
    transition, restarts the timekeeping (unless it has been restarted already
    by another CPU that woke up earlier) and the scheduler tick on that CPU is
    unfrozen.

    If the interrupt that has woken up the CPU was armed for system wakeup,
    the system resume transition begins.

 2. Resuming devices and restoring the working-state configuration of IRQs.

    Devices are resumed in four phases called *noirq resume*, *early resume*,
    *resume* and *complete* (see :ref:`driverapi_pm_devices` for more
    information on what exactly happens in each phase).

    Every device is visited in each phase, but typically it is not physically
    accessed in more than two of them.

    The working-state configuration of IRQs is restored after the *noirq* resume
    phase and the runtime PM API is re-enabled for every device whose driver
    supports it during the *early* resume phase.

 3. Thawing tasks.

    Tasks frozen in step 2 of the preceding `suspend <s2idle_suspend_>`_
    transition are "thawed", which means that they are woken up from the
    uninterruptible sleep that they went into at that time and user space tasks
    are allowed to exit the kernel.

 4. Invoking system-wide resume notifiers.

    This is analogous to step 1 of the `suspend <s2idle_suspend_>`_ transition
    and the same set of callbacks is invoked at this point, but a different
    "notification type" parameter value is passed to them.

Platform-dependent Suspend Code Flow
====================================

The following steps are taken in order to transition the system from the working
state to platform-dependent suspend state:

 1. Invoking system-wide suspend notifiers.

    This step is the same as step 1 of the suspend-to-idle suspend transition
    described `above <s2idle_suspend_>`_.

 2. Freezing tasks.

    This step is the same as step 2 of the suspend-to-idle suspend transition
    described `above <s2idle_suspend_>`_.

 3. Suspending devices and reconfiguring IRQs.

    This step is analogous to step 3 of the suspend-to-idle suspend transition
    described `above <s2idle_suspend_>`_, but the arming of IRQs for system
    wakeup generally does not have any effect on the platform.

    There are platforms that can go into a very deep low-power state internally
    when all CPUs in them are in sufficiently deep idle states and all I/O
    devices have been put into low-power states.  On those platforms,
    suspend-to-idle can reduce system power very effectively.

    On the other platforms, however, low-level components (like interrupt
    controllers) need to be turned off in a platform-specific way (implemented
    in the hooks provided by the platform driver) to achieve comparable power
    reduction.

    That usually prevents in-band hardware interrupts from waking up the system,
    which must be done in a special platform-dependent way.  Then, the
    configuration of system wakeup sources usually starts when system wakeup
    devices are suspended and is finalized by the platform suspend hooks later
    on.

 4. Disabling non-boot CPUs.

    On some platforms the suspend hooks mentioned above must run in a one-CPU
    configuration of the system (in particular, the hardware cannot be accessed
    by any code running in parallel with the platform suspend hooks that may,
    and often do, trap into the platform firmware in order to finalize the
    suspend transition).

    For this reason, the CPU offline/online (CPU hotplug) framework is used
    to take all of the CPUs in the system, except for one (the boot CPU),
    offline (typically, the CPUs that have been taken offline go into deep idle
    states).

    This means that all tasks are migrated away from those CPUs and all IRQs are
    rerouted to the only CPU that remains online.

 5. Suspending core system components.

    This prepares the core system components for (possibly) losing power going
    forward and suspends the timekeeping.

 6. Platform-specific power removal.

    This is expected to remove power from all of the system components except
    for the memory controller and RAM (in order to preserve the contents of the
    latter) and some devices designated for system wakeup.

    In many cases control is passed to the platform firmware which is expected
    to finalize the suspend transition as needed.

Platform-dependent Resume Code Flow
===================================

The following steps are taken in order to transition the system from a
platform-dependent suspend state into the working state:

 1. Platform-specific system wakeup.

    The platform is woken up by a signal from one of the designated system
    wakeup devices (which need not be an in-band hardware interrupt)  and
    control is passed back to the kernel (the working configuration of the
    platform may need to be restored by the platform firmware before the
    kernel gets control again).

 2. Resuming core system components.

    The suspend-time configuration of the core system components is restored and
    the timekeeping is resumed.

 3. Re-enabling non-boot CPUs.

    The CPUs disabled in step 4 of the preceding suspend transition are taken
    back online and their suspend-time configuration is restored.

 4. Resuming devices and restoring the working-state configuration of IRQs.

    This step is the same as step 2 of the suspend-to-idle suspend transition
    described `above <s2idle_resume_>`_.

 5. Thawing tasks.

    This step is the same as step 3 of the suspend-to-idle suspend transition
    described `above <s2idle_resume_>`_.

 6. Invoking system-wide resume notifiers.

    This step is the same as step 4 of the suspend-to-idle suspend transition
    described `above <s2idle_resume_>`_.


code:
.. SPDX-License-Identifier: GPL-2.0

============================
System-Wide Power Management
============================

.. toctree::
   :maxdepth: 2

   sleep-states
   suspend-flows


code:
.. SPDX-License-Identifier: GPL-2.0

==============================
Working-State Power Management
==============================

.. toctree::
   :maxdepth: 2

   cpuidle
   intel_idle
   cpufreq
   intel_pstate
   amd-pstate
   cpufreq_drivers
   intel_epb
   intel-speed-select
   intel_uncore_frequency_scaling


code:
=================================
Linux Plug and Play Documentation
=================================

:Author: Adam Belay <ambx1@neo.rr.com>
:Last updated: Oct. 16, 2002

Overview
--------

Plug and Play provides a means of detecting and setting resources for legacy or
otherwise unconfigurable devices.  The Linux Plug and Play Layer provides these
services to compatible drivers.

The User Interface
------------------

The Linux Plug and Play user interface provides a means to activate PnP devices
for legacy and user level drivers that do not support Linux Plug and Play.  The
user interface is integrated into sysfs.

In addition to the standard sysfs file the following are created in each
device's directory:
- id - displays a list of support EISA IDs
- options - displays possible resource configurations
- resources - displays currently allocated resources and allows resource changes

activating a device
^^^^^^^^^^^^^^^^^^^

::

	# echo "auto" > resources

this will invoke the automatic resource config system to activate the device

manually activating a device
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

::

	# echo "manual <depnum> <mode>" > resources

	<depnum> - the configuration number
	<mode> - static or dynamic
		 static = for next boot
		 dynamic = now

disabling a device
^^^^^^^^^^^^^^^^^^

::

	# echo "disable" > resources

EXAMPLE:

Suppose you need to activate the floppy disk controller.

1. change to the proper directory, in my case it is
   /driver/bus/pnp/devices/00:0f::

	# cd /driver/bus/pnp/devices/00:0f
	# cat name
	PC standard floppy disk controller

2. check if the device is already active::

	# cat resources
	DISABLED

  - Notice the string "DISABLED".  This means the device is not active.

3. check the device's possible configurations (optional)::

	# cat options
	Dependent: 01 - Priority acceptable
	    port 0x3f0-0x3f0, align 0x7, size 0x6, 16-bit address decoding
	    port 0x3f7-0x3f7, align 0x0, size 0x1, 16-bit address decoding
	    irq 6
	    dma 2 8-bit compatible
	Dependent: 02 - Priority acceptable
	    port 0x370-0x370, align 0x7, size 0x6, 16-bit address decoding
	    port 0x377-0x377, align 0x0, size 0x1, 16-bit address decoding
	    irq 6
	    dma 2 8-bit compatible

4. now activate the device::

	# echo "auto" > resources

5. finally check if the device is active::

	# cat resources
	io 0x3f0-0x3f5
	io 0x3f7-0x3f7
	irq 6
	dma 2

also there are a series of kernel parameters::

	pnp_reserve_irq=irq1[,irq2] ....
	pnp_reserve_dma=dma1[,dma2] ....
	pnp_reserve_io=io1,size1[,io2,size2] ....
	pnp_reserve_mem=mem1,size1[,mem2,size2] ....

The Unified Plug and Play Layer
-------------------------------

All Plug and Play drivers, protocols, and services meet at a central location
called the Plug and Play Layer.  This layer is responsible for the exchange of
information between PnP drivers and PnP protocols.  Thus it automatically
forwards commands to the proper protocol.  This makes writing PnP drivers
significantly easier.

The following functions are available from the Plug and Play Layer:

pnp_get_protocol
  increments the number of uses by one

pnp_put_protocol
  deincrements the number of uses by one

pnp_register_protocol
  use this to register a new PnP protocol

pnp_unregister_protocol
  use this function to remove a PnP protocol from the Plug and Play Layer

pnp_register_driver
  adds a PnP driver to the Plug and Play Layer

  this includes driver model integration
  returns zero for success or a negative error number for failure; count
  calls to the .add() method if you need to know how many devices bind to
  the driver

pnp_unregister_driver
  removes a PnP driver from the Plug and Play Layer

Plug and Play Protocols
-----------------------

This section contains information for PnP protocol developers.

The following Protocols are currently available in the computing world:

- PNPBIOS:
    used for system devices such as serial and parallel ports.
- ISAPNP:
    provides PnP support for the ISA bus
- ACPI:
    among its many uses, ACPI provides information about system level
    devices.

It is meant to replace the PNPBIOS.  It is not currently supported by Linux
Plug and Play but it is planned to be in the near future.

Requirements for a Linux PnP protocol:
1. the protocol must use EISA IDs
2. the protocol must inform the PnP Layer of a device's current configuration

- the ability to set resources is optional but preferred.

The following are PnP protocol related functions:

pnp_add_device
  use this function to add a PnP device to the PnP layer

  only call this function when all wanted values are set in the pnp_dev
  structure

pnp_init_device
  call this to initialize the PnP structure

pnp_remove_device
  call this to remove a device from the Plug and Play Layer.
  it will fail if the device is still in use.
  automatically will free mem used by the device and related structures

pnp_add_id
  adds an EISA ID to the list of supported IDs for the specified device

For more information consult the source of a protocol such as
/drivers/pnp/pnpbios/core.c.

Linux Plug and Play Drivers
---------------------------

This section contains information for Linux PnP driver developers.

The New Way
^^^^^^^^^^^

1. first make a list of supported EISA IDS

   ex::

	static const struct pnp_id pnp_dev_table[] = {
		/* Standard LPT Printer Port */
		{.id = "PNP0400", .driver_data = 0},
		/* ECP Printer Port */
		{.id = "PNP0401", .driver_data = 0},
		{.id = ""}
	};

   Please note that the character 'X' can be used as a wild card in the function
   portion (last four characters).

   ex::

	/* Unknown PnP modems */
	{	"PNPCXXX",		UNKNOWN_DEV	},

   Supported PnP card IDs can optionally be defined.
   ex::

	static const struct pnp_id pnp_card_table[] = {
		{	"ANYDEVS",		0	},
		{	"",			0	}
	};

2. Optionally define probe and remove functions.  It may make sense not to
   define these functions if the driver already has a reliable method of detecting
   the resources, such as the parport_pc driver.

   ex::

	static int
	serial_pnp_probe(struct pnp_dev * dev, const struct pnp_id *card_id, const
			struct pnp_id *dev_id)
	{
	. . .

   ex::

	static void serial_pnp_remove(struct pnp_dev * dev)
	{
	. . .

   consult /drivers/serial/8250_pnp.c for more information.

3. create a driver structure

   ex::

	static struct pnp_driver serial_pnp_driver = {
		.name		= "serial",
		.card_id_table	= pnp_card_table,
		.id_table	= pnp_dev_table,
		.probe		= serial_pnp_probe,
		.remove		= serial_pnp_remove,
	};

   * name and id_table cannot be NULL.

4. register the driver

   ex::

	static int __init serial8250_pnp_init(void)
	{
		return pnp_register_driver(&serial_pnp_driver);
	}

The Old Way
^^^^^^^^^^^

A series of compatibility functions have been created to make it easy to convert
ISAPNP drivers.  They should serve as a temporary solution only.

They are as follows::

	struct pnp_dev *pnp_find_dev(struct pnp_card *card,
				     unsigned short vendor,
				     unsigned short function,
				     struct pnp_dev *from)


code:
.. SPDX-License-Identifier: GPL-2.0

pstore block oops/panic logger
==============================

Introduction
------------

pstore block (pstore/blk) is an oops/panic logger that writes its logs to a
block device and non-block device before the system crashes. You can get
these log files by mounting pstore filesystem like::

    mount -t pstore pstore /sys/fs/pstore

pstore block concepts
---------------------

pstore/blk provides efficient configuration method for pstore/blk, which
divides all configurations into two parts, configurations for user and
configurations for driver.

Configurations for user determine how pstore/blk works, such as pmsg_size,
kmsg_size and so on. All of them support both Kconfig and module parameters,
but module parameters have priority over Kconfig.

Configurations for driver are all about block device and non-block device,
such as total_size of block device and read/write operations.

Configurations for user
-----------------------

All of these configurations support both Kconfig and module parameters, but
module parameters have priority over Kconfig.

Here is an example for module parameters::

        pstore_blk.blkdev=/dev/mmcblk0p7 pstore_blk.kmsg_size=64 best_effort=y

The detail of each configurations may be of interest to you.

blkdev
~~~~~~

The block device to use. Most of the time, it is a partition of block device.
It's required for pstore/blk. It is also used for MTD device.

When pstore/blk is built as a module, "blkdev" accepts the following variants:

1. /dev/<disk_name> represents the device number of disk
#. /dev/<disk_name><decimal> represents the device number of partition - device
   number of disk plus the partition number
#. /dev/<disk_name>p<decimal> - same as the above; this form is used when disk
   name of partitioned disk ends with a digit.

When pstore/blk is built into the kernel, "blkdev" accepts the following variants:

#. <hex_major><hex_minor> device number in hexadecimal representation,
   with no leading 0x, for example b302.
#. PARTUUID=00112233-4455-6677-8899-AABBCCDDEEFF represents the unique id of
   a partition if the partition table provides it. The UUID may be either an
   EFI/GPT UUID, or refer to an MSDOS partition using the format SSSSSSSS-PP,
   where SSSSSSSS is a zero-filled hex representation of the 32-bit
   "NT disk signature", and PP is a zero-filled hex representation of the
   1-based partition number.
#. PARTUUID=<UUID>/PARTNROFF=<int> to select a partition in relation to a
   partition with a known unique id.
#. <major>:<minor> major and minor number of the device separated by a colon.

It accepts the following variants for MTD device:

1. <device name> MTD device name. "pstore" is recommended.
#. <device number> MTD device number.

kmsg_size
~~~~~~~~~

The chunk size in KB for oops/panic front-end. It **MUST** be a multiple of 4.
It's optional if you do not care oops/panic log.

There are multiple chunks for oops/panic front-end depending on the remaining
space except other pstore front-ends.

pstore/blk will log to oops/panic chunks one by one, and always overwrite the
oldest chunk if there is no more free chunk.

pmsg_size
~~~~~~~~~

The chunk size in KB for pmsg front-end. It **MUST** be a multiple of 4.
It's optional if you do not care pmsg log.

Unlike oops/panic front-end, there is only one chunk for pmsg front-end.

Pmsg is a user space accessible pstore object. Writes to */dev/pmsg0* are
appended to the chunk. On reboot the contents are available in
*/sys/fs/pstore/pmsg-pstore-blk-0*.

console_size
~~~~~~~~~~~~

The chunk size in KB for console front-end.  It **MUST** be a multiple of 4.
It's optional if you do not care console log.

Similar to pmsg front-end, there is only one chunk for console front-end.

All log of console will be appended to the chunk. On reboot the contents are
available in */sys/fs/pstore/console-pstore-blk-0*.

ftrace_size
~~~~~~~~~~~

The chunk size in KB for ftrace front-end. It **MUST** be a multiple of 4.
It's optional if you do not care console log.

Similar to oops front-end, there are multiple chunks for ftrace front-end
depending on the count of cpu processors. Each chunk size is equal to
ftrace_size / processors_count.

All log of ftrace will be appended to the chunk. On reboot the contents are
combined and available in */sys/fs/pstore/ftrace-pstore-blk-0*.

Persistent function tracing might be useful for debugging software or hardware
related hangs. Here is an example of usage::

 # mount -t pstore pstore /sys/fs/pstore
 # mount -t debugfs debugfs /sys/kernel/debug/
 # echo 1 > /sys/kernel/debug/pstore/record_ftrace
 # reboot -f
 [...]
 # mount -t pstore pstore /sys/fs/pstore
 # tail /sys/fs/pstore/ftrace-pstore-blk-0
 CPU:0 ts:5914676 c0063828  c0063b94  call_cpuidle <- cpu_startup_entry+0x1b8/0x1e0
 CPU:0 ts:5914678 c039ecdc  c006385c  cpuidle_enter_state <- call_cpuidle+0x44/0x48
 CPU:0 ts:5914680 c039e9a0  c039ecf0  cpuidle_enter_freeze <- cpuidle_enter_state+0x304/0x314
 CPU:0 ts:5914681 c0063870  c039ea30  sched_idle_set_state <- cpuidle_enter_state+0x44/0x314
 CPU:1 ts:5916720 c0160f59  c015ee04  kernfs_unmap_bin_file <- __kernfs_remove+0x140/0x204
 CPU:1 ts:5916721 c05ca625  c015ee0c  __mutex_lock_slowpath <- __kernfs_remove+0x148/0x204
 CPU:1 ts:5916723 c05c813d  c05ca630  yield_to <- __mutex_lock_slowpath+0x314/0x358
 CPU:1 ts:5916724 c05ca2d1  c05ca638  __ww_mutex_lock <- __mutex_lock_slowpath+0x31c/0x358

max_reason
~~~~~~~~~~

Limiting which kinds of kmsg dumps are stored can be controlled via
the ``max_reason`` value, as defined in include/linux/kmsg_dump.h's
``enum kmsg_dump_reason``. For example, to store both Oopses and Panics,
``max_reason`` should be set to 2 (KMSG_DUMP_OOPS), to store only Panics
``max_reason`` should be set to 1 (KMSG_DUMP_PANIC). Setting this to 0
(KMSG_DUMP_UNDEF), means the reason filtering will be controlled by the
``printk.always_kmsg_dump`` boot param: if unset, it'll be KMSG_DUMP_OOPS,
otherwise KMSG_DUMP_MAX.

Configurations for driver
-------------------------

A device driver uses ``register_pstore_device`` with
``struct pstore_device_info`` to register to pstore/blk.

.. kernel-doc:: fs/pstore/blk.c
   :export:

Compression and header
----------------------

Block device is large enough for uncompressed oops data. Actually we do not
recommend data compression because pstore/blk will insert some information into
the first line of oops/panic data. For example::

        Panic: Total 16 times

It means that it's OOPS|Panic for the 16th time since the first booting.
Sometimes the number of occurrences of oops|panic since the first booting is
important to judge whether the system is stable.

The following line is inserted by pstore filesystem. For example::

        Oops#2 Part1

It means that it's OOPS for the 2nd time on the last boot.

Reading the data
----------------

The dump data can be read from the pstore filesystem. The format for these
files is ``dmesg-pstore-blk-[N]`` for oops/panic front-end,
``pmsg-pstore-blk-0`` for pmsg front-end and so on.  The timestamp of the
dump file records the trigger time. To delete a stored record from block
device, simply unlink the respective pstore file.

Attentions in panic read/write APIs
-----------------------------------

If on panic, the kernel is not going to run for much longer, the tasks will not
be scheduled and most kernel resources will be out of service. It
looks like a single-threaded program running on a single-core computer.

The following points require special attention for panic read/write APIs:

1. Can **NOT** allocate any memory.
   If you need memory, just allocate while the block driver is initializing
   rather than waiting until the panic.
#. Must be polled, **NOT** interrupt driven.
   No task schedule any more. The block driver should delay to ensure the write
   succeeds, but NOT sleep.
#. Can **NOT** take any lock.
   There is no other task, nor any shared resource; you are safe to break all
   locks.
#. Just use CPU to transfer.
   Do not use DMA to transfer unless you are sure that DMA will not keep lock.
#. Control registers directly.
   Please control registers directly rather than use Linux kernel resources.
   Do I/O map while initializing rather than wait until a panic occurs.
#. Reset your block device and controller if necessary.
   If you are not sure of the state of your block device and controller when
   a panic occurs, you are safe to stop and reset them.

pstore/blk supports psblk_blkdev_info(), which is defined in
*linux/pstore_blk.h*, to get information of using block device, such as the
device number, sector count and start sector of the whole disk.

pstore block internals
----------------------

For developer reference, here are all the important structures and APIs:

.. kernel-doc:: fs/pstore/zone.c
   :internal:

.. kernel-doc:: include/linux/pstore_zone.h
   :internal:

.. kernel-doc:: include/linux/pstore_blk.h
   :internal:


code:
Ramoops oops/panic logger
=========================

Sergiu Iordache <sergiu@chromium.org>

Updated: 10 Feb 2021

Introduction
------------

Ramoops is an oops/panic logger that writes its logs to RAM before the system
crashes. It works by logging oopses and panics in a circular buffer. Ramoops
needs a system with persistent RAM so that the content of that area can
survive after a restart.

Ramoops concepts
----------------

Ramoops uses a predefined memory area to store the dump. The start and size
and type of the memory area are set using three variables:

  * ``mem_address`` for the start
  * ``mem_size`` for the size. The memory size will be rounded down to a
    power of two.
  * ``mem_type`` to specify if the memory type (default is pgprot_writecombine).

Typically the default value of ``mem_type=0`` should be used as that sets the pstore
mapping to pgprot_writecombine. Setting ``mem_type=1`` attempts to use
``pgprot_noncached``, which only works on some platforms. This is because pstore
depends on atomic operations. At least on ARM, pgprot_noncached causes the
memory to be mapped strongly ordered, and atomic operations on strongly ordered
memory are implementation defined, and won't work on many ARMs such as omaps.
Setting ``mem_type=2`` attempts to treat the memory region as normal memory,
which enables full cache on it. This can improve the performance.

The memory area is divided into ``record_size`` chunks (also rounded down to
power of two) and each kmesg dump writes a ``record_size`` chunk of
information.

Limiting which kinds of kmsg dumps are stored can be controlled via
the ``max_reason`` value, as defined in include/linux/kmsg_dump.h's
``enum kmsg_dump_reason``. For example, to store both Oopses and Panics,
``max_reason`` should be set to 2 (KMSG_DUMP_OOPS), to store only Panics
``max_reason`` should be set to 1 (KMSG_DUMP_PANIC). Setting this to 0
(KMSG_DUMP_UNDEF), means the reason filtering will be controlled by the
``printk.always_kmsg_dump`` boot param: if unset, it'll be KMSG_DUMP_OOPS,
otherwise KMSG_DUMP_MAX.

The module uses a counter to record multiple dumps but the counter gets reset
on restart (i.e. new dumps after the restart will overwrite old ones).

Ramoops also supports software ECC protection of persistent memory regions.
This might be useful when a hardware reset was used to bring the machine back
to life (i.e. a watchdog triggered). In such cases, RAM may be somewhat
corrupt, but usually it is restorable.

Setting the parameters
----------------------

Setting the ramoops parameters can be done in several different manners:

 A. Use the module parameters (which have the names of the variables described
 as before). For quick debugging, you can also reserve parts of memory during
 boot and then use the reserved memory for ramoops. For example, assuming a
 machine with > 128 MB of memory, the following kernel command line will tell
 the kernel to use only the first 128 MB of memory, and place ECC-protected
 ramoops region at 128 MB boundary::

	mem=128M ramoops.mem_address=0x8000000 ramoops.ecc=1

 B. Use Device Tree bindings, as described in
 ``Documentation/devicetree/bindings/reserved-memory/ramoops.yaml``.
 For example::

	reserved-memory {
		#address-cells = <2>;
		#size-cells = <2>;
		ranges;

		ramoops@8f000000 {
			compatible = "ramoops";
			reg = <0 0x8f000000 0 0x100000>;
			record-size = <0x4000>;
			console-size = <0x4000>;
		};
	};

 C. Use a platform device and set the platform data. The parameters can then
 be set through that platform data. An example of doing that is:

 .. code-block:: c

  #include <linux/pstore_ram.h>
  [...]

  static struct ramoops_platform_data ramoops_data = {
        .mem_size               = <...>,
        .mem_address            = <...>,
        .mem_type               = <...>,
        .record_size            = <...>,
        .max_reason             = <...>,
        .ecc                    = <...>,
  };

  static struct platform_device ramoops_dev = {
        .name = "ramoops",
        .dev = {
                .platform_data = &ramoops_data,
        },
  };

  [... inside a function ...]
  int ret;

  ret = platform_device_register(&ramoops_dev);
  if (ret) {
	printk(KERN_ERR "unable to register platform device\n");
	return ret;
  }

You can specify either RAM memory or peripheral devices' memory. However, when
specifying RAM, be sure to reserve the memory by issuing memblock_reserve()
very early in the architecture code, e.g.::

	#include <linux/memblock.h>

	memblock_reserve(ramoops_data.mem_address, ramoops_data.mem_size);

Dump format
-----------

The data dump begins with a header, currently defined as ``====`` followed by a
timestamp and a new line. The dump then continues with the actual data.

Reading the data
----------------

The dump data can be read from the pstore filesystem. The format for these
files is ``dmesg-ramoops-N``, where N is the record number in memory. To delete
a stored record from RAM, simply unlink the respective pstore file.

Persistent function tracing
---------------------------

Persistent function tracing might be useful for debugging software or hardware
related hangs. The functions call chain log is stored in a ``ftrace-ramoops``
file. Here is an example of usage::

 # mount -t debugfs debugfs /sys/kernel/debug/
 # echo 1 > /sys/kernel/debug/pstore/record_ftrace
 # reboot -f
 [...]
 # mount -t pstore pstore /mnt/
 # tail /mnt/ftrace-ramoops
 0 ffffffff8101ea64  ffffffff8101bcda  native_apic_mem_read <- disconnect_bsp_APIC+0x6a/0xc0
 0 ffffffff8101ea44  ffffffff8101bcf6  native_apic_mem_write <- disconnect_bsp_APIC+0x86/0xc0
 0 ffffffff81020084  ffffffff8101a4b5  hpet_disable <- native_machine_shutdown+0x75/0x90
 0 ffffffff81005f94  ffffffff8101a4bb  iommu_shutdown_noop <- native_machine_shutdown+0x7b/0x90
 0 ffffffff8101a6a1  ffffffff8101a437  native_machine_emergency_restart <- native_machine_restart+0x37/0x40
 0 ffffffff811f9876  ffffffff8101a73a  acpi_reboot <- native_machine_emergency_restart+0xaa/0x1e0
 0 ffffffff8101a514  ffffffff8101a772  mach_reboot_fixups <- native_machine_emergency_restart+0xe2/0x1e0
 0 ffffffff811d9c54  ffffffff8101a7a0  __const_udelay <- native_machine_emergency_restart+0x110/0x1e0
 0 ffffffff811d9c34  ffffffff811d9c80  __delay <- __const_udelay+0x30/0x40
 0 ffffffff811d9d14  ffffffff811d9c3f  delay_tsc <- __delay+0xf/0x20


code:
=======================
RapidIO Subsystem Guide
=======================

:Author: Matt Porter

Introduction
============

RapidIO is a high speed switched fabric interconnect with features aimed
at the embedded market. RapidIO provides support for memory-mapped I/O
as well as message-based transactions over the switched fabric network.
RapidIO has a standardized discovery mechanism not unlike the PCI bus
standard that allows simple detection of devices in a network.

This documentation is provided for developers intending to support
RapidIO on new architectures, write new drivers, or to understand the
subsystem internals.

Known Bugs and Limitations
==========================

Bugs
----

None. ;)

Limitations
-----------

1. Access/management of RapidIO memory regions is not supported

2. Multiple host enumeration is not supported

RapidIO driver interface
========================

Drivers are provided a set of calls in order to interface with the
subsystem to gather info on devices, request/map memory region
resources, and manage mailboxes/doorbells.

Functions
---------

.. kernel-doc:: include/linux/rio_drv.h
   :internal:

.. kernel-doc:: drivers/rapidio/rio-driver.c
   :export:

.. kernel-doc:: drivers/rapidio/rio.c
   :export:

Internals
=========

This chapter contains the autogenerated documentation of the RapidIO
subsystem.

Structures
----------

.. kernel-doc:: include/linux/rio.h
   :internal:

Enumeration and Discovery
-------------------------

.. kernel-doc:: drivers/rapidio/rio-scan.c
   :internal:

Driver functionality
--------------------

.. kernel-doc:: drivers/rapidio/rio.c
   :internal:

.. kernel-doc:: drivers/rapidio/rio-access.c
   :internal:

Device model support
--------------------

.. kernel-doc:: drivers/rapidio/rio-driver.c
   :internal:

PPC32 support
-------------

.. kernel-doc:: arch/powerpc/sysdev/fsl_rio.c
   :internal:

Credits
=======

The following people have contributed to the RapidIO subsystem directly
or indirectly:

1. Matt Porter\ mporter@kernel.crashing.org

2. Randy Vinson\ rvinson@mvista.com

3. Dan Malek\ dan@embeddedalley.com

The following people have contributed to this document:

1. Matt Porter\ mporter@kernel.crashing.org