code:
======================================
vlocks for Bare-Metal Mutual Exclusion
======================================

Voting Locks, or "vlocks" provide a simple low-level mutual exclusion
mechanism, with reasonable but minimal requirements on the memory
system.

These are intended to be used to coordinate critical activity among CPUs
which are otherwise non-coherent, in situations where the hardware
provides no other mechanism to support this and ordinary spinlocks
cannot be used.

vlocks make use of the atomicity provided by the memory system for
writes to a single memory location.  To arbitrate, every CPU "votes for
itself", by storing a unique number to a common memory location.  The
final value seen in that memory location when all the votes have been
cast identifies the winner.

In order to make sure that the election produces an unambiguous result
in finite time, a CPU will only enter the election in the first place if
no winner has been chosen and the election does not appear to have
started yet.

Algorithm
---------

The easiest way to explain the vlocks algorithm is with some pseudo-code::

	int currently_voting[NR_CPUS] = { 0, };
	int last_vote = -1; /* no votes yet */

	bool vlock_trylock(int this_cpu)
	{
		/* signal our desire to vote */
		currently_voting[this_cpu] = 1;
		if (last_vote != -1) {
			/* someone already volunteered himself */
			currently_voting[this_cpu] = 0;
			return false; /* not ourself */
		}

		/* let's suggest ourself */
		last_vote = this_cpu;
		currently_voting[this_cpu] = 0;

		/* then wait until everyone else is done voting */
		for_each_cpu(i) {
			while (currently_voting[i] != 0)
				/* wait */;
		}

		/* result */
		if (last_vote == this_cpu)
			return true; /* we won */
		return false;
	}

	bool vlock_unlock(void)
	{
		last_vote = -1;
	}

The currently_voting[] array provides a way for the CPUs to determine
whether an election is in progress, and plays a role analogous to the
"entering" array in Lamport's bakery algorithm [1].

However, once the election has started, the underlying memory system
atomicity is used to pick the winner.  This avoids the need for a static
priority rule to act as a tie-breaker, or any counters which could
overflow.

As long as the last_vote variable is globally visible to all CPUs, it
will contain only one value that won't change once every CPU has cleared
its currently_voting flag.

Features and limitations
------------------------

 * vlocks are not intended to be fair.  In the contended case, it is the
   _last_ CPU which attempts to get the lock which will be most likely
   to win.

   vlocks are therefore best suited to situations where it is necessary
   to pick a unique winner, but it does not matter which CPU actually
   wins.

 * Like other similar mechanisms, vlocks will not scale well to a large
   number of CPUs.

   vlocks can be cascaded in a voting hierarchy to permit better scaling
   if necessary, as in the following hypothetical example for 4096 CPUs::

	/* first level: local election */
	my_town = towns[(this_cpu >> 4) & 0xf];
	I_won = vlock_trylock(my_town, this_cpu & 0xf);
	if (I_won) {
		/* we won the town election, let's go for the state */
		my_state = states[(this_cpu >> 8) & 0xf];
		I_won = vlock_lock(my_state, this_cpu & 0xf));
		if (I_won) {
			/* and so on */
			I_won = vlock_lock(the_whole_country, this_cpu & 0xf];
			if (I_won) {
				/* ... */
			}
			vlock_unlock(the_whole_country);
		}
		vlock_unlock(my_state);
	}
	vlock_unlock(my_town);

ARM implementation
------------------

The current ARM implementation [2] contains some optimisations beyond
the basic algorithm:

 * By packing the members of the currently_voting array close together,
   we can read the whole array in one transaction (providing the number
   of CPUs potentially contending the lock is small enough).  This
   reduces the number of round-trips required to external memory.

   In the ARM implementation, this means that we can use a single load
   and comparison::

	LDR	Rt, [Rn]
	CMP	Rt, #0

   ...in place of code equivalent to::

	LDRB	Rt, [Rn]
	CMP	Rt, #0
	LDRBEQ	Rt, [Rn, #1]
	CMPEQ	Rt, #0
	LDRBEQ	Rt, [Rn, #2]
	CMPEQ	Rt, #0
	LDRBEQ	Rt, [Rn, #3]
	CMPEQ	Rt, #0

   This cuts down on the fast-path latency, as well as potentially
   reducing bus contention in contended cases.

   The optimisation relies on the fact that the ARM memory system
   guarantees coherency between overlapping memory accesses of
   different sizes, similarly to many other architectures.  Note that
   we do not care which element of currently_voting appears in which
   bits of Rt, so there is no need to worry about endianness in this
   optimisation.

   If there are too many CPUs to read the currently_voting array in
   one transaction then multiple transations are still required.  The
   implementation uses a simple loop of word-sized loads for this
   case.  The number of transactions is still fewer than would be
   required if bytes were loaded individually.

   In principle, we could aggregate further by using LDRD or LDM, but
   to keep the code simple this was not attempted in the initial
   implementation.

 * vlocks are currently only used to coordinate between CPUs which are
   unable to enable their caches yet.  This means that the
   implementation removes many of the barriers which would be required
   when executing the algorithm in cached memory.

   packing of the currently_voting array does not work with cached
   memory unless all CPUs contending the lock are cache-coherent, due
   to cache writebacks from one CPU clobbering values written by other
   CPUs.  (Though if all the CPUs are cache-coherent, you should be
   probably be using proper spinlocks instead anyway).

 * The "no votes yet" value used for the last_vote variable is 0 (not
   -1 as in the pseudocode).  This allows statically-allocated vlocks
   to be implicitly initialised to an unlocked state simply by putting
   them in .bss.

   An offset is added to each CPU's ID for the purpose of setting this
   variable, so that no CPU uses the value 0 for its ID.

Colophon
--------

Originally created and documented by Dave Martin for Linaro Limited, for
use in ARM-based big.LITTLE platforms, with review and input gratefully
received from Nicolas Pitre and Achin Gupta.  Thanks to Nicolas for
grabbing most of this text out of the relevant mail thread and writing
up the pseudocode.

Copyright (C) 2012-2013  Linaro Limited
Distributed under the terms of Version 2 of the GNU General Public
License, as defined in linux/COPYING.

References
----------

[1] Lamport, L. "A New Solution of Dijkstra's Concurrent Programming
    Problem", Communications of the ACM 17, 8 (August 1974), 453-455.

    https://en.wikipedia.org/wiki/Lamport%27s_bakery_algorithm

[2] linux/arch/arm/common/vlock.S, www.kernel.org.


code:
===========
ACPI Tables
===========

The expectations of individual ACPI tables are discussed in the list that
follows.

If a section number is used, it refers to a section number in the ACPI
specification where the object is defined.  If "Signature Reserved" is used,
the table signature (the first four bytes of the table) is the only portion
of the table recognized by the specification, and the actual table is defined
outside of the UEFI Forum (see Section 5.2.6 of the specification).

For ACPI on arm64, tables also fall into the following categories:

       -  Required: DSDT, FADT, GTDT, MADT, MCFG, RSDP, SPCR, XSDT

       -  Recommended: BERT, EINJ, ERST, HEST, PCCT, SSDT

       -  Optional: BGRT, CPEP, CSRT, DBG2, DRTM, ECDT, FACS, FPDT, IBFT,
          IORT, MCHI, MPST, MSCT, NFIT, PMTT, RASF, SBST, SLIT, SPMI, SRAT,
          STAO, TCPA, TPM2, UEFI, XENV

       -  Not supported: BOOT, DBGP, DMAR, ETDT, HPET, IVRS, LPIT, MSDM, OEMx,
          PSDT, RSDT, SLIC, WAET, WDAT, WDRT, WPBT

====== ========================================================================
Table  Usage for ARMv8 Linux
====== ========================================================================
BERT   Section 18.3 (signature == "BERT")

       **Boot Error Record Table**

       Must be supplied if RAS support is provided by the platform.  It
       is recommended this table be supplied.

BOOT   Signature Reserved (signature == "BOOT")

       **simple BOOT flag table**

       Microsoft only table, will not be supported.

BGRT   Section 5.2.22 (signature == "BGRT")

       **Boot Graphics Resource Table**

       Optional, not currently supported, with no real use-case for an
       ARM server.

CPEP   Section 5.2.18 (signature == "CPEP")

       **Corrected Platform Error Polling table**

       Optional, not currently supported, and not recommended until such
       time as ARM-compatible hardware is available, and the specification
       suitably modified.

CSRT   Signature Reserved (signature == "CSRT")

       **Core System Resources Table**

       Optional, not currently supported.

DBG2   Signature Reserved (signature == "DBG2")

       **DeBuG port table 2**

       License has changed and should be usable.  Optional if used instead
       of earlycon=<device> on the command line.

DBGP   Signature Reserved (signature == "DBGP")

       **DeBuG Port table**

       Microsoft only table, will not be supported.

DSDT   Section 5.2.11.1 (signature == "DSDT")

       **Differentiated System Description Table**

       A DSDT is required; see also SSDT.

       ACPI tables contain only one DSDT but can contain one or more SSDTs,
       which are optional.  Each SSDT can only add to the ACPI namespace,
       but cannot modify or replace anything in the DSDT.

DMAR   Signature Reserved (signature == "DMAR")

       **DMA Remapping table**

       x86 only table, will not be supported.

DRTM   Signature Reserved (signature == "DRTM")

       **Dynamic Root of Trust for Measurement table**

       Optional, not currently supported.

ECDT   Section 5.2.16 (signature == "ECDT")

       **Embedded Controller Description Table**

       Optional, not currently supported, but could be used on ARM if and
       only if one uses the GPE_BIT field to represent an IRQ number, since
       there are no GPE blocks defined in hardware reduced mode.  This would
       need to be modified in the ACPI specification.

EINJ   Section 18.6 (signature == "EINJ")

       **Error Injection table**

       This table is very useful for testing platform response to error
       conditions; it allows one to inject an error into the system as
       if it had actually occurred.  However, this table should not be
       shipped with a production system; it should be dynamically loaded
       and executed with the ACPICA tools only during testing.

ERST   Section 18.5 (signature == "ERST")

       **Error Record Serialization Table**

       On a platform supports RAS, this table must be supplied if it is not
       UEFI-based; if it is UEFI-based, this table may be supplied. When this
       table is not present, UEFI run time service will be utilized to save
       and retrieve hardware error information to and from a persistent store.

ETDT   Signature Reserved (signature == "ETDT")

       **Event Timer Description Table**

       Obsolete table, will not be supported.

FACS   Section 5.2.10 (signature == "FACS")

       **Firmware ACPI Control Structure**

       It is unlikely that this table will be terribly useful.  If it is
       provided, the Global Lock will NOT be used since it is not part of
       the hardware reduced profile, and only 64-bit address fields will
       be considered valid.

FADT   Section 5.2.9 (signature == "FACP")

       **Fixed ACPI Description Table**
       Required for arm64.

       The HW_REDUCED_ACPI flag must be set.  All of the fields that are
       to be ignored when HW_REDUCED_ACPI is set are expected to be set to
       zero.

       If an FACS table is provided, the X_FIRMWARE_CTRL field is to be
       used, not FIRMWARE_CTRL.

       If PSCI is used (as is recommended), make sure that ARM_BOOT_ARCH is
       filled in properly - that the PSCI_COMPLIANT flag is set and that
       PSCI_USE_HVC is set or unset as needed (see table 5-37).

       For the DSDT that is also required, the X_DSDT field is to be used,
       not the DSDT field.

FPDT   Section 5.2.23 (signature == "FPDT")

       **Firmware Performance Data Table**

       Optional, useful for boot performance profiling.

GTDT   Section 5.2.24 (signature == "GTDT")

       **Generic Timer Description Table**

       Required for arm64.

HEST   Section 18.3.2 (signature == "HEST")

       **Hardware Error Source Table**

       ARM-specific error sources have been defined; please use those or the
       PCI types such as type 6 (AER Root Port), 7 (AER Endpoint), or 8 (AER
       Bridge), or use type 9 (Generic Hardware Error Source).  Firmware first
       error handling is possible if and only if Trusted Firmware is being
       used on arm64.

       Must be supplied if RAS support is provided by the platform.  It
       is recommended this table be supplied.

HPET   Signature Reserved (signature == "HPET")

       **High Precision Event timer Table**

       x86 only table, will not be supported.

IBFT   Signature Reserved (signature == "IBFT")

       **iSCSI Boot Firmware Table**

       Microsoft defined table, support TBD.

IORT   Signature Reserved (signature == "IORT")

       **Input Output Remapping Table**

       arm64 only table, required in order to describe IO topology, SMMUs,
       and GIC ITSs, and how those various components are connected together,
       such as identifying which components are behind which SMMUs/ITSs.
       This table will only be required on certain SBSA platforms (e.g.,
       when using GICv3-ITS and an SMMU); on SBSA Level 0 platforms, it
       remains optional.

IVRS   Signature Reserved (signature == "IVRS")

       **I/O Virtualization Reporting Structure**

       x86_64 (AMD) only table, will not be supported.

LPIT   Signature Reserved (signature == "LPIT")

       **Low Power Idle Table**

       x86 only table as of ACPI 5.1; starting with ACPI 6.0, processor
       descriptions and power states on ARM platforms should use the DSDT
       and define processor container devices (_HID ACPI0010, Section 8.4,
       and more specifically 8.4.3 and 8.4.4).

MADT   Section 5.2.12 (signature == "APIC")

       **Multiple APIC Description Table**

       Required for arm64.  Only the GIC interrupt controller structures
       should be used (types 0xA - 0xF).

MCFG   Signature Reserved (signature == "MCFG")

       **Memory-mapped ConFiGuration space**

       If the platform supports PCI/PCIe, an MCFG table is required.

MCHI   Signature Reserved (signature == "MCHI")

       **Management Controller Host Interface table**

       Optional, not currently supported.

MPST   Section 5.2.21 (signature == "MPST")

       **Memory Power State Table**

       Optional, not currently supported.

MSCT   Section 5.2.19 (signature == "MSCT")

       **Maximum System Characteristic Table**

       Optional, not currently supported.

MSDM   Signature Reserved (signature == "MSDM")

       **Microsoft Data Management table**

       Microsoft only table, will not be supported.

NFIT   Section 5.2.25 (signature == "NFIT")

       **NVDIMM Firmware Interface Table**

       Optional, not currently supported.

OEMx   Signature of "OEMx" only

       **OEM Specific Tables**

       All tables starting with a signature of "OEM" are reserved for OEM
       use.  Since these are not meant to be of general use but are limited
       to very specific end users, they are not recommended for use and are
       not supported by the kernel for arm64.

PCCT   Section 14.1 (signature == "PCCT)

       **Platform Communications Channel Table**

       Recommend for use on arm64; use of PCC is recommended when using CPPC
       to control performance and power for platform processors.

PMTT   Section 5.2.21.12 (signature == "PMTT")

       **Platform Memory Topology Table**

       Optional, not currently supported.

PSDT   Section 5.2.11.3 (signature == "PSDT")

       **Persistent System Description Table**

       Obsolete table, will not be supported.

RASF   Section 5.2.20 (signature == "RASF")

       **RAS Feature table**

       Optional, not currently supported.

RSDP   Section 5.2.5 (signature == "RSD PTR")

       **Root System Description PoinTeR**

       Required for arm64.

RSDT   Section 5.2.7 (signature == "RSDT")

       **Root System Description Table**

       Since this table can only provide 32-bit addresses, it is deprecated
       on arm64, and will not be used.  If provided, it will be ignored.

SBST   Section 5.2.14 (signature == "SBST")

       **Smart Battery Subsystem Table**

       Optional, not currently supported.

SLIC   Signature Reserved (signature == "SLIC")

       **Software LIcensing table**

       Microsoft only table, will not be supported.

SLIT   Section 5.2.17 (signature == "SLIT")

       **System Locality distance Information Table**

       Optional in general, but required for NUMA systems.

SPCR   Signature Reserved (signature == "SPCR")

       **Serial Port Console Redirection table**

       Required for arm64.

SPMI   Signature Reserved (signature == "SPMI")

       **Server Platform Management Interface table**

       Optional, not currently supported.

SRAT   Section 5.2.16 (signature == "SRAT")

       **System Resource Affinity Table**

       Optional, but if used, only the GICC Affinity structures are read.
       To support arm64 NUMA, this table is required.

SSDT   Section 5.2.11.2 (signature == "SSDT")

       **Secondary System Description Table**

       These tables are a continuation of the DSDT; these are recommended
       for use with devices that can be added to a running system, but can
       also serve the purpose of dividing up device descriptions into more
       manageable pieces.

       An SSDT can only ADD to the ACPI namespace.  It cannot modify or
       replace existing device descriptions already in the namespace.

       These tables are optional, however.  ACPI tables should contain only
       one DSDT but can contain many SSDTs.

STAO   Signature Reserved (signature == "STAO")

       **_STA Override table**

       Optional, but only necessary in virtualized environments in order to
       hide devices from guest OSs.

TCPA   Signature Reserved (signature == "TCPA")

       **Trusted Computing Platform Alliance table**

       Optional, not currently supported, and may need changes to fully
       interoperate with arm64.

TPM2   Signature Reserved (signature == "TPM2")

       **Trusted Platform Module 2 table**

       Optional, not currently supported, and may need changes to fully
       interoperate with arm64.

UEFI   Signature Reserved (signature == "UEFI")

       **UEFI ACPI data table**

       Optional, not currently supported.  No known use case for arm64,
       at present.

WAET   Signature Reserved (signature == "WAET")

       **Windows ACPI Emulated devices Table**

       Microsoft only table, will not be supported.

WDAT   Signature Reserved (signature == "WDAT")

       **Watch Dog Action Table**

       Microsoft only table, will not be supported.

WDRT   Signature Reserved (signature == "WDRT")

       **Watch Dog Resource Table**

       Microsoft only table, will not be supported.

WPBT   Signature Reserved (signature == "WPBT")

       **Windows Platform Binary Table**

       Microsoft only table, will not be supported.

XENV   Signature Reserved (signature == "XENV")

       **Xen project table**

       Optional, used only by Xen at present.

XSDT   Section 5.2.8 (signature == "XSDT")

       **eXtended System Description Table**

       Required for arm64.
====== ========================================================================

ACPI Objects
------------
The expectations on individual ACPI objects that are likely to be used are
shown in the list that follows; any object not explicitly mentioned below
should be used as needed for a particular platform or particular subsystem,
such as power management or PCI.

===== ================ ========================================================
Name   Section         Usage for ARMv8 Linux
===== ================ ========================================================
_CCA   6.2.17          This method must be defined for all bus masters
                       on arm64 - there are no assumptions made about
                       whether such devices are cache coherent or not.
                       The _CCA value is inherited by all descendants of
                       these devices so it does not need to be repeated.
                       Without _CCA on arm64, the kernel does not know what
                       to do about setting up DMA for the device.

                       NB: this method provides default cache coherency
                       attributes; the presence of an SMMU can be used to
                       modify that, however.  For example, a master could
                       default to non-coherent, but be made coherent with
                       the appropriate SMMU configuration (see Table 17 of
                       the IORT specification, ARM Document DEN 0049B).

_CID   6.1.2           Use as needed, see also _HID.

_CLS   6.1.3           Use as needed, see also _HID.

_CPC   8.4.7.1         Use as needed, power management specific.  CPPC is
                       recommended on arm64.

_CRS   6.2.2           Required on arm64.

_CSD   8.4.2.2         Use as needed, used only in conjunction with _CST.

_CST   8.4.2.1         Low power idle states (8.4.4) are recommended instead
                       of C-states.

_DDN   6.1.4           This field can be used for a device name.  However,
                       it is meant for DOS device names (e.g., COM1), so be
                       careful of its use across OSes.

_DSD   6.2.5           To be used with caution.  If this object is used, try
                       to use it within the constraints already defined by the
                       Device Properties UUID.  Only in rare circumstances
                       should it be necessary to create a new _DSD UUID.

                       In either case, submit the _DSD definition along with
                       any driver patches for discussion, especially when
                       device properties are used.  A driver will not be
                       considered complete without a corresponding _DSD
                       description.  Once approved by kernel maintainers,
                       the UUID or device properties must then be registered
                       with the UEFI Forum; this may cause some iteration as
                       more than one OS will be registering entries.

_DSM   9.1.1           Do not use this method.  It is not standardized, the
                       return values are not well documented, and it is
                       currently a frequent source of error.

\_GL   5.7.1           This object is not to be used in hardware reduced
                       mode, and therefore should not be used on arm64.

_GLK   6.5.7           This object requires a global lock be defined; there
                       is no global lock on arm64 since it runs in hardware
                       reduced mode.  Hence, do not use this object on arm64.

\_GPE  5.3.1           This namespace is for x86 use only.  Do not use it
                       on arm64.

_HID   6.1.5           This is the primary object to use in device probing,
		       though _CID and _CLS may also be used.

_INI   6.5.1           Not required, but can be useful in setting up devices
                       when UEFI leaves them in a state that may not be what
                       the driver expects before it starts probing.

_LPI   8.4.4.3         Recommended for use with processor definitions (_HID
		       ACPI0010) on arm64.  See also _RDI.

_MLS   6.1.7           Highly recommended for use in internationalization.

_OFF   7.2.2           It is recommended to define this method for any device
                       that can be turned on or off.

_ON    7.2.3           It is recommended to define this method for any device
                       that can be turned on or off.

\_OS   5.7.3           This method will return "Linux" by default (this is
                       the value of the macro ACPI_OS_NAME on Linux).  The
                       command line parameter acpi_os=<string> can be used
                       to set it to some other value.

_OSC   6.2.11          This method can be a global method in ACPI (i.e.,
                       \_SB._OSC), or it may be associated with a specific
                       device (e.g., \_SB.DEV0._OSC), or both.  When used
                       as a global method, only capabilities published in
                       the ACPI specification are allowed.  When used as
                       a device-specific method, the process described for
                       using _DSD MUST be used to create an _OSC definition;
                       out-of-process use of _OSC is not allowed.  That is,
                       submit the device-specific _OSC usage description as
                       part of the kernel driver submission, get it approved
                       by the kernel community, then register it with the
                       UEFI Forum.

\_OSI  5.7.2           Deprecated on ARM64.  As far as ACPI firmware is
		       concerned, _OSI is not to be used to determine what
		       sort of system is being used or what functionality
		       is provided.  The _OSC method is to be used instead.

_PDC   8.4.1           Deprecated, do not use on arm64.

\_PIC  5.8.1           The method should not be used.  On arm64, the only
                       interrupt model available is GIC.

\_PR   5.3.1           This namespace is for x86 use only on legacy systems.
                       Do not use it on arm64.

_PRT   6.2.13          Required as part of the definition of all PCI root
                       devices.

_PRx   7.3.8-11        Use as needed; power management specific.  If _PR0 is
                       defined, _PR3 must also be defined.

_PSx   7.3.2-5         Use as needed; power management specific.  If _PS0 is
                       defined, _PS3 must also be defined.  If clocks or
                       regulators need adjusting to be consistent with power
                       usage, change them in these methods.

_RDI   8.4.4.4         Recommended for use with processor definitions (_HID
		       ACPI0010) on arm64.  This should only be used in
		       conjunction with _LPI.

\_REV  5.7.4           Always returns the latest version of ACPI supported.

\_SB   5.3.1           Required on arm64; all devices must be defined in this
                       namespace.

_SLI   6.2.15          Use is recommended when SLIT table is in use.

_STA   6.3.7,          It is recommended to define this method for any device
       7.2.4           that can be turned on or off.  See also the STAO table
                       that provides overrides to hide devices in virtualized
                       environments.

_SRS   6.2.16          Use as needed; see also _PRS.

_STR   6.1.10          Recommended for conveying device names to end users;
                       this is preferred over using _DDN.

_SUB   6.1.9           Use as needed; _HID or _CID are preferred.

_SUN   6.1.11          Use as needed, but recommended.

_SWS   7.4.3           Use as needed; power management specific; this may
                       require specification changes for use on arm64.

_UID   6.1.12          Recommended for distinguishing devices of the same
                       class; define it if at all possible.
===== ================ ========================================================

ACPI Event Model
----------------
Do not use GPE block devices; these are not supported in the hardware reduced
profile used by arm64.  Since there are no GPE blocks defined for use on ARM
platforms, ACPI events must be signaled differently.

There are two options: GPIO-signaled interrupts (Section 5.6.5), and
interrupt-signaled events (Section 5.6.9).  Interrupt-signaled events are a
new feature in the ACPI 6.1 specification.  Either - or both - can be used
on a given platform, and which to use may be dependent of limitations in any
given SoC.  If possible, interrupt-signaled events are recommended.

ACPI Processor Control
----------------------
Section 8 of the ACPI specification changed significantly in version 6.0.
Processors should now be defined as Device objects with _HID ACPI0007; do
not use the deprecated Processor statement in ASL.  All multiprocessor systems
should also define a hierarchy of processors, done with Processor Container
Devices (see Section 8.4.3.1, _HID ACPI0010); do not use processor aggregator
devices (Section 8.5) to describe processor topology.  Section 8.4 of the
specification describes the semantics of these object definitions and how
they interrelate.

Most importantly, the processor hierarchy defined also defines the low power
idle states that are available to the platform, along with the rules for
determining which processors can be turned on or off and the circumstances
that control that.  Without this information, the processors will run in
whatever power state they were left in by UEFI.

Note too, that the processor Device objects defined and the entries in the
MADT for GICs are expected to be in synchronization.  The _UID of the Device
object must correspond to processor IDs used in the MADT.

It is recommended that CPPC (8.4.5) be used as the primary model for processor
performance control on arm64.  C-states and P-states may become available at
some point in the future, but most current design work appears to favor CPPC.

Further, it is essential that the ARMv8 SoC provide a fully functional
implementation of PSCI; this will be the only mechanism supported by ACPI
to control CPU power state.  Booting of secondary CPUs using the ACPI
parking protocol is possible, but discouraged, since only PSCI is supported
for ARM servers.

ACPI System Address Map Interfaces
----------------------------------
In Section 15 of the ACPI specification, several methods are mentioned as
possible mechanisms for conveying memory resource information to the kernel.
For arm64, we will only support UEFI for booting with ACPI, hence the UEFI
GetMemoryMap() boot service is the only mechanism that will be used.

ACPI Platform Error Interfaces (APEI)
-------------------------------------
The APEI tables supported are described above.

APEI requires the equivalent of an SCI and an NMI on ARMv8.  The SCI is used
to notify the OSPM of errors that have occurred but can be corrected and the
system can continue correct operation, even if possibly degraded.  The NMI is
used to indicate fatal errors that cannot be corrected, and require immediate
attention.

Since there is no direct equivalent of the x86 SCI or NMI, arm64 handles
these slightly differently.  The SCI is handled as a high priority interrupt;
given that these are corrected (or correctable) errors being reported, this
is sufficient.  The NMI is emulated as the highest priority interrupt
possible.  This implies some caution must be used since there could be
interrupts at higher privilege levels or even interrupts at the same priority
as the emulated NMI.  In Linux, this should not be the case but one should
be aware it could happen.

ACPI Objects Not Supported on ARM64
-----------------------------------
While this may change in the future, there are several classes of objects
that can be defined, but are not currently of general interest to ARM servers.
Some of these objects have x86 equivalents, and may actually make sense in ARM
servers.  However, there is either no hardware available at present, or there
may not even be a non-ARM implementation yet.  Hence, they are not currently
supported.

The following classes of objects are not supported:

       -  Section 9.2: ambient light sensor devices

       -  Section 9.3: battery devices

       -  Section 9.4: lids (e.g., laptop lids)

       -  Section 9.8.2: IDE controllers

       -  Section 9.9: floppy controllers

       -  Section 9.10: GPE block devices

       -  Section 9.15: PC/AT RTC/CMOS devices

       -  Section 9.16: user presence detection devices

       -  Section 9.17: I/O APIC devices; all GICs must be enumerable via MADT

       -  Section 9.18: time and alarm devices (see 9.15)

       -  Section 10: power source and power meter devices

       -  Section 11: thermal management

       -  Section 12: embedded controllers interface

       -  Section 13: SMBus interfaces

This also means that there is no support for the following objects:

====   =========================== ====   ==========
Name   Section                     Name   Section
====   =========================== ====   ==========
_ALC   9.3.4                       _FDM   9.10.3
_ALI   9.3.2                       _FIX   6.2.7
_ALP   9.3.6                       _GAI   10.4.5
_ALR   9.3.5                       _GHL   10.4.7
_ALT   9.3.3                       _GTM   9.9.2.1.1
_BCT   10.2.2.10                   _LID   9.5.1
_BDN   6.5.3                       _PAI   10.4.4
_BIF   10.2.2.1                    _PCL   10.3.2
_BIX   10.2.2.1                    _PIF   10.3.3
_BLT   9.2.3                       _PMC   10.4.1
_BMA   10.2.2.4                    _PMD   10.4.8
_BMC   10.2.2.12                   _PMM   10.4.3
_BMD   10.2.2.11                   _PRL   10.3.4
_BMS   10.2.2.5                    _PSR   10.3.1
_BST   10.2.2.6                    _PTP   10.4.2
_BTH   10.2.2.7                    _SBS   10.1.3
_BTM   10.2.2.9                    _SHL   10.4.6
_BTP   10.2.2.8                    _STM   9.9.2.1.1
_DCK   6.5.2                       _UPD   9.16.1
_EC    12.12                       _UPP   9.16.2
_FDE   9.10.1                      _WPC   10.5.2
_FDI   9.10.2                      _WPP   10.5.3
====   =========================== ====   ==========

repo_name: angery/linux


code:
.. _amu_index:

=======================================================
Activity Monitors Unit (AMU) extension in AArch64 Linux
=======================================================

Author: Ionela Voinescu <ionela.voinescu@arm.com>

Date: 2019-09-10

This document briefly describes the provision of Activity Monitors Unit
support in AArch64 Linux.

Architecture overview
---------------------

The activity monitors extension is an optional extension introduced by the
ARMv8.4 CPU architecture.

The activity monitors unit, implemented in each CPU, provides performance
counters intended for system management use. The AMU extension provides a
system register interface to the counter registers and also supports an
optional external memory-mapped interface.

Version 1 of the Activity Monitors architecture implements a counter group
of four fixed and architecturally defined 64-bit event counters.

  - CPU cycle counter: increments at the frequency of the CPU.
  - Constant counter: increments at the fixed frequency of the system
    clock.
  - Instructions retired: increments with every architecturally executed
    instruction.
  - Memory stall cycles: counts instruction dispatch stall cycles caused by
    misses in the last level cache within the clock domain.

When in WFI or WFE these counters do not increment.

The Activity Monitors architecture provides space for up to 16 architected
event counters. Future versions of the architecture may use this space to
implement additional architected event counters.

Additionally, version 1 implements a counter group of up to 16 auxiliary
64-bit event counters.

On cold reset all counters reset to 0.

Basic support
-------------

The kernel can safely run a mix of CPUs with and without support for the
activity monitors extension. Therefore, when CONFIG_ARM64_AMU_EXTN is
selected we unconditionally enable the capability to allow any late CPU
(secondary or hotplugged) to detect and use the feature.

When the feature is detected on a CPU, we flag the availability of the
feature but this does not guarantee the correct functionality of the
counters, only the presence of the extension.

Firmware (code running at higher exception levels, e.g. arm-tf) support is
needed to:

 - Enable access for lower exception levels (EL2 and EL1) to the AMU
   registers.
 - Enable the counters. If not enabled these will read as 0.
 - Save/restore the counters before/after the CPU is being put/brought up
   from the 'off' power state.

When using kernels that have this feature enabled but boot with broken
firmware the user may experience panics or lockups when accessing the
counter registers. Even if these symptoms are not observed, the values
returned by the register reads might not correctly reflect reality. Most
commonly, the counters will read as 0, indicating that they are not
enabled.

If proper support is not provided in firmware it's best to disable
CONFIG_ARM64_AMU_EXTN. To be noted that for security reasons, this does not
bypass the setting of AMUSERENR_EL0 to trap accesses from EL0 (userspace) to
EL1 (kernel). Therefore, firmware should still ensure accesses to AMU registers
are not trapped in EL2/EL3.

The fixed counters of AMUv1 are accessible though the following system
register definitions:

 - SYS_AMEVCNTR0_CORE_EL0
 - SYS_AMEVCNTR0_CONST_EL0
 - SYS_AMEVCNTR0_INST_RET_EL0
 - SYS_AMEVCNTR0_MEM_STALL_EL0

Auxiliary platform specific counters can be accessed using
SYS_AMEVCNTR1_EL0(n), where n is a value between 0 and 15.

Details can be found in: arch/arm64/include/asm/sysreg.h.

Userspace access
----------------

Currently, access from userspace to the AMU registers is disabled due to:

 - Security reasons: they might expose information about code executed in
   secure mode.
 - Purpose: AMU counters are intended for system management use.

Also, the presence of the feature is not visible to userspace.

Virtualization
--------------

Currently, access from userspace (EL0) and kernelspace (EL1) on the KVM
guest side is disabled due to:

 - Security reasons: they might expose information about code executed
   by other guests or the host.

Any attempt to access the AMU registers will result in an UNDEFINED
exception being injected into the guest.


code:
=====================
ACPI on ARMv8 Servers
=====================

ACPI can be used for ARMv8 general purpose servers designed to follow
the ARM SBSA (Server Base System Architecture) [0] and SBBR (Server
Base Boot Requirements) [1] specifications.  Please note that the SBBR
can be retrieved simply by visiting [1], but the SBSA is currently only
available to those with an ARM login due to ARM IP licensing concerns.

The ARMv8 kernel implements the reduced hardware model of ACPI version
5.1 or later.  Links to the specification and all external documents
it refers to are managed by the UEFI Forum.  The specification is
available at http://www.uefi.org/specifications and documents referenced
by the specification can be found via http://www.uefi.org/acpi.

If an ARMv8 system does not meet the requirements of the SBSA and SBBR,
or cannot be described using the mechanisms defined in the required ACPI
specifications, then ACPI may not be a good fit for the hardware.

While the documents mentioned above set out the requirements for building
industry-standard ARMv8 servers, they also apply to more than one operating
system.  The purpose of this document is to describe the interaction between
ACPI and Linux only, on an ARMv8 system -- that is, what Linux expects of
ACPI and what ACPI can expect of Linux.

Why ACPI on ARM?
----------------
Before examining the details of the interface between ACPI and Linux, it is
useful to understand why ACPI is being used.  Several technologies already
exist in Linux for describing non-enumerable hardware, after all.  In this
section we summarize a blog post [2] from Grant Likely that outlines the
reasoning behind ACPI on ARMv8 servers.  Actually, we snitch a good portion
of the summary text almost directly, to be honest.

The short form of the rationale for ACPI on ARM is:

-  ACPI’s byte code (AML) allows the platform to encode hardware behavior,
   while DT explicitly does not support this.  For hardware vendors, being
   able to encode behavior is a key tool used in supporting operating
   system releases on new hardware.

-  ACPI’s OSPM defines a power management model that constrains what the
   platform is allowed to do into a specific model, while still providing
   flexibility in hardware design.

-  In the enterprise server environment, ACPI has established bindings (such
   as for RAS) which are currently used in production systems.  DT does not.
   Such bindings could be defined in DT at some point, but doing so means ARM
   and x86 would end up using completely different code paths in both firmware
   and the kernel.

-  Choosing a single interface to describe the abstraction between a platform
   and an OS is important.  Hardware vendors would not be required to implement
   both DT and ACPI if they want to support multiple operating systems.  And,
   agreeing on a single interface instead of being fragmented into per OS
   interfaces makes for better interoperability overall.

-  The new ACPI governance process works well and Linux is now at the same
   table as hardware vendors and other OS vendors.  In fact, there is no
   longer any reason to feel that ACPI only belongs to Windows or that
   Linux is in any way secondary to Microsoft in this arena.  The move of
   ACPI governance into the UEFI forum has significantly opened up the
   specification development process, and currently, a large portion of the
   changes being made to ACPI are being driven by Linux.

Key to the use of ACPI is the support model.  For servers in general, the
responsibility for hardware behaviour cannot solely be the domain of the
kernel, but rather must be split between the platform and the kernel, in
order to allow for orderly change over time.  ACPI frees the OS from needing
to understand all the minute details of the hardware so that the OS doesn’t
need to be ported to each and every device individually.  It allows the
hardware vendors to take responsibility for power management behaviour without
depending on an OS release cycle which is not under their control.

ACPI is also important because hardware and OS vendors have already worked
out the mechanisms for supporting a general purpose computing ecosystem.  The
infrastructure is in place, the bindings are in place, and the processes are
in place.  DT does exactly what Linux needs it to when working with vertically
integrated devices, but there are no good processes for supporting what the
server vendors need.  Linux could potentially get there with DT, but doing so
really just duplicates something that already works.  ACPI already does what
the hardware vendors need, Microsoft won’t collaborate on DT, and hardware
vendors would still end up providing two completely separate firmware
interfaces -- one for Linux and one for Windows.

Kernel Compatibility
--------------------
One of the primary motivations for ACPI is standardization, and using that
to provide backward compatibility for Linux kernels.  In the server market,
software and hardware are often used for long periods.  ACPI allows the
kernel and firmware to agree on a consistent abstraction that can be
maintained over time, even as hardware or software change.  As long as the
abstraction is supported, systems can be updated without necessarily having
to replace the kernel.

When a Linux driver or subsystem is first implemented using ACPI, it by
definition ends up requiring a specific version of the ACPI specification
-- it's baseline.  ACPI firmware must continue to work, even though it may
not be optimal, with the earliest kernel version that first provides support
for that baseline version of ACPI.  There may be a need for additional drivers,
but adding new functionality (e.g., CPU power management) should not break
older kernel versions.  Further, ACPI firmware must also work with the most
recent version of the kernel.

Relationship with Device Tree
-----------------------------
ACPI support in drivers and subsystems for ARMv8 should never be mutually
exclusive with DT support at compile time.

At boot time the kernel will only use one description method depending on
parameters passed from the boot loader (including kernel bootargs).

Regardless of whether DT or ACPI is used, the kernel must always be capable
of booting with either scheme (in kernels with both schemes enabled at compile
time).

Booting using ACPI tables
-------------------------
The only defined method for passing ACPI tables to the kernel on ARMv8
is via the UEFI system configuration table.  Just so it is explicit, this
means that ACPI is only supported on platforms that boot via UEFI.

When an ARMv8 system boots, it can either have DT information, ACPI tables,
or in some very unusual cases, both.  If no command line parameters are used,
the kernel will try to use DT for device enumeration; if there is no DT
present, the kernel will try to use ACPI tables, but only if they are present.
In neither is available, the kernel will not boot.  If acpi=force is used
on the command line, the kernel will attempt to use ACPI tables first, but
fall back to DT if there are no ACPI tables present.  The basic idea is that
the kernel will not fail to boot unless it absolutely has no other choice.

Processing of ACPI tables may be disabled by passing acpi=off on the kernel
command line; this is the default behavior.

In order for the kernel to load and use ACPI tables, the UEFI implementation
MUST set the ACPI_20_TABLE_GUID to point to the RSDP table (the table with
the ACPI signature "RSD PTR ").  If this pointer is incorrect and acpi=force
is used, the kernel will disable ACPI and try to use DT to boot instead; the
kernel has, in effect, determined that ACPI tables are not present at that
point.

If the pointer to the RSDP table is correct, the table will be mapped into
the kernel by the ACPI core, using the address provided by UEFI.

The ACPI core will then locate and map in all other ACPI tables provided by
using the addresses in the RSDP table to find the XSDT (eXtended System
Description Table).  The XSDT in turn provides the addresses to all other
ACPI tables provided by the system firmware; the ACPI core will then traverse
this table and map in the tables listed.

The ACPI core will ignore any provided RSDT (Root System Description Table).
RSDTs have been deprecated and are ignored on arm64 since they only allow
for 32-bit addresses.

Further, the ACPI core will only use the 64-bit address fields in the FADT
(Fixed ACPI Description Table).  Any 32-bit address fields in the FADT will
be ignored on arm64.

Hardware reduced mode (see Section 4.1 of the ACPI 6.1 specification) will
be enforced by the ACPI core on arm64.  Doing so allows the ACPI core to
run less complex code since it no longer has to provide support for legacy
hardware from other architectures.  Any fields that are not to be used for
hardware reduced mode must be set to zero.

For the ACPI core to operate properly, and in turn provide the information
the kernel needs to configure devices, it expects to find the following
tables (all section numbers refer to the ACPI 6.1 specification):

    -  RSDP (Root System Description Pointer), section 5.2.5

    -  XSDT (eXtended System Description Table), section 5.2.8

    -  FADT (Fixed ACPI Description Table), section 5.2.9

    -  DSDT (Differentiated System Description Table), section
       5.2.11.1

    -  MADT (Multiple APIC Description Table), section 5.2.12

    -  GTDT (Generic Timer Description Table), section 5.2.24

    -  If PCI is supported, the MCFG (Memory mapped ConFiGuration
       Table), section 5.2.6, specifically Table 5-31.

    -  If booting without a console=<device> kernel parameter is
       supported, the SPCR (Serial Port Console Redirection table),
       section 5.2.6, specifically Table 5-31.

    -  If necessary to describe the I/O topology, SMMUs and GIC ITSs,
       the IORT (Input Output Remapping Table, section 5.2.6, specifically
       Table 5-31).

    -  If NUMA is supported, the SRAT (System Resource Affinity Table)
       and SLIT (System Locality distance Information Table), sections
       5.2.16 and 5.2.17, respectively.

If the above tables are not all present, the kernel may or may not be
able to boot properly since it may not be able to configure all of the
devices available.  This list of tables is not meant to be all inclusive;
in some environments other tables may be needed (e.g., any of the APEI
tables from section 18) to support specific functionality.

ACPI Detection
--------------
Drivers should determine their probe() type by checking for a null
value for ACPI_HANDLE, or checking .of_node, or other information in
the device structure.  This is detailed further in the "Driver
Recommendations" section.

In non-driver code, if the presence of ACPI needs to be detected at
run time, then check the value of acpi_disabled. If CONFIG_ACPI is not
set, acpi_disabled will always be 1.

Device Enumeration
------------------
Device descriptions in ACPI should use standard recognized ACPI interfaces.
These may contain less information than is typically provided via a Device
Tree description for the same device.  This is also one of the reasons that
ACPI can be useful -- the driver takes into account that it may have less
detailed information about the device and uses sensible defaults instead.
If done properly in the driver, the hardware can change and improve over
time without the driver having to change at all.

Clocks provide an excellent example.  In DT, clocks need to be specified
and the drivers need to take them into account.  In ACPI, the assumption
is that UEFI will leave the device in a reasonable default state, including
any clock settings.  If for some reason the driver needs to change a clock
value, this can be done in an ACPI method; all the driver needs to do is
invoke the method and not concern itself with what the method needs to do
to change the clock.  Changing the hardware can then take place over time
by changing what the ACPI method does, and not the driver.

In DT, the parameters needed by the driver to set up clocks as in the example
above are known as "bindings"; in ACPI, these are known as "Device Properties"
and provided to a driver via the _DSD object.

ACPI tables are described with a formal language called ASL, the ACPI
Source Language (section 19 of the specification).  This means that there
are always multiple ways to describe the same thing -- including device
properties.  For example, device properties could use an ASL construct
that looks like this: Name(KEY0, "value0").  An ACPI device driver would
then retrieve the value of the property by evaluating the KEY0 object.
However, using Name() this way has multiple problems: (1) ACPI limits
names ("KEY0") to four characters unlike DT; (2) there is no industry
wide registry that maintains a list of names, minimizing re-use; (3)
there is also no registry for the definition of property values ("value0"),
again making re-use difficult; and (4) how does one maintain backward
compatibility as new hardware comes out?  The _DSD method was created
to solve precisely these sorts of problems; Linux drivers should ALWAYS
use the _DSD method for device properties and nothing else.

The _DSM object (ACPI Section 9.14.1) could also be used for conveying
device properties to a driver.  Linux drivers should only expect it to
be used if _DSD cannot represent the data required, and there is no way
to create a new UUID for the _DSD object.  Note that there is even less
regulation of the use of _DSM than there is of _DSD.  Drivers that depend
on the contents of _DSM objects will be more difficult to maintain over
time because of this; as of this writing, the use of _DSM is the cause
of quite a few firmware problems and is not recommended.

Drivers should look for device properties in the _DSD object ONLY; the _DSD
object is described in the ACPI specification section 6.2.5, but this only
describes how to define the structure of an object returned via _DSD, and
how specific data structures are defined by specific UUIDs.  Linux should
only use the _DSD Device Properties UUID [5]:

   - UUID: daffd814-6eba-4d8c-8a91-bc9bbf4aa301

   - https://www.uefi.org/sites/default/files/resources/_DSD-device-properties-UUID.pdf

The UEFI Forum provides a mechanism for registering device properties [4]
so that they may be used across all operating systems supporting ACPI.
Device properties that have not been registered with the UEFI Forum should
not be used.

Before creating new device properties, check to be sure that they have not
been defined before and either registered in the Linux kernel documentation
as DT bindings, or the UEFI Forum as device properties.  While we do not want
to simply move all DT bindings into ACPI device properties, we can learn from
what has been previously defined.

If it is necessary to define a new device property, or if it makes sense to
synthesize the definition of a binding so it can be used in any firmware,
both DT bindings and ACPI device properties for device drivers have review
processes.  Use them both.  When the driver itself is submitted for review
to the Linux mailing lists, the device property definitions needed must be
submitted at the same time.  A driver that supports ACPI and uses device
properties will not be considered complete without their definitions.  Once
the device property has been accepted by the Linux community, it must be
registered with the UEFI Forum [4], which will review it again for consistency
within the registry.  This may require iteration.  The UEFI Forum, though,
will always be the canonical site for device property definitions.

It may make sense to provide notice to the UEFI Forum that there is the
intent to register a previously unused device property name as a means of
reserving the name for later use.  Other operating system vendors will
also be submitting registration requests and this may help smooth the
process.

Once registration and review have been completed, the kernel provides an
interface for looking up device properties in a manner independent of
whether DT or ACPI is being used.  This API should be used [6]; it can
eliminate some duplication of code paths in driver probing functions and
discourage divergence between DT bindings and ACPI device properties.

Programmable Power Control Resources
------------------------------------
Programmable power control resources include such resources as voltage/current
providers (regulators) and clock sources.

With ACPI, the kernel clock and regulator framework is not expected to be used
at all.

The kernel assumes that power control of these resources is represented with
Power Resource Objects (ACPI section 7.1).  The ACPI core will then handle
correctly enabling and disabling resources as they are needed.  In order to
get that to work, ACPI assumes each device has defined D-states and that these
can be controlled through the optional ACPI methods _PS0, _PS1, _PS2, and _PS3;
in ACPI, _PS0 is the method to invoke to turn a device full on, and _PS3 is for
turning a device full off.

There are two options for using those Power Resources.  They can:

   -  be managed in a _PSx method which gets called on entry to power
      state Dx.

   -  be declared separately as power resources with their own _ON and _OFF
      methods.  They are then tied back to D-states for a particular device
      via _PRx which specifies which power resources a device needs to be on
      while in Dx.  Kernel then tracks number of devices using a power resource
      and calls _ON/_OFF as needed.

The kernel ACPI code will also assume that the _PSx methods follow the normal
ACPI rules for such methods:

   -  If either _PS0 or _PS3 is implemented, then the other method must also
      be implemented.

   -  If a device requires usage or setup of a power resource when on, the ASL
      should organize that it is allocated/enabled using the _PS0 method.

   -  Resources allocated or enabled in the _PS0 method should be disabled
      or de-allocated in the _PS3 method.

   -  Firmware will leave the resources in a reasonable state before handing
      over control to the kernel.

Such code in _PSx methods will of course be very platform specific.  But,
this allows the driver to abstract out the interface for operating the device
and avoid having to read special non-standard values from ACPI tables. Further,
abstracting the use of these resources allows the hardware to change over time
without requiring updates to the driver.

Clocks
------
ACPI makes the assumption that clocks are initialized by the firmware --
UEFI, in this case -- to some working value before control is handed over
to the kernel.  This has implications for devices such as UARTs, or SoC-driven
LCD displays, for example.

When the kernel boots, the clocks are assumed to be set to reasonable
working values.  If for some reason the frequency needs to change -- e.g.,
throttling for power management -- the device driver should expect that
process to be abstracted out into some ACPI method that can be invoked
(please see the ACPI specification for further recommendations on standard
methods to be expected).  The only exceptions to this are CPU clocks where
CPPC provides a much richer interface than ACPI methods.  If the clocks
are not set, there is no direct way for Linux to control them.

If an SoC vendor wants to provide fine-grained control of the system clocks,
they could do so by providing ACPI methods that could be invoked by Linux
drivers.  However, this is NOT recommended and Linux drivers should NOT use
such methods, even if they are provided.  Such methods are not currently
standardized in the ACPI specification, and using them could tie a kernel
to a very specific SoC, or tie an SoC to a very specific version of the
kernel, both of which we are trying to avoid.

Driver Recommendations
----------------------
DO NOT remove any DT handling when adding ACPI support for a driver.  The
same device may be used on many different systems.

DO try to structure the driver so that it is data-driven.  That is, set up
a struct containing internal per-device state based on defaults and whatever
else must be discovered by the driver probe function.  Then, have the rest
of the driver operate off of the contents of that struct.  Doing so should
allow most divergence between ACPI and DT functionality to be kept local to
the probe function instead of being scattered throughout the driver.  For
example::

  static int device_probe_dt(struct platform_device *pdev)
  {
         /* DT specific functionality */
         ...
  }

  static int device_probe_acpi(struct platform_device *pdev)
  {
         /* ACPI specific functionality */
         ...
  }

  static int device_probe(struct platform_device *pdev)
  {
         ...
         struct device_node node = pdev->dev.of_node;
         ...

         if (node)
                 ret = device_probe_dt(pdev);
         else if (ACPI_HANDLE(&pdev->dev))
                 ret = device_probe_acpi(pdev);
         else
                 /* other initialization */
                 ...
         /* Continue with any generic probe operations */
         ...
  }

DO keep the MODULE_DEVICE_TABLE entries together in the driver to make it
clear the different names the driver is probed for, both from DT and from
ACPI::

  static struct of_device_id virtio_mmio_match[] = {
          { .compatible = "virtio,mmio", },
          { }
  };
  MODULE_DEVICE_TABLE(of, virtio_mmio_match);

  static const struct acpi_device_id virtio_mmio_acpi_match[] = {
          { "LNRO0005", },
          { }
  };
  MODULE_DEVICE_TABLE(acpi, virtio_mmio_acpi_match);

ASWG
----
The ACPI specification changes regularly.  During the year 2014, for instance,
version 5.1 was released and version 6.0 substantially completed, with most of
the changes being driven by ARM-specific requirements.  Proposed changes are
presented and discussed in the ASWG (ACPI Specification Working Group) which
is a part of the UEFI Forum.  The current version of the ACPI specification
is 6.1 release in January 2016.

Participation in this group is open to all UEFI members.  Please see
http://www.uefi.org/workinggroup for details on group membership.

It is the intent of the ARMv8 ACPI kernel code to follow the ACPI specification
as closely as possible, and to only implement functionality that complies with
the released standards from UEFI ASWG.  As a practical matter, there will be
vendors that provide bad ACPI tables or violate the standards in some way.
If this is because of errors, quirks and fix-ups may be necessary, but will
be avoided if possible.  If there are features missing from ACPI that preclude
it from being used on a platform, ECRs (Engineering Change Requests) should be
submitted to ASWG and go through the normal approval process; for those that
are not UEFI members, many other members of the Linux community are and would
likely be willing to assist in submitting ECRs.

Linux Code
----------
Individual items specific to Linux on ARM, contained in the Linux
source code, are in the list that follows:

ACPI_OS_NAME
                       This macro defines the string to be returned when
                       an ACPI method invokes the _OS method.  On ARM64
                       systems, this macro will be "Linux" by default.
                       The command line parameter acpi_os=<string>
                       can be used to set it to some other value.  The
                       default value for other architectures is "Microsoft
                       Windows NT", for example.

ACPI Objects
------------
Detailed expectations for ACPI tables and object are listed in the file
Documentation/arm64/acpi_object_usage.rst.

References
----------
[0] http://silver.arm.com
    document ARM-DEN-0029, or newer:
    "Server Base System Architecture", version 2.3, dated 27 Mar 2014

[1] http://infocenter.arm.com/help/topic/com.arm.doc.den0044a/Server_Base_Boot_Requirements.pdf
    Document ARM-DEN-0044A, or newer: "Server Base Boot Requirements, System
    Software on ARM Platforms", dated 16 Aug 2014

[2] http://www.secretlab.ca/archives/151,
    10 Jan 2015, Copyright (c) 2015,
    Linaro Ltd., written by Grant Likely.

[3] AMD ACPI for Seattle platform documentation
    http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2012/10/Seattle_ACPI_Guide.pdf

[4] http://www.uefi.org/acpi
    please see the link for the "ACPI _DSD Device
    Property Registry Instructions"

[5] http://www.uefi.org/acpi
    please see the link for the "_DSD (Device
    Specific Data) Implementation Guide"

[6] Kernel code for the unified device
    property interface can be found in
    include/linux/property.h and drivers/base/property.c.

Authors
-------
- Al Stone <al.stone@linaro.org>
- Graeme Gregory <graeme.gregory@linaro.org>
- Hanjun Guo <hanjun.guo@linaro.org>

- Grant Likely <grant.likely@linaro.org>, for the "Why ACPI on ARM?" section


code:
======================
Asymmetric 32-bit SoCs
======================

Author: Will Deacon <will@kernel.org>

This document describes the impact of asymmetric 32-bit SoCs on the
execution of 32-bit (``AArch32``) applications.

Date: 2021-05-17

Introduction
============

Some Armv9 SoCs suffer from a big.LITTLE misfeature where only a subset
of the CPUs are capable of executing 32-bit user applications. On such
a system, Linux by default treats the asymmetry as a "mismatch" and
disables support for both the ``PER_LINUX32`` personality and
``execve(2)`` of 32-bit ELF binaries, with the latter returning
``-ENOEXEC``. If the mismatch is detected during late onlining of a
64-bit-only CPU, then the onlining operation fails and the new CPU is
unavailable for scheduling.

Surprisingly, these SoCs have been produced with the intention of
running legacy 32-bit binaries. Unsurprisingly, that doesn't work very
well with the default behaviour of Linux.

It seems inevitable that future SoCs will drop 32-bit support
altogether, so if you're stuck in the unenviable position of needing to
run 32-bit code on one of these transitionary platforms then you would
be wise to consider alternatives such as recompilation, emulation or
retirement. If neither of those options are practical, then read on.

Enabling kernel support
=======================

Since the kernel support is not completely transparent to userspace,
allowing 32-bit tasks to run on an asymmetric 32-bit system requires an
explicit "opt-in" and can be enabled by passing the
``allow_mismatched_32bit_el0`` parameter on the kernel command-line.

For the remainder of this document we will refer to an *asymmetric
system* to mean an asymmetric 32-bit SoC running Linux with this kernel
command-line option enabled.

Userspace impact
================

32-bit tasks running on an asymmetric system behave in mostly the same
way as on a homogeneous system, with a few key differences relating to
CPU affinity.

sysfs
-----

The subset of CPUs capable of running 32-bit tasks is described in
``/sys/devices/system/cpu/aarch32_el0`` and is documented further in
``Documentation/ABI/testing/sysfs-devices-system-cpu``.

**Note:** CPUs are advertised by this file as they are detected and so
late-onlining of 32-bit-capable CPUs can result in the file contents
being modified by the kernel at runtime. Once advertised, CPUs are never
removed from the file.

``execve(2)``
-------------

On a homogeneous system, the CPU affinity of a task is preserved across
``execve(2)``. This is not always possible on an asymmetric system,
specifically when the new program being executed is 32-bit yet the
affinity mask contains 64-bit-only CPUs. In this situation, the kernel
determines the new affinity mask as follows:

  1. If the 32-bit-capable subset of the affinity mask is not empty,
     then the affinity is restricted to that subset and the old affinity
     mask is saved. This saved mask is inherited over ``fork(2)`` and
     preserved across ``execve(2)`` of 32-bit programs.

     **Note:** This step does not apply to ``SCHED_DEADLINE`` tasks.
     See `SCHED_DEADLINE`_.

  2. Otherwise, the cpuset hierarchy of the task is walked until an
     ancestor is found containing at least one 32-bit-capable CPU. The
     affinity of the task is then changed to match the 32-bit-capable
     subset of the cpuset determined by the walk.

  3. On failure (i.e. out of memory), the affinity is changed to the set
     of all 32-bit-capable CPUs of which the kernel is aware.

A subsequent ``execve(2)`` of a 64-bit program by the 32-bit task will
invalidate the affinity mask saved in (1) and attempt to restore the CPU
affinity of the task using the saved mask if it was previously valid.
This restoration may fail due to intervening changes to the deadline
policy or cpuset hierarchy, in which case the ``execve(2)`` continues
with the affinity unchanged.

Calls to ``sched_setaffinity(2)`` for a 32-bit task will consider only
the 32-bit-capable CPUs of the requested affinity mask. On success, the
affinity for the task is updated and any saved mask from a prior
``execve(2)`` is invalidated.

``SCHED_DEADLINE``
------------------

Explicit admission of a 32-bit deadline task to the default root domain
(e.g. by calling ``sched_setattr(2)``) is rejected on an asymmetric
32-bit system unless admission control is disabled by writing -1 to
``/proc/sys/kernel/sched_rt_runtime_us``.

``execve(2)`` of a 32-bit program from a 64-bit deadline task will
return ``-ENOEXEC`` if the root domain for the task contains any
64-bit-only CPUs and admission control is enabled. Concurrent offlining
of 32-bit-capable CPUs may still necessitate the procedure described in
`execve(2)`_, in which case step (1) is skipped and a warning is
emitted on the console.

**Note:** It is recommended that a set of 32-bit-capable CPUs are placed
into a separate root domain if ``SCHED_DEADLINE`` is to be used with
32-bit tasks on an asymmetric system. Failure to do so is likely to
result in missed deadlines.

Cpusets
-------

The affinity of a 32-bit task on an asymmetric system may include CPUs
that are not explicitly allowed by the cpuset to which it is attached.
This can occur as a result of the following two situations:

  - A 64-bit task attached to a cpuset which allows only 64-bit CPUs
    executes a 32-bit program.

  - All of the 32-bit-capable CPUs allowed by a cpuset containing a
    32-bit task are offlined.

In both of these cases, the new affinity is calculated according to step
(2) of the process described in `execve(2)`_ and the cpuset hierarchy is
unchanged irrespective of the cgroup version.

CPU hotplug
-----------

On an asymmetric system, the first detected 32-bit-capable CPU is
prevented from being offlined by userspace and any such attempt will
return ``-EPERM``. Note that suspend is still permitted even if the
primary CPU (i.e. CPU 0) is 64-bit-only.

KVM
---

Although KVM will not advertise 32-bit EL0 support to any vCPUs on an
asymmetric system, a broken guest at EL1 could still attempt to execute
32-bit code at EL0. In this case, an exit from a vCPU thread in 32-bit
mode will return to host userspace with an ``exit_reason`` of
``KVM_EXIT_FAIL_ENTRY`` and will remain non-runnable until successfully
re-initialised by a subsequent ``KVM_ARM_VCPU_INIT`` operation.


code:
=====================
Booting AArch64 Linux
=====================

Author: Will Deacon <will.deacon@arm.com>

Date  : 07 September 2012

This document is based on the ARM booting document by Russell King and
is relevant to all public releases of the AArch64 Linux kernel.

The AArch64 exception model is made up of a number of exception levels
(EL0 - EL3), with EL0, EL1 and EL2 having a secure and a non-secure
counterpart.  EL2 is the hypervisor level, EL3 is the highest priority
level and exists only in secure mode. Both are architecturally optional.

For the purposes of this document, we will use the term `boot loader`
simply to define all software that executes on the CPU(s) before control
is passed to the Linux kernel.  This may include secure monitor and
hypervisor code, or it may just be a handful of instructions for
preparing a minimal boot environment.

Essentially, the boot loader should provide (as a minimum) the
following:

1. Setup and initialise the RAM
2. Setup the device tree
3. Decompress the kernel image
4. Call the kernel image

1. Setup and initialise RAM
---------------------------

Requirement: MANDATORY

The boot loader is expected to find and initialise all RAM that the
kernel will use for volatile data storage in the system.  It performs
this in a machine dependent manner.  (It may use internal algorithms
to automatically locate and size all RAM, or it may use knowledge of
the RAM in the machine, or any other method the boot loader designer
sees fit.)

2. Setup the device tree
-------------------------

Requirement: MANDATORY

The device tree blob (dtb) must be placed on an 8-byte boundary and must
not exceed 2 megabytes in size. Since the dtb will be mapped cacheable
using blocks of up to 2 megabytes in size, it must not be placed within
any 2M region which must be mapped with any specific attributes.

NOTE: versions prior to v4.2 also require that the DTB be placed within
the 512 MB region starting at text_offset bytes below the kernel Image.

3. Decompress the kernel image
------------------------------

Requirement: OPTIONAL

The AArch64 kernel does not currently provide a decompressor and
therefore requires decompression (gzip etc.) to be performed by the boot
loader if a compressed Image target (e.g. Image.gz) is used.  For
bootloaders that do not implement this requirement, the uncompressed
Image target is available instead.

4. Call the kernel image
------------------------

Requirement: MANDATORY

The decompressed kernel image contains a 64-byte header as follows::

  u32 code0;			/* Executable code */
  u32 code1;			/* Executable code */
  u64 text_offset;		/* Image load offset, little endian */
  u64 image_size;		/* Effective Image size, little endian */
  u64 flags;			/* kernel flags, little endian */
  u64 res2	= 0;		/* reserved */
  u64 res3	= 0;		/* reserved */
  u64 res4	= 0;		/* reserved */
  u32 magic	= 0x644d5241;	/* Magic number, little endian, "ARM\x64" */
  u32 res5;			/* reserved (used for PE COFF offset) */

Header notes:

- As of v3.17, all fields are little endian unless stated otherwise.

- code0/code1 are responsible for branching to stext.

- when booting through EFI, code0/code1 are initially skipped.
  res5 is an offset to the PE header and the PE header has the EFI
  entry point (efi_stub_entry).  When the stub has done its work, it
  jumps to code0 to resume the normal boot process.

- Prior to v3.17, the endianness of text_offset was not specified.  In
  these cases image_size is zero and text_offset is 0x80000 in the
  endianness of the kernel.  Where image_size is non-zero image_size is
  little-endian and must be respected.  Where image_size is zero,
  text_offset can be assumed to be 0x80000.

- The flags field (introduced in v3.17) is a little-endian 64-bit field
  composed as follows:

  ============= ===============================================================
  Bit 0		Kernel endianness.  1 if BE, 0 if LE.
  Bit 1-2	Kernel Page size.

			* 0 - Unspecified.
			* 1 - 4K
			* 2 - 16K
			* 3 - 64K
  Bit 3		Kernel physical placement

			0
			  2MB aligned base should be as close as possible
			  to the base of DRAM, since memory below it is not
			  accessible via the linear mapping
			1
			  2MB aligned base such that all image_size bytes
			  counted from the start of the image are within
			  the 48-bit addressable range of physical memory
  Bits 4-63	Reserved.
  ============= ===============================================================

- When image_size is zero, a bootloader should attempt to keep as much
  memory as possible free for use by the kernel immediately after the
  end of the kernel image. The amount of space required will vary
  depending on selected features, and is effectively unbound.

The Image must be placed text_offset bytes from a 2MB aligned base
address anywhere in usable system RAM and called there. The region
between the 2 MB aligned base address and the start of the image has no
special significance to the kernel, and may be used for other purposes.
At least image_size bytes from the start of the image must be free for
use by the kernel.
NOTE: versions prior to v4.6 cannot make use of memory below the
physical offset of the Image so it is recommended that the Image be
placed as close as possible to the start of system RAM.

If an initrd/initramfs is passed to the kernel at boot, it must reside
entirely within a 1 GB aligned physical memory window of up to 32 GB in
size that fully covers the kernel Image as well.

Any memory described to the kernel (even that below the start of the
image) which is not marked as reserved from the kernel (e.g., with a
memreserve region in the device tree) will be considered as available to
the kernel.

Before jumping into the kernel, the following conditions must be met:

- Quiesce all DMA capable devices so that memory does not get
  corrupted by bogus network packets or disk data.  This will save
  you many hours of debug.

- Primary CPU general-purpose register settings:

    - x0 = physical address of device tree blob (dtb) in system RAM.
    - x1 = 0 (reserved for future use)
    - x2 = 0 (reserved for future use)
    - x3 = 0 (reserved for future use)

- CPU mode

  All forms of interrupts must be masked in PSTATE.DAIF (Debug, SError,
  IRQ and FIQ).
  The CPU must be in non-secure state, either in EL2 (RECOMMENDED in order
  to have access to the virtualisation extensions), or in EL1.

- Caches, MMUs

  The MMU must be off.

  The instruction cache may be on or off, and must not hold any stale
  entries corresponding to the loaded kernel image.

  The address range corresponding to the loaded kernel image must be
  cleaned to the PoC. In the presence of a system cache or other
  coherent masters with caches enabled, this will typically require
  cache maintenance by VA rather than set/way operations.
  System caches which respect the architected cache maintenance by VA
  operations must be configured and may be enabled.
  System caches which do not respect architected cache maintenance by VA
  operations (not recommended) must be configured and disabled.

- Architected timers

  CNTFRQ must be programmed with the timer frequency and CNTVOFF must
  be programmed with a consistent value on all CPUs.  If entering the
  kernel at EL1, CNTHCTL_EL2 must have EL1PCTEN (bit 0) set where
  available.

- Coherency

  All CPUs to be booted by the kernel must be part of the same coherency
  domain on entry to the kernel.  This may require IMPLEMENTATION DEFINED
  initialisation to enable the receiving of maintenance operations on
  each CPU.

- System registers

  All writable architected system registers at or below the exception
  level where the kernel image will be entered must be initialised by
  software at a higher exception level to prevent execution in an UNKNOWN
  state.

  For all systems:
  - If EL3 is present:

    - SCR_EL3.FIQ must have the same value across all CPUs the kernel is
      executing on.
    - The value of SCR_EL3.FIQ must be the same as the one present at boot
      time whenever the kernel is executing.

  - If EL3 is present and the kernel is entered at EL2:

    - SCR_EL3.HCE (bit 8) must be initialised to 0b1.

  For systems with a GICv3 interrupt controller to be used in v3 mode:
  - If EL3 is present:

      - ICC_SRE_EL3.Enable (bit 3) must be initialiased to 0b1.
      - ICC_SRE_EL3.SRE (bit 0) must be initialised to 0b1.
      - ICC_CTLR_EL3.PMHE (bit 6) must be set to the same value across
        all CPUs the kernel is executing on, and must stay constant
        for the lifetime of the kernel.

  - If the kernel is entered at EL1:

      - ICC.SRE_EL2.Enable (bit 3) must be initialised to 0b1
      - ICC_SRE_EL2.SRE (bit 0) must be initialised to 0b1.

  - The DT or ACPI tables must describe a GICv3 interrupt controller.

  For systems with a GICv3 interrupt controller to be used in
  compatibility (v2) mode:

  - If EL3 is present:

      ICC_SRE_EL3.SRE (bit 0) must be initialised to 0b0.

  - If the kernel is entered at EL1:

      ICC_SRE_EL2.SRE (bit 0) must be initialised to 0b0.

  - The DT or ACPI tables must describe a GICv2 interrupt controller.

  For CPUs with pointer authentication functionality:

  - If EL3 is present:

    - SCR_EL3.APK (bit 16) must be initialised to 0b1
    - SCR_EL3.API (bit 17) must be initialised to 0b1

  - If the kernel is entered at EL1:

    - HCR_EL2.APK (bit 40) must be initialised to 0b1
    - HCR_EL2.API (bit 41) must be initialised to 0b1

  For CPUs with Activity Monitors Unit v1 (AMUv1) extension present:

  - If EL3 is present:

    - CPTR_EL3.TAM (bit 30) must be initialised to 0b0
    - CPTR_EL2.TAM (bit 30) must be initialised to 0b0
    - AMCNTENSET0_EL0 must be initialised to 0b1111
    - AMCNTENSET1_EL0 must be initialised to a platform specific value
      having 0b1 set for the corresponding bit for each of the auxiliary
      counters present.

  - If the kernel is entered at EL1:

    - AMCNTENSET0_EL0 must be initialised to 0b1111
    - AMCNTENSET1_EL0 must be initialised to a platform specific value
      having 0b1 set for the corresponding bit for each of the auxiliary
      counters present.

  For CPUs with the Fine Grained Traps (FEAT_FGT) extension present:

  - If EL3 is present and the kernel is entered at EL2:

    - SCR_EL3.FGTEn (bit 27) must be initialised to 0b1.

  For CPUs with support for HCRX_EL2 (FEAT_HCX) present:

  - If EL3 is present and the kernel is entered at EL2:

    - SCR_EL3.HXEn (bit 38) must be initialised to 0b1.

  For CPUs with Advanced SIMD and floating point support:

  - If EL3 is present:

    - CPTR_EL3.TFP (bit 10) must be initialised to 0b0.

  - If EL2 is present and the kernel is entered at EL1:

    - CPTR_EL2.TFP (bit 10) must be initialised to 0b0.

  For CPUs with the Scalable Vector Extension (FEAT_SVE) present:

  - if EL3 is present:

    - CPTR_EL3.EZ (bit 8) must be initialised to 0b1.

    - ZCR_EL3.LEN must be initialised to the same value for all CPUs the
      kernel is executed on.

  - If the kernel is entered at EL1 and EL2 is present:

    - CPTR_EL2.TZ (bit 8) must be initialised to 0b0.

    - CPTR_EL2.ZEN (bits 17:16) must be initialised to 0b11.

    - ZCR_EL2.LEN must be initialised to the same value for all CPUs the
      kernel will execute on.

  For CPUs with the Scalable Matrix Extension (FEAT_SME):

  - If EL3 is present:

    - CPTR_EL3.ESM (bit 12) must be initialised to 0b1.

    - SCR_EL3.EnTP2 (bit 41) must be initialised to 0b1.

    - SMCR_EL3.LEN must be initialised to the same value for all CPUs the
      kernel will execute on.

 - If the kernel is entered at EL1 and EL2 is present:

    - CPTR_EL2.TSM (bit 12) must be initialised to 0b0.

    - CPTR_EL2.SMEN (bits 25:24) must be initialised to 0b11.

    - SCTLR_EL2.EnTP2 (bit 60) must be initialised to 0b1.

    - SMCR_EL2.LEN must be initialised to the same value for all CPUs the
      kernel will execute on.

    - HWFGRTR_EL2.nTPIDR2_EL0 (bit 55) must be initialised to 0b01.

    - HWFGWTR_EL2.nTPIDR2_EL0 (bit 55) must be initialised to 0b01.

    - HWFGRTR_EL2.nSMPRI_EL1 (bit 54) must be initialised to 0b01.

    - HWFGWTR_EL2.nSMPRI_EL1 (bit 54) must be initialised to 0b01.

  For CPUs with the Scalable Matrix Extension FA64 feature (FEAT_SME_FA64):

  - If EL3 is present:

    - SMCR_EL3.FA64 (bit 31) must be initialised to 0b1.

 - If the kernel is entered at EL1 and EL2 is present:

    - SMCR_EL2.FA64 (bit 31) must be initialised to 0b1.

  For CPUs with the Memory Tagging Extension feature (FEAT_MTE2):

  - If EL3 is present:

    - SCR_EL3.ATA (bit 26) must be initialised to 0b1.

  - If the kernel is entered at EL1 and EL2 is present:

    - HCR_EL2.ATA (bit 56) must be initialised to 0b1.

The requirements described above for CPU mode, caches, MMUs, architected
timers, coherency and system registers apply to all CPUs.  All CPUs must
enter the kernel in the same exception level.  Where the values documented
disable traps it is permissible for these traps to be enabled so long as
those traps are handled transparently by higher exception levels as though
the values documented were set.

The boot loader is expected to enter the kernel on each CPU in the
following manner:

- The primary CPU must jump directly to the first instruction of the
  kernel image.  The device tree blob passed by this CPU must contain
  an 'enable-method' property for each cpu node.  The supported
  enable-methods are described below.

  It is expected that the bootloader will generate these device tree
  properties and insert them into the blob prior to kernel entry.

- CPUs with a "spin-table" enable-method must have a 'cpu-release-addr'
  property in their cpu node.  This property identifies a
  naturally-aligned 64-bit zero-initalised memory location.

  These CPUs should spin outside of the kernel in a reserved area of
  memory (communicated to the kernel by a /memreserve/ region in the
  device tree) polling their cpu-release-addr location, which must be
  contained in the reserved region.  A wfe instruction may be inserted
  to reduce the overhead of the busy-loop and a sev will be issued by
  the primary CPU.  When a read of the location pointed to by the
  cpu-release-addr returns a non-zero value, the CPU must jump to this
  value.  The value will be written as a single 64-bit little-endian
  value, so CPUs must convert the read value to their native endianness
  before jumping to it.

- CPUs with a "psci" enable method should remain outside of
  the kernel (i.e. outside of the regions of memory described to the
  kernel in the memory node, or in a reserved area of memory described
  to the kernel by a /memreserve/ region in the device tree).  The
  kernel will issue CPU_ON calls as described in ARM document number ARM
  DEN 0022A ("Power State Coordination Interface System Software on ARM
  processors") to bring CPUs into the kernel.

  The device tree should contain a 'psci' node, as described in
  Documentation/devicetree/bindings/arm/psci.yaml.

- Secondary CPU general-purpose register settings

  - x0 = 0 (reserved for future use)
  - x1 = 0 (reserved for future use)
  - x2 = 0 (reserved for future use)
  - x3 = 0 (reserved for future use)