code:
.. _userfaultfd:

===========
Userfaultfd
===========

Objective
=========

Userfaults allow the implementation of on-demand paging from userland
and more generally they allow userland to take control of various
memory page faults, something otherwise only the kernel code could do.

For example userfaults allows a proper and more optimal implementation
of the ``PROT_NONE+SIGSEGV`` trick.

Design
======

Userspace creates a new userfaultfd, initializes it, and registers one or more
regions of virtual memory with it. Then, any page faults which occur within the
region(s) result in a message being delivered to the userfaultfd, notifying
userspace of the fault.

The ``userfaultfd`` (aside from registering and unregistering virtual
memory ranges) provides two primary functionalities:

1) ``read/POLLIN`` protocol to notify a userland thread of the faults
   happening

2) various ``UFFDIO_*`` ioctls that can manage the virtual memory regions
   registered in the ``userfaultfd`` that allows userland to efficiently
   resolve the userfaults it receives via 1) or to manage the virtual
   memory in the background

The real advantage of userfaults if compared to regular virtual memory
management of mremap/mprotect is that the userfaults in all their
operations never involve heavyweight structures like vmas (in fact the
``userfaultfd`` runtime load never takes the mmap_lock for writing).
Vmas are not suitable for page- (or hugepage) granular fault tracking
when dealing with virtual address spaces that could span
Terabytes. Too many vmas would be needed for that.

The ``userfaultfd``, once created, can also be
passed using unix domain sockets to a manager process, so the same
manager process could handle the userfaults of a multitude of
different processes without them being aware about what is going on
(well of course unless they later try to use the ``userfaultfd``
themselves on the same region the manager is already tracking, which
is a corner case that would currently return ``-EBUSY``).

API
===

Creating a userfaultfd
----------------------

There are two ways to create a new userfaultfd, each of which provide ways to
restrict access to this functionality (since historically userfaultfds which
handle kernel page faults have been a useful tool for exploiting the kernel).

The first way, supported since userfaultfd was introduced, is the
userfaultfd(2) syscall. Access to this is controlled in several ways:

- Any user can always create a userfaultfd which traps userspace page faults
  only. Such a userfaultfd can be created using the userfaultfd(2) syscall
  with the flag UFFD_USER_MODE_ONLY.

- In order to also trap kernel page faults for the address space, either the
  process needs the CAP_SYS_PTRACE capability, or the system must have
  vm.unprivileged_userfaultfd set to 1. By default, vm.unprivileged_userfaultfd
  is set to 0.

The second way, added to the kernel more recently, is by opening
/dev/userfaultfd and issuing a USERFAULTFD_IOC_NEW ioctl to it. This method
yields equivalent userfaultfds to the userfaultfd(2) syscall.

Unlike userfaultfd(2), access to /dev/userfaultfd is controlled via normal
filesystem permissions (user/group/mode), which gives fine grained access to
userfaultfd specifically, without also granting other unrelated privileges at
the same time (as e.g. granting CAP_SYS_PTRACE would do). Users who have access
to /dev/userfaultfd can always create userfaultfds that trap kernel page faults;
vm.unprivileged_userfaultfd is not considered.

Initializing a userfaultfd
--------------------------

When first opened the ``userfaultfd`` must be enabled invoking the
``UFFDIO_API`` ioctl specifying a ``uffdio_api.api`` value set to ``UFFD_API`` (or
a later API version) which will specify the ``read/POLLIN`` protocol
userland intends to speak on the ``UFFD`` and the ``uffdio_api.features``
userland requires. The ``UFFDIO_API`` ioctl if successful (i.e. if the
requested ``uffdio_api.api`` is spoken also by the running kernel and the
requested features are going to be enabled) will return into
``uffdio_api.features`` and ``uffdio_api.ioctls`` two 64bit bitmasks of
respectively all the available features of the read(2) protocol and
the generic ioctl available.

The ``uffdio_api.features`` bitmask returned by the ``UFFDIO_API`` ioctl
defines what memory types are supported by the ``userfaultfd`` and what
events, except page fault notifications, may be generated:

- The ``UFFD_FEATURE_EVENT_*`` flags indicate that various other events
  other than page faults are supported. These events are described in more
  detail below in the `Non-cooperative userfaultfd`_ section.

- ``UFFD_FEATURE_MISSING_HUGETLBFS`` and ``UFFD_FEATURE_MISSING_SHMEM``
  indicate that the kernel supports ``UFFDIO_REGISTER_MODE_MISSING``
  registrations for hugetlbfs and shared memory (covering all shmem APIs,
  i.e. tmpfs, ``IPCSHM``, ``/dev/zero``, ``MAP_SHARED``, ``memfd_create``,
  etc) virtual memory areas, respectively.

- ``UFFD_FEATURE_MINOR_HUGETLBFS`` indicates that the kernel supports
  ``UFFDIO_REGISTER_MODE_MINOR`` registration for hugetlbfs virtual memory
  areas. ``UFFD_FEATURE_MINOR_SHMEM`` is the analogous feature indicating
  support for shmem virtual memory areas.

The userland application should set the feature flags it intends to use
when invoking the ``UFFDIO_API`` ioctl, to request that those features be
enabled if supported.

Once the ``userfaultfd`` API has been enabled the ``UFFDIO_REGISTER``
ioctl should be invoked (if present in the returned ``uffdio_api.ioctls``
bitmask) to register a memory range in the ``userfaultfd`` by setting the
uffdio_register structure accordingly. The ``uffdio_register.mode``
bitmask will specify to the kernel which kind of faults to track for
the range. The ``UFFDIO_REGISTER`` ioctl will return the
``uffdio_register.ioctls`` bitmask of ioctls that are suitable to resolve
userfaults on the range registered. Not all ioctls will necessarily be
supported for all memory types (e.g. anonymous memory vs. shmem vs.
hugetlbfs), or all types of intercepted faults.

Userland can use the ``uffdio_register.ioctls`` to manage the virtual
address space in the background (to add or potentially also remove
memory from the ``userfaultfd`` registered range). This means a userfault
could be triggering just before userland maps in the background the
user-faulted page.

Resolving Userfaults
--------------------

There are three basic ways to resolve userfaults:

- ``UFFDIO_COPY`` atomically copies some existing page contents from
  userspace.

- ``UFFDIO_ZEROPAGE`` atomically zeros the new page.

- ``UFFDIO_CONTINUE`` maps an existing, previously-populated page.

These operations are atomic in the sense that they guarantee nothing can
see a half-populated page, since readers will keep userfaulting until the
operation has finished.

By default, these wake up userfaults blocked on the range in question.
They support a ``UFFDIO_*_MODE_DONTWAKE`` ``mode`` flag, which indicates
that waking will be done separately at some later time.

Which ioctl to choose depends on the kind of page fault, and what we'd
like to do to resolve it:

- For ``UFFDIO_REGISTER_MODE_MISSING`` faults, the fault needs to be
  resolved by either providing a new page (``UFFDIO_COPY``), or mapping
  the zero page (``UFFDIO_ZEROPAGE``). By default, the kernel would map
  the zero page for a missing fault. With userfaultfd, userspace can
  decide what content to provide before the faulting thread continues.

- For ``UFFDIO_REGISTER_MODE_MINOR`` faults, there is an existing page (in
  the page cache). Userspace has the option of modifying the page's
  contents before resolving the fault. Once the contents are correct
  (modified or not), userspace asks the kernel to map the page and let the
  faulting thread continue with ``UFFDIO_CONTINUE``.

Notes:

- You can tell which kind of fault occurred by examining
  ``pagefault.flags`` within the ``uffd_msg``, checking for the
  ``UFFD_PAGEFAULT_FLAG_*`` flags.

- None of the page-delivering ioctls default to the range that you
  registered with.  You must fill in all fields for the appropriate
  ioctl struct including the range.

- You get the address of the access that triggered the missing page
  event out of a struct uffd_msg that you read in the thread from the
  uffd.  You can supply as many pages as you want with these IOCTLs.
  Keep in mind that unless you used DONTWAKE then the first of any of
  those IOCTLs wakes up the faulting thread.

- Be sure to test for all errors including
  (``pollfd[0].revents & POLLERR``).  This can happen, e.g. when ranges
  supplied were incorrect.

Write Protect Notifications
---------------------------

This is equivalent to (but faster than) using mprotect and a SIGSEGV
signal handler.

Firstly you need to register a range with ``UFFDIO_REGISTER_MODE_WP``.
Instead of using mprotect(2) you use
``ioctl(uffd, UFFDIO_WRITEPROTECT, struct *uffdio_writeprotect)``
while ``mode = UFFDIO_WRITEPROTECT_MODE_WP``
in the struct passed in.  The range does not default to and does not
have to be identical to the range you registered with.  You can write
protect as many ranges as you like (inside the registered range).
Then, in the thread reading from uffd the struct will have
``msg.arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WP`` set. Now you send
``ioctl(uffd, UFFDIO_WRITEPROTECT, struct *uffdio_writeprotect)``
again while ``pagefault.mode`` does not have ``UFFDIO_WRITEPROTECT_MODE_WP``
set. This wakes up the thread which will continue to run with writes. This
allows you to do the bookkeeping about the write in the uffd reading
thread before the ioctl.

If you registered with both ``UFFDIO_REGISTER_MODE_MISSING`` and
``UFFDIO_REGISTER_MODE_WP`` then you need to think about the sequence in
which you supply a page and undo write protect.  Note that there is a
difference between writes into a WP area and into a !WP area.  The
former will have ``UFFD_PAGEFAULT_FLAG_WP`` set, the latter
``UFFD_PAGEFAULT_FLAG_WRITE``.  The latter did not fail on protection but
you still need to supply a page when ``UFFDIO_REGISTER_MODE_MISSING`` was
used.

QEMU/KVM
========

QEMU/KVM is using the ``userfaultfd`` syscall to implement postcopy live
migration. Postcopy live migration is one form of memory
externalization consisting of a virtual machine running with part or
all of its memory residing on a different node in the cloud. The
``userfaultfd`` abstraction is generic enough that not a single line of
KVM kernel code had to be modified in order to add postcopy live
migration to QEMU.

Guest async page faults, ``FOLL_NOWAIT`` and all other ``GUP*`` features work
just fine in combination with userfaults. Userfaults trigger async
page faults in the guest scheduler so those guest processes that
aren't waiting for userfaults (i.e. network bound) can keep running in
the guest vcpus.

It is generally beneficial to run one pass of precopy live migration
just before starting postcopy live migration, in order to avoid
generating userfaults for readonly guest regions.

The implementation of postcopy live migration currently uses one
single bidirectional socket but in the future two different sockets
will be used (to reduce the latency of the userfaults to the minimum
possible without having to decrease ``/proc/sys/net/ipv4/tcp_wmem``).

The QEMU in the source node writes all pages that it knows are missing
in the destination node, into the socket, and the migration thread of
the QEMU running in the destination node runs ``UFFDIO_COPY|ZEROPAGE``
ioctls on the ``userfaultfd`` in order to map the received pages into the
guest (``UFFDIO_ZEROCOPY`` is used if the source page was a zero page).

A different postcopy thread in the destination node listens with
poll() to the ``userfaultfd`` in parallel. When a ``POLLIN`` event is
generated after a userfault triggers, the postcopy thread read() from
the ``userfaultfd`` and receives the fault address (or ``-EAGAIN`` in case the
userfault was already resolved and waken by a ``UFFDIO_COPY|ZEROPAGE`` run
by the parallel QEMU migration thread).

After the QEMU postcopy thread (running in the destination node) gets
the userfault address it writes the information about the missing page
into the socket. The QEMU source node receives the information and
roughly "seeks" to that page address and continues sending all
remaining missing pages from that new page offset. Soon after that
(just the time to flush the tcp_wmem queue through the network) the
migration thread in the QEMU running in the destination node will
receive the page that triggered the userfault and it'll map it as
usual with the ``UFFDIO_COPY|ZEROPAGE`` (without actually knowing if it
was spontaneously sent by the source or if it was an urgent page
requested through a userfault).

By the time the userfaults start, the QEMU in the destination node
doesn't need to keep any per-page state bitmap relative to the live
migration around and a single per-page bitmap has to be maintained in
the QEMU running in the source node to know which pages are still
missing in the destination node. The bitmap in the source node is
checked to find which missing pages to send in round robin and we seek
over it when receiving incoming userfaults. After sending each page of
course the bitmap is updated accordingly. It's also useful to avoid
sending the same page twice (in case the userfault is read by the
postcopy thread just before ``UFFDIO_COPY|ZEROPAGE`` runs in the migration
thread).

Non-cooperative userfaultfd
===========================

When the ``userfaultfd`` is monitored by an external manager, the manager
must be able to track changes in the process virtual memory
layout. Userfaultfd can notify the manager about such changes using
the same read(2) protocol as for the page fault notifications. The
manager has to explicitly enable these events by setting appropriate
bits in ``uffdio_api.features`` passed to ``UFFDIO_API`` ioctl:

``UFFD_FEATURE_EVENT_FORK``
	enable ``userfaultfd`` hooks for fork(). When this feature is
	enabled, the ``userfaultfd`` context of the parent process is
	duplicated into the newly created process. The manager
	receives ``UFFD_EVENT_FORK`` with file descriptor of the new
	``userfaultfd`` context in the ``uffd_msg.fork``.

``UFFD_FEATURE_EVENT_REMAP``
	enable notifications about mremap() calls. When the
	non-cooperative process moves a virtual memory area to a
	different location, the manager will receive
	``UFFD_EVENT_REMAP``. The ``uffd_msg.remap`` will contain the old and
	new addresses of the area and its original length.

``UFFD_FEATURE_EVENT_REMOVE``
	enable notifications about madvise(MADV_REMOVE) and
	madvise(MADV_DONTNEED) calls. The event ``UFFD_EVENT_REMOVE`` will
	be generated upon these calls to madvise(). The ``uffd_msg.remove``
	will contain start and end addresses of the removed area.

``UFFD_FEATURE_EVENT_UNMAP``
	enable notifications about memory unmapping. The manager will
	get ``UFFD_EVENT_UNMAP`` with ``uffd_msg.remove`` containing start and
	end addresses of the unmapped area.

Although the ``UFFD_FEATURE_EVENT_REMOVE`` and ``UFFD_FEATURE_EVENT_UNMAP``
are pretty similar, they quite differ in the action expected from the
``userfaultfd`` manager. In the former case, the virtual memory is
removed, but the area is not, the area remains monitored by the
``userfaultfd``, and if a page fault occurs in that area it will be
delivered to the manager. The proper resolution for such page fault is
to zeromap the faulting address. However, in the latter case, when an
area is unmapped, either explicitly (with munmap() system call), or
implicitly (e.g. during mremap()), the area is removed and in turn the
``userfaultfd`` context for such area disappears too and the manager will
not get further userland page faults from the removed area. Still, the
notification is required in order to prevent manager from using
``UFFDIO_COPY`` on the unmapped area.

Unlike userland page faults which have to be synchronous and require
explicit or implicit wakeup, all the events are delivered
asynchronously and the non-cooperative process resumes execution as
soon as manager executes read(). The ``userfaultfd`` manager should
carefully synchronize calls to ``UFFDIO_COPY`` with the events
processing. To aid the synchronization, the ``UFFDIO_COPY`` ioctl will
return ``-ENOSPC`` when the monitored process exits at the time of
``UFFDIO_COPY``, and ``-ENOENT``, when the non-cooperative process has changed
its virtual memory layout simultaneously with outstanding ``UFFDIO_COPY``
operation.

The current asynchronous model of the event delivery is optimal for
single threaded non-cooperative ``userfaultfd`` manager implementations. A
synchronous event delivery model can be added later as a new
``userfaultfd`` feature to facilitate multithreading enhancements of the
non cooperative manager, for example to allow ``UFFDIO_COPY`` ioctls to
run in parallel to the event reception. Single threaded
implementations should continue to use the current async event
delivery model instead.


code:
.. _zswap:

=====
zswap
=====

Overview
========

Zswap is a lightweight compressed cache for swap pages. It takes pages that are
in the process of being swapped out and attempts to compress them into a
dynamically allocated RAM-based memory pool.  zswap basically trades CPU cycles
for potentially reduced swap I/O.  This trade-off can also result in a
significant performance improvement if reads from the compressed cache are
faster than reads from a swap device.

Some potential benefits:

* Desktop/laptop users with limited RAM capacities can mitigate the
  performance impact of swapping.
* Overcommitted guests that share a common I/O resource can
  dramatically reduce their swap I/O pressure, avoiding heavy handed I/O
  throttling by the hypervisor. This allows more work to get done with less
  impact to the guest workload and guests sharing the I/O subsystem
* Users with SSDs as swap devices can extend the life of the device by
  drastically reducing life-shortening writes.

Zswap evicts pages from compressed cache on an LRU basis to the backing swap
device when the compressed pool reaches its size limit.  This requirement had
been identified in prior community discussions.

Whether Zswap is enabled at the boot time depends on whether
the ``CONFIG_ZSWAP_DEFAULT_ON`` Kconfig option is enabled or not.
This setting can then be overridden by providing the kernel command line
``zswap.enabled=`` option, for example ``zswap.enabled=0``.
Zswap can also be enabled and disabled at runtime using the sysfs interface.
An example command to enable zswap at runtime, assuming sysfs is mounted
at ``/sys``, is::

	echo 1 > /sys/module/zswap/parameters/enabled

When zswap is disabled at runtime it will stop storing pages that are
being swapped out.  However, it will _not_ immediately write out or fault
back into memory all of the pages stored in the compressed pool.  The
pages stored in zswap will remain in the compressed pool until they are
either invalidated or faulted back into memory.  In order to force all
pages out of the compressed pool, a swapoff on the swap device(s) will
fault back into memory all swapped out pages, including those in the
compressed pool.

Design
======

Zswap receives pages for compression through the Frontswap API and is able to
evict pages from its own compressed pool on an LRU basis and write them back to
the backing swap device in the case that the compressed pool is full.

Zswap makes use of zpool for the managing the compressed memory pool.  Each
allocation in zpool is not directly accessible by address.  Rather, a handle is
returned by the allocation routine and that handle must be mapped before being
accessed.  The compressed memory pool grows on demand and shrinks as compressed
pages are freed.  The pool is not preallocated.  By default, a zpool
of type selected in ``CONFIG_ZSWAP_ZPOOL_DEFAULT`` Kconfig option is created,
but it can be overridden at boot time by setting the ``zpool`` attribute,
e.g. ``zswap.zpool=zbud``. It can also be changed at runtime using the sysfs
``zpool`` attribute, e.g.::

	echo zbud > /sys/module/zswap/parameters/zpool

The zbud type zpool allocates exactly 1 page to store 2 compressed pages, which
means the compression ratio will always be 2:1 or worse (because of half-full
zbud pages).  The zsmalloc type zpool has a more complex compressed page
storage method, and it can achieve greater storage densities.  However,
zsmalloc does not implement compressed page eviction, so once zswap fills it
cannot evict the oldest page, it can only reject new pages.

When a swap page is passed from frontswap to zswap, zswap maintains a mapping
of the swap entry, a combination of the swap type and swap offset, to the zpool
handle that references that compressed swap page.  This mapping is achieved
with a red-black tree per swap type.  The swap offset is the search key for the
tree nodes.

During a page fault on a PTE that is a swap entry, frontswap calls the zswap
load function to decompress the page into the page allocated by the page fault
handler.

Once there are no PTEs referencing a swap page stored in zswap (i.e. the count
in the swap_map goes to 0) the swap code calls the zswap invalidate function,
via frontswap, to free the compressed entry.

Zswap seeks to be simple in its policies.  Sysfs attributes allow for one user
controlled policy:

* max_pool_percent - The maximum percentage of memory that the compressed
  pool can occupy.

The default compressor is selected in ``CONFIG_ZSWAP_COMPRESSOR_DEFAULT``
Kconfig option, but it can be overridden at boot time by setting the
``compressor`` attribute, e.g. ``zswap.compressor=lzo``.
It can also be changed at runtime using the sysfs "compressor"
attribute, e.g.::

	echo lzo > /sys/module/zswap/parameters/compressor

When the zpool and/or compressor parameter is changed at runtime, any existing
compressed pages are not modified; they are left in their own zpool.  When a
request is made for a page in an old zpool, it is uncompressed using its
original compressor.  Once all pages are removed from an old zpool, the zpool
and its compressor are freed.

Some of the pages in zswap are same-value filled pages (i.e. contents of the
page have same value or repetitive pattern). These pages include zero-filled
pages and they are handled differently. During store operation, a page is
checked if it is a same-value filled page before compressing it. If true, the
compressed length of the page is set to zero and the pattern or same-filled
value is stored.

Same-value filled pages identification feature is enabled by default and can be
disabled at boot time by setting the ``same_filled_pages_enabled`` attribute
to 0, e.g. ``zswap.same_filled_pages_enabled=0``. It can also be enabled and
disabled at runtime using the sysfs ``same_filled_pages_enabled``
attribute, e.g.::

	echo 1 > /sys/module/zswap/parameters/same_filled_pages_enabled

When zswap same-filled page identification is disabled at runtime, it will stop
checking for the same-value filled pages during store operation.
In other words, every page will be then considered non-same-value filled.
However, the existing pages which are marked as same-value filled pages remain
stored unchanged in zswap until they are either loaded or invalidated.

In some circumstances it might be advantageous to make use of just the zswap
ability to efficiently store same-filled pages without enabling the whole
compressed page storage.
In this case the handling of non-same-value pages by zswap (enabled by default)
can be disabled by setting the ``non_same_filled_pages_enabled`` attribute
to 0, e.g. ``zswap.non_same_filled_pages_enabled=0``.
It can also be enabled and disabled at runtime using the sysfs
``non_same_filled_pages_enabled`` attribute, e.g.::

	echo 1 > /sys/module/zswap/parameters/non_same_filled_pages_enabled

Disabling both ``zswap.same_filled_pages_enabled`` and
``zswap.non_same_filled_pages_enabled`` effectively disables accepting any new
pages by zswap.

To prevent zswap from shrinking pool when zswap is full and there's a high
pressure on swap (this will result in flipping pages in and out zswap pool
without any real benefit but with a performance drop for the system), a
special parameter has been introduced to implement a sort of hysteresis to
refuse taking pages into zswap pool until it has sufficient space if the limit
has been hit. To set the threshold at which zswap would start accepting pages
again after it became full, use the sysfs ``accept_threshold_percent``
attribute, e. g.::

	echo 80 > /sys/module/zswap/parameters/accept_threshold_percent

Setting this parameter to 100 will disable the hysteresis.

A debugfs interface is provided for various statistic about pool size, number
of pages stored, same-value filled pages and various counters for the reasons
pages are rejected.

repo_name: angery/linux


code:
Kernel module signing facility
------------------------------

.. CONTENTS
..
.. - Overview.
.. - Configuring module signing.
.. - Generating signing keys.
.. - Public keys in the kernel.
.. - Manually signing modules.
.. - Signed modules and stripping.
.. - Loading signed modules.
.. - Non-valid signatures and unsigned modules.
.. - Administering/protecting the private key.

========
Overview
========

The kernel module signing facility cryptographically signs modules during
installation and then checks the signature upon loading the module.  This
allows increased kernel security by disallowing the loading of unsigned modules
or modules signed with an invalid key.  Module signing increases security by
making it harder to load a malicious module into the kernel.  The module
signature checking is done by the kernel so that it is not necessary to have
trusted userspace bits.

This facility uses X.509 ITU-T standard certificates to encode the public keys
involved.  The signatures are not themselves encoded in any industrial standard
type.  The facility currently only supports the RSA public key encryption
standard (though it is pluggable and permits others to be used).  The possible
hash algorithms that can be used are SHA-1, SHA-224, SHA-256, SHA-384, and
SHA-512 (the algorithm is selected by data in the signature).

==========================
Configuring module signing
==========================

The module signing facility is enabled by going to the
:menuselection:`Enable Loadable Module Support` section of
the kernel configuration and turning on::

	CONFIG_MODULE_SIG	"Module signature verification"

This has a number of options available:

 (1) :menuselection:`Require modules to be validly signed`
     (``CONFIG_MODULE_SIG_FORCE``)

     This specifies how the kernel should deal with a module that has a
     signature for which the key is not known or a module that is unsigned.

     If this is off (ie. "permissive"), then modules for which the key is not
     available and modules that are unsigned are permitted, but the kernel will
     be marked as being tainted, and the concerned modules will be marked as
     tainted, shown with the character 'E'.

     If this is on (ie. "restrictive"), only modules that have a valid
     signature that can be verified by a public key in the kernel's possession
     will be loaded.  All other modules will generate an error.

     Irrespective of the setting here, if the module has a signature block that
     cannot be parsed, it will be rejected out of hand.

 (2) :menuselection:`Automatically sign all modules`
     (``CONFIG_MODULE_SIG_ALL``)

     If this is on then modules will be automatically signed during the
     modules_install phase of a build.  If this is off, then the modules must
     be signed manually using::

	scripts/sign-file

 (3) :menuselection:`Which hash algorithm should modules be signed with?`

     This presents a choice of which hash algorithm the installation phase will
     sign the modules with:

        =============================== ==========================================
	``CONFIG_MODULE_SIG_SHA1``	:menuselection:`Sign modules with SHA-1`
	``CONFIG_MODULE_SIG_SHA224``	:menuselection:`Sign modules with SHA-224`
	``CONFIG_MODULE_SIG_SHA256``	:menuselection:`Sign modules with SHA-256`
	``CONFIG_MODULE_SIG_SHA384``	:menuselection:`Sign modules with SHA-384`
	``CONFIG_MODULE_SIG_SHA512``	:menuselection:`Sign modules with SHA-512`
        =============================== ==========================================

     The algorithm selected here will also be built into the kernel (rather
     than being a module) so that modules signed with that algorithm can have
     their signatures checked without causing a dependency loop.

 (4) :menuselection:`File name or PKCS#11 URI of module signing key`
     (``CONFIG_MODULE_SIG_KEY``)

     Setting this option to something other than its default of
     ``certs/signing_key.pem`` will disable the autogeneration of signing keys
     and allow the kernel modules to be signed with a key of your choosing.
     The string provided should identify a file containing both a private key
     and its corresponding X.509 certificate in PEM form, or — on systems where
     the OpenSSL ENGINE_pkcs11 is functional — a PKCS#11 URI as defined by
     RFC7512. In the latter case, the PKCS#11 URI should reference both a
     certificate and a private key.

     If the PEM file containing the private key is encrypted, or if the
     PKCS#11 token requires a PIN, this can be provided at build time by
     means of the ``KBUILD_SIGN_PIN`` variable.

 (5) :menuselection:`Additional X.509 keys for default system keyring`
     (``CONFIG_SYSTEM_TRUSTED_KEYS``)

     This option can be set to the filename of a PEM-encoded file containing
     additional certificates which will be included in the system keyring by
     default.

Note that enabling module signing adds a dependency on the OpenSSL devel
packages to the kernel build processes for the tool that does the signing.

=======================
Generating signing keys
=======================

Cryptographic keypairs are required to generate and check signatures.  A
private key is used to generate a signature and the corresponding public key is
used to check it.  The private key is only needed during the build, after which
it can be deleted or stored securely.  The public key gets built into the
kernel so that it can be used to check the signatures as the modules are
loaded.

Under normal conditions, when ``CONFIG_MODULE_SIG_KEY`` is unchanged from its
default, the kernel build will automatically generate a new keypair using
openssl if one does not exist in the file::

	certs/signing_key.pem

during the building of vmlinux (the public part of the key needs to be built
into vmlinux) using parameters in the::

	certs/x509.genkey

file (which is also generated if it does not already exist).

It is strongly recommended that you provide your own x509.genkey file.

Most notably, in the x509.genkey file, the req_distinguished_name section
should be altered from the default::

	[ req_distinguished_name ]
	#O = Unspecified company
	CN = Build time autogenerated kernel key
	#emailAddress = unspecified.user@unspecified.company

The generated RSA key size can also be set with::

	[ req ]
	default_bits = 4096

It is also possible to manually generate the key private/public files using the
x509.genkey key generation configuration file in the root node of the Linux
kernel sources tree and the openssl command.  The following is an example to
generate the public/private key files::

	openssl req -new -nodes -utf8 -sha256 -days 36500 -batch -x509 \
	   -config x509.genkey -outform PEM -out kernel_key.pem \
	   -keyout kernel_key.pem

The full pathname for the resulting kernel_key.pem file can then be specified
in the ``CONFIG_MODULE_SIG_KEY`` option, and the certificate and key therein will
be used instead of an autogenerated keypair.

=========================
Public keys in the kernel
=========================

The kernel contains a ring of public keys that can be viewed by root.  They're
in a keyring called ".builtin_trusted_keys" that can be seen by::

	[root@deneb ~]# cat /proc/keys
	...
	223c7853 I------     1 perm 1f030000     0     0 keyring   .builtin_trusted_keys: 1
	302d2d52 I------     1 perm 1f010000     0     0 asymmetri Fedora kernel signing key: d69a84e6bce3d216b979e9505b3e3ef9a7118079: X509.RSA a7118079 []
	...

Beyond the public key generated specifically for module signing, additional
trusted certificates can be provided in a PEM-encoded file referenced by the
``CONFIG_SYSTEM_TRUSTED_KEYS`` configuration option.

Further, the architecture code may take public keys from a hardware store and
add those in also (e.g. from the UEFI key database).

Finally, it is possible to add additional public keys by doing::

	keyctl padd asymmetric "" [.builtin_trusted_keys-ID] <[key-file]

e.g.::

	keyctl padd asymmetric "" 0x223c7853 <my_public_key.x509

Note, however, that the kernel will only permit keys to be added to
``.builtin_trusted_keys`` **if** the new key's X.509 wrapper is validly signed by a key
that is already resident in the ``.builtin_trusted_keys`` at the time the key was added.

========================
Manually signing modules
========================

To manually sign a module, use the scripts/sign-file tool available in
the Linux kernel source tree.  The script requires 4 arguments:

	1.  The hash algorithm (e.g., sha256)
	2.  The private key filename or PKCS#11 URI
	3.  The public key filename
	4.  The kernel module to be signed

The following is an example to sign a kernel module::

	scripts/sign-file sha512 kernel-signkey.priv \
		kernel-signkey.x509 module.ko

The hash algorithm used does not have to match the one configured, but if it
doesn't, you should make sure that hash algorithm is either built into the
kernel or can be loaded without requiring itself.

If the private key requires a passphrase or PIN, it can be provided in the
$KBUILD_SIGN_PIN environment variable.

============================
Signed modules and stripping
============================

A signed module has a digital signature simply appended at the end.  The string
``~Module signature appended~.`` at the end of the module's file confirms that a
signature is present but it does not confirm that the signature is valid!

Signed modules are BRITTLE as the signature is outside of the defined ELF
container.  Thus they MAY NOT be stripped once the signature is computed and
attached.  Note the entire module is the signed payload, including any and all
debug information present at the time of signing.

======================
Loading signed modules
======================

Modules are loaded with insmod, modprobe, ``init_module()`` or
``finit_module()``, exactly as for unsigned modules as no processing is
done in userspace.  The signature checking is all done within the kernel.

=========================================
Non-valid signatures and unsigned modules
=========================================

If ``CONFIG_MODULE_SIG_FORCE`` is enabled or module.sig_enforce=1 is supplied on
the kernel command line, the kernel will only load validly signed modules
for which it has a public key.   Otherwise, it will also load modules that are
unsigned.   Any module for which the kernel has a key, but which proves to have
a signature mismatch will not be permitted to load.

Any module that has an unparseable signature will be rejected.

=========================================
Administering/protecting the private key
=========================================

Since the private key is used to sign modules, viruses and malware could use
the private key to sign modules and compromise the operating system.  The
private key must be either destroyed or moved to a secure location and not kept
in the root node of the kernel source tree.

If you use the same private key to sign modules for multiple kernel
configurations, you must ensure that the module version information is
sufficient to prevent loading a module into a different kernel.  Either
set ``CONFIG_MODVERSIONS=y`` or ensure that each configuration has a different
kernel release string by changing ``EXTRAVERSION`` or ``CONFIG_LOCALVERSION``.


code:
Mono(tm) Binary Kernel Support for Linux
-----------------------------------------

To configure Linux to automatically execute Mono-based .NET binaries
(in the form of .exe files) without the need to use the mono CLR
wrapper, you can use the BINFMT_MISC kernel support.

This will allow you to execute Mono-based .NET binaries just like any
other program after you have done the following:

1) You MUST FIRST install the Mono CLR support, either by downloading
   a binary package, a source tarball or by installing from Git. Binary
   packages for several distributions can be found at:

	https://www.mono-project.com/download/

   Instructions for compiling Mono can be found at:

	https://www.mono-project.com/docs/compiling-mono/linux/

   Once the Mono CLR support has been installed, just check that
   ``/usr/bin/mono`` (which could be located elsewhere, for example
   ``/usr/local/bin/mono``) is working.

2) You have to compile BINFMT_MISC either as a module or into
   the kernel (``CONFIG_BINFMT_MISC``) and set it up properly.
   If you choose to compile it as a module, you will have
   to insert it manually with modprobe/insmod, as kmod
   cannot be easily supported with binfmt_misc.
   Read the file ``binfmt_misc.txt`` in this directory to know
   more about the configuration process.

3) Add the following entries to ``/etc/rc.local`` or similar script
   to be run at system startup:

   .. code-block:: sh

    # Insert BINFMT_MISC module into the kernel
    if [ ! -e /proc/sys/fs/binfmt_misc/register ]; then
        /sbin/modprobe binfmt_misc
	# Some distributions, like Fedora Core, perform
	# the following command automatically when the
	# binfmt_misc module is loaded into the kernel
	# or during normal boot up (systemd-based systems).
	# Thus, it is possible that the following line
	# is not needed at all.
	mount -t binfmt_misc none /proc/sys/fs/binfmt_misc
    fi

    # Register support for .NET CLR binaries
    if [ -e /proc/sys/fs/binfmt_misc/register ]; then
	# Replace /usr/bin/mono with the correct pathname to
	# the Mono CLR runtime (usually /usr/local/bin/mono
	# when compiling from sources or CVS).
        echo ':CLR:M::MZ::/usr/bin/mono:' > /proc/sys/fs/binfmt_misc/register
    else
        echo "No binfmt_misc support"
        exit 1
    fi

4) Check that ``.exe`` binaries can be ran without the need of a
   wrapper script, simply by launching the ``.exe`` file directly
   from a command prompt, for example::

	/usr/bin/xsd.exe

   .. note::

      If this fails with a permission denied error, check
      that the ``.exe`` file has execute permissions.


code:
=============================
Namespaces compatibility list
=============================

This document contains the information about the problems user
may have when creating tasks living in different namespaces.

Here's the summary. This matrix shows the known problems, that
occur when tasks share some namespace (the columns) while living
in different other namespaces (the rows):

====	===	===	===	===	====	===
-	UTS	IPC	VFS	PID	User	Net
====	===	===	===	===	====	===
UTS	 X
IPC		 X	 1
VFS			 X
PID		 1	 1	 X
User		 2	 2		 X
Net						 X
====	===	===	===	===	====	===

1. Both the IPC and the PID namespaces provide IDs to address
   object inside the kernel. E.g. semaphore with IPCID or
   process group with pid.

   In both cases, tasks shouldn't try exposing this ID to some
   other task living in a different namespace via a shared filesystem
   or IPC shmem/message. The fact is that this ID is only valid
   within the namespace it was obtained in and may refer to some
   other object in another namespace.

2. Intentionally, two equal user IDs in different user namespaces
   should not be equal from the VFS point of view. In other
   words, user 10 in one user namespace shouldn't have the same
   access permissions to files, belonging to user 10 in another
   namespace.

   The same is true for the IPC namespaces being shared - two users
   from different user namespaces should not access the same IPC objects
   even having equal UIDs.

   But currently this is not so.


code:
.. SPDX-License-Identifier: GPL-2.0

==========
Namespaces
==========

.. toctree::
   :maxdepth: 1

   compatibility-list
   resource-control


code:
===========================
Namespaces research control
===========================

There are a lot of kinds of objects in the kernel that don't have
individual limits or that have limits that are ineffective when a set
of processes is allowed to switch user ids.  With user namespaces
enabled in a kernel for people who don't trust their users or their
users programs to play nice this problems becomes more acute.

Therefore it is recommended that memory control groups be enabled in
kernels that enable user namespaces, and it is further recommended
that userspace configure memory control groups to limit how much
memory user's they don't trust to play nice can use.

Memory control groups can be configured by installing the libcgroup
package present on most distros editing /etc/cgrules.conf,
/etc/cgconfig.conf and setting up libpam-cgroup.


code:
=============
NFS
=============

.. toctree::
    :maxdepth: 1

    nfs-client
    nfsroot
    nfs-rdma
    nfsd-admin-interfaces
    nfs-idmapper
    pnfs-block-server
    pnfs-scsi-server


code:
==========
NFS Client
==========

The NFS client
==============

The NFS version 2 protocol was first documented in RFC1094 (March 1989).
Since then two more major releases of NFS have been published, with NFSv3
being documented in RFC1813 (June 1995), and NFSv4 in RFC3530 (April
2003).

The Linux NFS client currently supports all the above published versions,
and work is in progress on adding support for minor version 1 of the NFSv4
protocol.

The purpose of this document is to provide information on some of the
special features of the NFS client that can be configured by system
administrators.

The nfs4_unique_id parameter
============================

NFSv4 requires clients to identify themselves to servers with a unique
string.  File open and lock state shared between one client and one server
is associated with this identity.  To support robust NFSv4 state recovery
and transparent state migration, this identity string must not change
across client reboots.

Without any other intervention, the Linux client uses a string that contains
the local system's node name.  System administrators, however, often do not
take care to ensure that node names are fully qualified and do not change
over the lifetime of a client system.  Node names can have other
administrative requirements that require particular behavior that does not
work well as part of an nfs_client_id4 string.

The nfs.nfs4_unique_id boot parameter specifies a unique string that can be
used together with  a system's node name when an NFS client identifies itself to
a server.  Thus, if the system's node name is not unique, its
nfs.nfs4_unique_id can help prevent collisions with other clients.

The nfs.nfs4_unique_id string is typically a UUID, though it can contain
anything that is believed to be unique across all NFS clients.  An
nfs4_unique_id string should be chosen when a client system is installed,
just as a system's root file system gets a fresh UUID in its label at
install time.

The string should remain fixed for the lifetime of the client.  It can be
changed safely if care is taken that the client shuts down cleanly and all
outstanding NFSv4 state has expired, to prevent loss of NFSv4 state.

This string can be stored in an NFS client's grub.conf, or it can be provided
via a net boot facility such as PXE.  It may also be specified as an nfs.ko
module parameter.

This uniquifier string will be the same for all NFS clients running in
containers unless it is overridden by a value written to
/sys/fs/nfs/net/nfs_client/identifier which will be local to the network
namespace of the process which writes.

The DNS resolver
================

NFSv4 allows for one server to refer the NFS client to data that has been
migrated onto another server by means of the special "fs_locations"
attribute. See `RFC3530 Section 6: Filesystem Migration and Replication`_ and
`Implementation Guide for Referrals in NFSv4`_.

.. _RFC3530 Section 6\: Filesystem Migration and Replication: https://tools.ietf.org/html/rfc3530#section-6
.. _Implementation Guide for Referrals in NFSv4: https://tools.ietf.org/html/draft-ietf-nfsv4-referrals-00

The fs_locations information can take the form of either an ip address and
a path, or a DNS hostname and a path. The latter requires the NFS client to
do a DNS lookup in order to mount the new volume, and hence the need for an
upcall to allow userland to provide this service.

Assuming that the user has the 'rpc_pipefs' filesystem mounted in the usual
/var/lib/nfs/rpc_pipefs, the upcall consists of the following steps:

   (1) The process checks the dns_resolve cache to see if it contains a
       valid entry. If so, it returns that entry and exits.

   (2) If no valid entry exists, the helper script '/sbin/nfs_cache_getent'
       (may be changed using the 'nfs.cache_getent' kernel boot parameter)
       is run, with two arguments:
       - the cache name, "dns_resolve"
       - the hostname to resolve

   (3) After looking up the corresponding ip address, the helper script
       writes the result into the rpc_pipefs pseudo-file
       '/var/lib/nfs/rpc_pipefs/cache/dns_resolve/channel'
       in the following (text) format:

		"<ip address> <hostname> <ttl>\n"

       Where <ip address> is in the usual IPv4 (123.456.78.90) or IPv6
       (ffee:ddcc:bbaa:9988:7766:5544:3322:1100, ffee::1100, ...) format.
       <hostname> is identical to the second argument of the helper
       script, and <ttl> is the 'time to live' of this cache entry (in
       units of seconds).

       .. note::
            If <ip address> is invalid, say the string "0", then a negative
            entry is created, which will cause the kernel to treat the hostname
            as having no valid DNS translation.

A basic sample /sbin/nfs_cache_getent
=====================================
.. code-block:: sh

    #!/bin/bash
    #
    ttl=600
    #
    cut=/usr/bin/cut
    getent=/usr/bin/getent
    rpc_pipefs=/var/lib/nfs/rpc_pipefs
    #
    die()
    {
        echo "Usage: $0 cache_name entry_name"
        exit 1
    }

    [ $# -lt 2 ] && die
    cachename="$1"
    cache_path=${rpc_pipefs}/cache/${cachename}/channel

    case "${cachename}" in
        dns_resolve)
            name="$2"
            result="$(${getent} hosts ${name} | ${cut} -f1 -d\ )"
            [ -z "${result}" ] && result="0"
            ;;
        *)
            die
            ;;
    esac
    echo "${result} ${name} ${ttl}" >${cache_path}


code:
=============
NFS ID Mapper
=============

Id mapper is used by NFS to translate user and group ids into names, and to
translate user and group names into ids.  Part of this translation involves
performing an upcall to userspace to request the information.  There are two
ways NFS could obtain this information: placing a call to /sbin/request-key
or by placing a call to the rpc.idmap daemon.

NFS will attempt to call /sbin/request-key first.  If this succeeds, the
result will be cached using the generic request-key cache.  This call should
only fail if /etc/request-key.conf is not configured for the id_resolver key
type, see the "Configuring" section below if you wish to use the request-key
method.

If the call to /sbin/request-key fails (if /etc/request-key.conf is not
configured with the id_resolver key type), then the idmapper will ask the
legacy rpc.idmap daemon for the id mapping.  This result will be stored
in a custom NFS idmap cache.

Configuring
===========

The file /etc/request-key.conf will need to be modified so /sbin/request-key can
direct the upcall.  The following line should be added:

``#OP	TYPE	DESCRIPTION	CALLOUT INFO	PROGRAM ARG1 ARG2 ARG3 ...``
``#======	=======	===============	===============	===============================``
``create	id_resolver	*	*		/usr/sbin/nfs.idmap %k %d 600``

This will direct all id_resolver requests to the program /usr/sbin/nfs.idmap.
The last parameter, 600, defines how many seconds into the future the key will
expire.  This parameter is optional for /usr/sbin/nfs.idmap.  When the timeout
is not specified, nfs.idmap will default to 600 seconds.

id mapper uses for key descriptions::

	  uid:  Find the UID for the given user
	  gid:  Find the GID for the given group
	 user:  Find the user  name for the given UID
	group:  Find the group name for the given GID

You can handle any of these individually, rather than using the generic upcall
program.  If you would like to use your own program for a uid lookup then you
would edit your request-key.conf so it look similar to this:

``#OP	TYPE	DESCRIPTION	CALLOUT INFO	PROGRAM ARG1 ARG2 ARG3 ...``
``#======	=======	===============	===============	===============================``
``create	id_resolver	uid:*	*		/some/other/program %k %d 600``
``create	id_resolver	*	*		/usr/sbin/nfs.idmap %k %d 600``

Notice that the new line was added above the line for the generic program.
request-key will find the first matching line and corresponding program.  In
this case, /some/other/program will handle all uid lookups and
/usr/sbin/nfs.idmap will handle gid, user, and group lookups.

See Documentation/security/keys/request-key.rst for more information
about the request-key function.

nfs.idmap
=========

nfs.idmap is designed to be called by request-key, and should not be run "by
hand".  This program takes two arguments, a serialized key and a key
description.  The serialized key is first converted into a key_serial_t, and
then passed as an argument to keyctl_instantiate (both are part of keyutils.h).

The actual lookups are performed by functions found in nfsidmap.h.  nfs.idmap
determines the correct function to call by looking at the first part of the
description string.  For example, a uid lookup description will appear as
"uid:user@domain".

nfs.idmap will return 0 if the key was instantiated, and non-zero otherwise.


code:
===================
Setting up NFS/RDMA
===================

:Author:
  NetApp and Open Grid Computing (May 29, 2008)

.. warning::
  This document is probably obsolete.

Overview
========

This document describes how to install and setup the Linux NFS/RDMA client
and server software.

The NFS/RDMA client was first included in Linux 2.6.24. The NFS/RDMA server
was first included in the following release, Linux 2.6.25.

In our testing, we have obtained excellent performance results (full 10Gbit
wire bandwidth at minimal client CPU) under many workloads. The code passes
the full Connectathon test suite and operates over both Infiniband and iWARP
RDMA adapters.

Getting Help
============

If you get stuck, you can ask questions on the
nfs-rdma-devel@lists.sourceforge.net mailing list.

Installation
============

These instructions are a step by step guide to building a machine for
use with NFS/RDMA.

- Install an RDMA device

  Any device supported by the drivers in drivers/infiniband/hw is acceptable.

  Testing has been performed using several Mellanox-based IB cards, the
  Ammasso AMS1100 iWARP adapter, and the Chelsio cxgb3 iWARP adapter.

- Install a Linux distribution and tools

  The first kernel release to contain both the NFS/RDMA client and server was
  Linux 2.6.25  Therefore, a distribution compatible with this and subsequent
  Linux kernel release should be installed.

  The procedures described in this document have been tested with
  distributions from Red Hat's Fedora Project (http://fedora.redhat.com/).

- Install nfs-utils-1.1.2 or greater on the client

  An NFS/RDMA mount point can be obtained by using the mount.nfs command in
  nfs-utils-1.1.2 or greater (nfs-utils-1.1.1 was the first nfs-utils
  version with support for NFS/RDMA mounts, but for various reasons we
  recommend using nfs-utils-1.1.2 or greater). To see which version of
  mount.nfs you are using, type:

  .. code-block:: sh

    $ /sbin/mount.nfs -V

  If the version is less than 1.1.2 or the command does not exist,
  you should install the latest version of nfs-utils.

  Download the latest package from: https://www.kernel.org/pub/linux/utils/nfs

  Uncompress the package and follow the installation instructions.

  If you will not need the idmapper and gssd executables (you do not need
  these to create an NFS/RDMA enabled mount command), the installation
  process can be simplified by disabling these features when running
  configure:

  .. code-block:: sh

    $ ./configure --disable-gss --disable-nfsv4

  To build nfs-utils you will need the tcp_wrappers package installed. For
  more information on this see the package's README and INSTALL files.

  After building the nfs-utils package, there will be a mount.nfs binary in
  the utils/mount directory. This binary can be used to initiate NFS v2, v3,
  or v4 mounts. To initiate a v4 mount, the binary must be called
  mount.nfs4.  The standard technique is to create a symlink called
  mount.nfs4 to mount.nfs.

  This mount.nfs binary should be installed at /sbin/mount.nfs as follows:

  .. code-block:: sh

    $ sudo cp utils/mount/mount.nfs /sbin/mount.nfs

  In this location, mount.nfs will be invoked automatically for NFS mounts
  by the system mount command.

    .. note::
      mount.nfs and therefore nfs-utils-1.1.2 or greater is only needed
      on the NFS client machine. You do not need this specific version of
      nfs-utils on the server. Furthermore, only the mount.nfs command from
      nfs-utils-1.1.2 is needed on the client.

- Install a Linux kernel with NFS/RDMA

  The NFS/RDMA client and server are both included in the mainline Linux
  kernel version 2.6.25 and later. This and other versions of the Linux
  kernel can be found at: https://www.kernel.org/pub/linux/kernel/

  Download the sources and place them in an appropriate location.

- Configure the RDMA stack

  Make sure your kernel configuration has RDMA support enabled. Under
  Device Drivers -> InfiniBand support, update the kernel configuration
  to enable InfiniBand support [NOTE: the option name is misleading. Enabling
  InfiniBand support is required for all RDMA devices (IB, iWARP, etc.)].

  Enable the appropriate IB HCA support (mlx4, mthca, ehca, ipath, etc.) or
  iWARP adapter support (amso, cxgb3, etc.).

  If you are using InfiniBand, be sure to enable IP-over-InfiniBand support.

- Configure the NFS client and server

  Your kernel configuration must also have NFS file system support and/or
  NFS server support enabled. These and other NFS related configuration
  options can be found under File Systems -> Network File Systems.

- Build, install, reboot

  The NFS/RDMA code will be enabled automatically if NFS and RDMA
  are turned on. The NFS/RDMA client and server are configured via the hidden
  SUNRPC_XPRT_RDMA config option that depends on SUNRPC and INFINIBAND. The
  value of SUNRPC_XPRT_RDMA will be:

    #. N if either SUNRPC or INFINIBAND are N, in this case the NFS/RDMA client
       and server will not be built

    #. M if both SUNRPC and INFINIBAND are on (M or Y) and at least one is M,
       in this case the NFS/RDMA client and server will be built as modules

    #. Y if both SUNRPC and INFINIBAND are Y, in this case the NFS/RDMA client
       and server will be built into the kernel

  Therefore, if you have followed the steps above and turned no NFS and RDMA,
  the NFS/RDMA client and server will be built.

  Build a new kernel, install it, boot it.

Check RDMA and NFS Setup
========================

Before configuring the NFS/RDMA software, it is a good idea to test
your new kernel to ensure that the kernel is working correctly.
In particular, it is a good idea to verify that the RDMA stack
is functioning as expected and standard NFS over TCP/IP and/or UDP/IP
is working properly.

- Check RDMA Setup

  If you built the RDMA components as modules, load them at
  this time. For example, if you are using a Mellanox Tavor/Sinai/Arbel
  card:

  .. code-block:: sh

    $ modprobe ib_mthca
    $ modprobe ib_ipoib

  If you are using InfiniBand, make sure there is a Subnet Manager (SM)
  running on the network. If your IB switch has an embedded SM, you can
  use it. Otherwise, you will need to run an SM, such as OpenSM, on one
  of your end nodes.

  If an SM is running on your network, you should see the following:

  .. code-block:: sh

    $ cat /sys/class/infiniband/driverX/ports/1/state
    4: ACTIVE

  where driverX is mthca0, ipath5, ehca3, etc.

  To further test the InfiniBand software stack, use IPoIB (this
  assumes you have two IB hosts named host1 and host2):

  .. code-block:: sh

    host1$ ip link set dev ib0 up
    host1$ ip address add dev ib0 a.b.c.x
    host2$ ip link set dev ib0 up
    host2$ ip address add dev ib0 a.b.c.y
    host1$ ping a.b.c.y
    host2$ ping a.b.c.x

  For other device types, follow the appropriate procedures.

- Check NFS Setup

  For the NFS components enabled above (client and/or server),
  test their functionality over standard Ethernet using TCP/IP or UDP/IP.

NFS/RDMA Setup
==============

We recommend that you use two machines, one to act as the client and
one to act as the server.

One time configuration:
-----------------------

- On the server system, configure the /etc/exports file and start the NFS/RDMA server.

  Exports entries with the following formats have been tested::

  /vol0   192.168.0.47(fsid=0,rw,async,insecure,no_root_squash)
  /vol0   192.168.0.0/255.255.255.0(fsid=0,rw,async,insecure,no_root_squash)

  The IP address(es) is(are) the client's IPoIB address for an InfiniBand
  HCA or the client's iWARP address(es) for an RNIC.

  .. note::
    The "insecure" option must be used because the NFS/RDMA client does
    not use a reserved port.

Each time a machine boots:
--------------------------

- Load and configure the RDMA drivers

  For InfiniBand using a Mellanox adapter:

  .. code-block:: sh

    $ modprobe ib_mthca
    $ modprobe ib_ipoib
    $ ip li set dev ib0 up
    $ ip addr add dev ib0 a.b.c.d

  .. note::
    Please use unique addresses for the client and server!

- Start the NFS server

  If the NFS/RDMA server was built as a module (CONFIG_SUNRPC_XPRT_RDMA=m in
  kernel config), load the RDMA transport module:

  .. code-block:: sh

    $ modprobe svcrdma

  Regardless of how the server was built (module or built-in), start the
  server:

  .. code-block:: sh

    $ /etc/init.d/nfs start

  or

  .. code-block:: sh

    $ service nfs start

  Instruct the server to listen on the RDMA transport:

  .. code-block:: sh

    $ echo rdma 20049 > /proc/fs/nfsd/portlist

- On the client system

  If the NFS/RDMA client was built as a module (CONFIG_SUNRPC_XPRT_RDMA=m in
  kernel config), load the RDMA client module:

  .. code-block:: sh

    $ modprobe xprtrdma.ko

  Regardless of how the client was built (module or built-in), use this
  command to mount the NFS/RDMA server:

  .. code-block:: sh

    $ mount -o rdma,port=20049 <IPoIB-server-name-or-address>:/<export> /mnt

  To verify that the mount is using RDMA, run "cat /proc/mounts" and check
  the "proto" field for the given mount.

  Congratulations! You're using NFS/RDMA!


code:
==================================
Administrative interfaces for nfsd
==================================

Note that normally these interfaces are used only by the utilities in
nfs-utils.

nfsd is controlled mainly by pseudofiles under the "nfsd" filesystem,
which is normally mounted at /proc/fs/nfsd/.

The server is always started by the first write of a nonzero value to
nfsd/threads.

Before doing that, NFSD can be told which sockets to listen on by
writing to nfsd/portlist; that write may be:

	-  an ascii-encoded file descriptor, which should refer to a
	   bound (and listening, for tcp) socket, or
	-  "transportname port", where transportname is currently either
	   "udp", "tcp", or "rdma".

If nfsd is started without doing any of these, then it will create one
udp and one tcp listener at port 2049 (see nfsd_init_socks).

On startup, nfsd and lockd grace periods start. nfsd is shut down by a write of
0 to nfsd/threads.  All locks and state are thrown away at that point.

Between startup and shutdown, the number of threads may be adjusted up
or down by additional writes to nfsd/threads or by writes to
nfsd/pool_threads.

For more detail about files under nfsd/ and what they control, see
fs/nfsd/nfsctl.c; most of them have detailed comments.

Implementation notes
====================

Note that the rpc server requires the caller to serialize addition and
removal of listening sockets, and startup and shutdown of the server.
For nfsd this is done using nfsd_mutex.


code:
===============================================
Mounting the root filesystem via NFS (nfsroot)
===============================================

:Authors:
	Written 1996 by Gero Kuhlmann <gero@gkminix.han.de>

	Updated 1997 by Martin Mares <mj@atrey.karlin.mff.cuni.cz>

	Updated 2006 by Nico Schottelius <nico-kernel-nfsroot@schottelius.org>

	Updated 2006 by Horms <horms@verge.net.au>

	Updated 2018 by Chris Novakovic <chris@chrisn.me.uk>

In order to use a diskless system, such as an X-terminal or printer server for
example, it is necessary for the root filesystem to be present on a non-disk
device. This may be an initramfs (see
Documentation/filesystems/ramfs-rootfs-initramfs.rst), a ramdisk (see
Documentation/admin-guide/initrd.rst) or a filesystem mounted via NFS. The
following text describes on how to use NFS for the root filesystem. For the rest
of this text 'client' means the diskless system, and 'server' means the NFS
server.

Enabling nfsroot capabilities
=============================

In order to use nfsroot, NFS client support needs to be selected as
built-in during configuration. Once this has been selected, the nfsroot
option will become available, which should also be selected.

In the networking options, kernel level autoconfiguration can be selected,
along with the types of autoconfiguration to support. Selecting all of
DHCP, BOOTP and RARP is safe.

Kernel command line
===================

When the kernel has been loaded by a boot loader (see below) it needs to be
told what root fs device to use. And in the case of nfsroot, where to find
both the server and the name of the directory on the server to mount as root.
This can be established using the following kernel command line parameters:

root=/dev/nfs
  This is necessary to enable the pseudo-NFS-device. Note that it's not a
  real device but just a synonym to tell the kernel to use NFS instead of
  a real device.

nfsroot=[<server-ip>:]<root-dir>[,<nfs-options>]
  If the `nfsroot' parameter is NOT given on the command line,
  the default ``"/tftpboot/%s"`` will be used.

  <server-ip>	Specifies the IP address of the NFS server.
		The default address is determined by the ip parameter
		(see below). This parameter allows the use of different
		servers for IP autoconfiguration and NFS.

  <root-dir>	Name of the directory on the server to mount as root.
		If there is a "%s" token in the string, it will be
		replaced by the ASCII-representation of the client's
		IP address.

  <nfs-options>	Standard NFS options. All options are separated by commas.
		The following defaults are used::

			port		= as given by server portmap daemon
			rsize		= 4096
			wsize		= 4096
			timeo		= 7
			retrans		= 3
			acregmin	= 3
			acregmax	= 60
			acdirmin	= 30
			acdirmax	= 60
			flags		= hard, nointr, noposix, cto, ac

ip=<client-ip>:<server-ip>:<gw-ip>:<netmask>:<hostname>:<device>:<autoconf>:<dns0-ip>:<dns1-ip>:<ntp0-ip>
  This parameter tells the kernel how to configure IP addresses of devices
  and also how to set up the IP routing table. It was originally called
  nfsaddrs, but now the boot-time IP configuration works independently of
  NFS, so it was renamed to ip and the old name remained as an alias for
  compatibility reasons.

  If this parameter is missing from the kernel command line, all fields are
  assumed to be empty, and the defaults mentioned below apply. In general
  this means that the kernel tries to configure everything using
  autoconfiguration.

  The <autoconf> parameter can appear alone as the value to the ip
  parameter (without all the ':' characters before).  If the value is
  "ip=off" or "ip=none", no autoconfiguration will take place, otherwise
  autoconfiguration will take place.  The most common way to use this
  is "ip=dhcp".

  <client-ip>	IP address of the client.
  		Default:  Determined using autoconfiguration.

  <server-ip>	IP address of the NFS server.
		If RARP is used to determine
		the client address and this parameter is NOT empty only
		replies from the specified server are accepted.

		Only required for NFS root. That is autoconfiguration
		will not be triggered if it is missing and NFS root is not
		in operation.

		Value is exported to /proc/net/pnp with the prefix "bootserver "
		(see below).

		Default: Determined using autoconfiguration.
		The address of the autoconfiguration server is used.

  <gw-ip>	IP address of a gateway if the server is on a different subnet.
		Default: Determined using autoconfiguration.

  <netmask>	Netmask for local network interface.
		If unspecified the netmask is derived from the client IP address
		assuming classful addressing.

		Default:  Determined using autoconfiguration.

  <hostname>	Name of the client.
		If a '.' character is present, anything
		before the first '.' is used as the client's hostname, and anything
		after it is used as its NIS domain name. May be supplied by
		autoconfiguration, but its absence will not trigger autoconfiguration.
		If specified and DHCP is used, the user-provided hostname (and NIS
		domain name, if present) will be carried in the DHCP request; this
		may cause a DNS record to be created or updated for the client.

  		Default: Client IP address is used in ASCII notation.

  <device>	Name of network device to use.
		Default: If the host only has one device, it is used.
		Otherwise the device is determined using
		autoconfiguration. This is done by sending
		autoconfiguration requests out of all devices,
		and using the device that received the first reply.

  <autoconf>	Method to use for autoconfiguration.
		In the case of options
		which specify multiple autoconfiguration protocols,
		requests are sent using all protocols, and the first one
		to reply is used.

		Only autoconfiguration protocols that have been compiled
		into the kernel will be used, regardless of the value of
		this option::

                  off or none: don't use autoconfiguration
				(do static IP assignment instead)
		  on or any:   use any protocol available in the kernel
			       (default)
		  dhcp:        use DHCP
		  bootp:       use BOOTP
		  rarp:        use RARP
		  both:        use both BOOTP and RARP but not DHCP
		               (old option kept for backwards compatibility)

		if dhcp is used, the client identifier can be used by following
		format "ip=dhcp,client-id-type,client-id-value"

                Default: any

  <dns0-ip>	IP address of primary nameserver.
		Value is exported to /proc/net/pnp with the prefix "nameserver "
		(see below).

		Default: None if not using autoconfiguration; determined
		automatically if using autoconfiguration.

  <dns1-ip>	IP address of secondary nameserver.
		See <dns0-ip>.

  <ntp0-ip>	IP address of a Network Time Protocol (NTP) server.
		Value is exported to /proc/net/ipconfig/ntp_servers, but is
		otherwise unused (see below).

		Default: None if not using autoconfiguration; determined
		automatically if using autoconfiguration.

  After configuration (whether manual or automatic) is complete, two files
  are created in the following format; lines are omitted if their respective
  value is empty following configuration:

  - /proc/net/pnp:

	#PROTO: <DHCP|BOOTP|RARP|MANUAL>	(depending on configuration method)
	domain <dns-domain>			(if autoconfigured, the DNS domain)
	nameserver <dns0-ip>			(primary name server IP)
	nameserver <dns1-ip>			(secondary name server IP)
	nameserver <dns2-ip>			(tertiary name server IP)
	bootserver <server-ip>			(NFS server IP)

  - /proc/net/ipconfig/ntp_servers:

	<ntp0-ip>				(NTP server IP)
	<ntp1-ip>				(NTP server IP)
	<ntp2-ip>				(NTP server IP)

  <dns-domain> and <dns2-ip> (in /proc/net/pnp) and <ntp1-ip> and <ntp2-ip>
  (in /proc/net/ipconfig/ntp_servers) are requested during autoconfiguration;
  they cannot be specified as part of the "ip=" kernel command line parameter.

  Because the "domain" and "nameserver" options are recognised by DNS
  resolvers, /etc/resolv.conf is often linked to /proc/net/pnp on systems
  that use an NFS root filesystem.

  Note that the kernel will not synchronise the system time with any NTP
  servers it discovers; this is the responsibility of a user space process
  (e.g. an initrd/initramfs script that passes the IP addresses listed in
  /proc/net/ipconfig/ntp_servers to an NTP client before mounting the real
  root filesystem if it is on NFS).

nfsrootdebug
  This parameter enables debugging messages to appear in the kernel
  log at boot time so that administrators can verify that the correct
  NFS mount options, server address, and root path are passed to the
  NFS client.

rdinit=<executable file>
  To specify which file contains the program that starts system
  initialization, administrators can use this command line parameter.
  The default value of this parameter is "/init".  If the specified
  file exists and the kernel can execute it, root filesystem related
  kernel command line parameters, including 'nfsroot=', are ignored.

  A description of the process of mounting the root file system can be
  found in Documentation/driver-api/early-userspace/early_userspace_support.rst

Boot Loader
===========

To get the kernel into memory different approaches can be used.
They depend on various facilities being available:

- Booting from a floppy using syslinux

	When building kernels, an easy way to create a boot floppy that uses
	syslinux is to use the zdisk or bzdisk make targets which use zimage
      	and bzimage images respectively. Both targets accept the
     	FDARGS parameter which can be used to set the kernel command line.

	e.g::

	   make bzdisk FDARGS="root=/dev/nfs"

   	Note that the user running this command will need to have
     	access to the floppy drive device, /dev/fd0

     	For more information on syslinux, including how to create bootdisks
     	for prebuilt kernels, see https://syslinux.zytor.com/

	.. note::
		Previously it was possible to write a kernel directly to
		a floppy using dd, configure the boot device using rdev, and
		boot using the resulting floppy. Linux no longer supports this
		method of booting.

- Booting from a cdrom using isolinux

     	When building kernels, an easy way to create a bootable cdrom that
     	uses isolinux is to use the isoimage target which uses a bzimage
     	image. Like zdisk and bzdisk, this target accepts the FDARGS
     	parameter which can be used to set the kernel command line.

	e.g::

	  make isoimage FDARGS="root=/dev/nfs"

     	The resulting iso image will be arch/<ARCH>/boot/image.iso
     	This can be written to a cdrom using a variety of tools including
     	cdrecord.

	e.g::

	  cdrecord dev=ATAPI:1,0,0 arch/x86/boot/image.iso

     	For more information on isolinux, including how to create bootdisks
     	for prebuilt kernels, see https://syslinux.zytor.com/

- Using LILO

	When using LILO all the necessary command line parameters may be
	specified using the 'append=' directive in the LILO configuration
	file.

	However, to use the 'root=' directive you also need to create
	a dummy root device, which may be removed after LILO is run.

	e.g::

	  mknod /dev/boot255 c 0 255

	For information on configuring LILO, please refer to its documentation.

- Using GRUB

	When using GRUB, kernel parameter are simply appended after the kernel
	specification: kernel <kernel> <parameters>

- Using loadlin

	loadlin may be used to boot Linux from a DOS command prompt without
	requiring a local hard disk to mount as root. This has not been
	thoroughly tested by the authors of this document, but in general
	it should be possible configure the kernel command line similarly
	to the configuration of LILO.

	Please refer to the loadlin documentation for further information.

- Using a boot ROM

	This is probably the most elegant way of booting a diskless client.
	With a boot ROM the kernel is loaded using the TFTP protocol. The
	authors of this document are not aware of any no commercial boot
	ROMs that support booting Linux over the network. However, there
	are two free implementations of a boot ROM, netboot-nfs and
	etherboot, both of which are available on sunsite.unc.edu, and both
	of which contain everything you need to boot a diskless Linux client.

- Using pxelinux

	Pxelinux may be used to boot linux using the PXE boot loader
	which is present on many modern network cards.

	When using pxelinux, the kernel image is specified using
	"kernel <relative-path-below /tftpboot>". The nfsroot parameters
	are passed to the kernel by adding them to the "append" line.
	It is common to use serial console in conjunction with pxeliunx,
	see Documentation/admin-guide/serial-console.rst for more information.

	For more information on isolinux, including how to create bootdisks
	for prebuilt kernels, see https://syslinux.zytor.com/

Credits
=======

  The nfsroot code in the kernel and the RARP support have been written
  by Gero Kuhlmann <gero@gkminix.han.de>.

  The rest of the IP layer autoconfiguration code has been written
  by Martin Mares <mj@atrey.karlin.mff.cuni.cz>.

  In order to write the initial version of nfsroot I would like to thank
  Jens-Uwe Mager <jum@anubis.han.de> for his help.


code:
===================================
pNFS block layout server user guide
===================================

The Linux NFS server now supports the pNFS block layout extension.  In this
case the NFS server acts as Metadata Server (MDS) for pNFS, which in addition
to handling all the metadata access to the NFS export also hands out layouts
to the clients to directly access the underlying block devices that are
shared with the client.

To use pNFS block layouts with the Linux NFS server the exported file
system needs to support the pNFS block layouts (currently just XFS), and the
file system must sit on shared storage (typically iSCSI) that is accessible
to the clients in addition to the MDS.  As of now the file system needs to
sit directly on the exported volume, striping or concatenation of
volumes on the MDS and clients is not supported yet.

On the server, pNFS block volume support is automatically if the file system
support it.  On the client make sure the kernel has the CONFIG_PNFS_BLOCK
option enabled, the blkmapd daemon from nfs-utils is running, and the
file system is mounted using the NFSv4.1 protocol version (mount -o vers=4.1).

If the nfsd server needs to fence a non-responding client it calls
/sbin/nfsd-recall-failed with the first argument set to the IP address of
the client, and the second argument set to the device node without the /dev
prefix for the file system to be fenced. Below is an example file that shows
how to translate the device into a serial number from SCSI EVPD 0x80::

	cat > /sbin/nfsd-recall-failed << EOF

.. code-block:: sh

	#!/bin/sh

	CLIENT="$1"
	DEV="/dev/$2"
	EVPD=`sg_inq --page=0x80 ${DEV} | \
		grep "Unit serial number:" | \
		awk -F ': ' '{print $2}'`

	echo "fencing client ${CLIENT} serial ${EVPD}" >> /var/log/pnfsd-fence.log
	EOF


code:
==================================
pNFS SCSI layout server user guide
==================================

This document describes support for pNFS SCSI layouts in the Linux NFS server.
With pNFS SCSI layouts, the NFS server acts as Metadata Server (MDS) for pNFS,
which in addition to handling all the metadata access to the NFS export,
also hands out layouts to the clients so that they can directly access the
underlying SCSI LUNs that are shared with the client.

To use pNFS SCSI layouts with the Linux NFS server, the exported file
system needs to support the pNFS SCSI layouts (currently just XFS), and the
file system must sit on a SCSI LUN that is accessible to the clients in
addition to the MDS.  As of now the file system needs to sit directly on the
exported LUN, striping or concatenation of LUNs on the MDS and clients
is not supported yet.

On a server built with CONFIG_NFSD_SCSI, the pNFS SCSI volume support is
automatically enabled if the file system is exported using the "pnfs"
option and the underlying SCSI device support persistent reservations.
On the client make sure the kernel has the CONFIG_PNFS_BLOCK option
enabled, and the file system is mounted using the NFSv4.1 protocol
version (mount -o vers=4.1).


code:
===============================
Numa policy hit/miss statistics
===============================

/sys/devices/system/node/node*/numastat

All units are pages. Hugepages have separate counters.

The numa_hit, numa_miss and numa_foreign counters reflect how well processes
are able to allocate memory from nodes they prefer. If they succeed, numa_hit
is incremented on the preferred node, otherwise numa_foreign is incremented on
the preferred node and numa_miss on the node where allocation succeeded.

Usually preferred node is the one local to the CPU where the process executes,
but restrictions such as mempolicies can change that, so there are also two
counters based on CPU local node. local_node is similar to numa_hit and is
incremented on allocation from a node by CPU on the same node. other_node is
similar to numa_miss and is incremented on the node where allocation succeeds
from a CPU from a different node. Note there is no counter analogical to
numa_foreign.

In more detail:

=============== ============================================================
numa_hit	A process wanted to allocate memory from this node,
		and succeeded.

numa_miss	A process wanted to allocate memory from another node,
		but ended up with memory from this node.

numa_foreign	A process wanted to allocate on this node,
		but ended up with memory from another node.

local_node	A process ran on this node's CPU,
		and got memory from this node.

other_node	A process ran on a different node's CPU
		and got memory from this node.

interleave_hit 	Interleaving wanted to allocate from this node
		and succeeded.
=============== ============================================================

For easier reading you can use the numastat utility from the numactl package
(http://oss.sgi.com/projects/libnuma/). Note that it only works
well right now on machines with a small number of CPUs.

Note that on systems with memoryless nodes (where a node has CPUs but no
memory) the numa_hit, numa_miss and numa_foreign statistics can be skewed
heavily. In the current kernel implementation, if a process prefers a
memoryless node (i.e.  because it is running on one of its local CPU), the
implementation actually treats one of the nearest nodes with memory as the
preferred node. As a result, such allocation will not increase the numa_foreign
counter on the memoryless node, and will skew the numa_hit, numa_miss and
numa_foreign statistics of the nearest node.


code:
Parport
+++++++

The ``parport`` code provides parallel-port support under Linux.  This
includes the ability to share one port between multiple device
drivers.

You can pass parameters to the ``parport`` code to override its automatic
detection of your hardware.  This is particularly useful if you want
to use IRQs, since in general these can't be autoprobed successfully.
By default IRQs are not used even if they **can** be probed.  This is
because there are a lot of people using the same IRQ for their
parallel port and a sound card or network card.

The ``parport`` code is split into two parts: generic (which deals with
port-sharing) and architecture-dependent (which deals with actually
using the port).

Parport as modules
==================

If you load the `parport`` code as a module, say::

	# insmod parport

to load the generic ``parport`` code.  You then must load the
architecture-dependent code with (for example)::

	# insmod parport_pc io=0x3bc,0x378,0x278 irq=none,7,auto

to tell the ``parport`` code that you want three PC-style ports, one at
0x3bc with no IRQ, one at 0x378 using IRQ 7, and one at 0x278 with an
auto-detected IRQ.  Currently, PC-style (``parport_pc``), Sun ``bpp``,
Amiga, Atari, and MFC3 hardware is supported.

PCI parallel I/O card support comes from ``parport_pc``.  Base I/O
addresses should not be specified for supported PCI cards since they
are automatically detected.

modprobe
--------

If you use modprobe , you will find it useful to add lines as below to a
configuration file in /etc/modprobe.d/ directory::

	alias parport_lowlevel parport_pc
	options parport_pc io=0x378,0x278 irq=7,auto

modprobe will load ``parport_pc`` (with the options ``io=0x378,0x278 irq=7,auto``)
whenever a parallel port device driver (such as ``lp``) is loaded.

Note that these are example lines only!  You shouldn't in general need
to specify any options to ``parport_pc`` in order to be able to use a
parallel port.

Parport probe [optional]
------------------------

In 2.2 kernels there was a module called ``parport_probe``, which was used
for collecting IEEE 1284 device ID information.  This has now been
enhanced and now lives with the IEEE 1284 support.  When a parallel
port is detected, the devices that are connected to it are analysed,
and information is logged like this::

	parport0: Printer, BJC-210 (Canon)

The probe information is available from files in ``/proc/sys/dev/parport/``.

Parport linked into the kernel statically
=========================================

If you compile the ``parport`` code into the kernel, then you can use
kernel boot parameters to get the same effect.  Add something like the
following to your LILO command line::

	parport=0x3bc parport=0x378,7 parport=0x278,auto,nofifo

You can have many ``parport=...`` statements, one for each port you want
to add.  Adding ``parport=0`` to the kernel command-line will disable
parport support entirely.  Adding ``parport=auto`` to the kernel
command-line will make ``parport`` use any IRQ lines or DMA channels that
it auto-detects.

Files in /proc
==============

If you have configured the ``/proc`` filesystem into your kernel, you will
see a new directory entry: ``/proc/sys/dev/parport``.  In there will be a
directory entry for each parallel port for which parport is
configured.  In each of those directories are a collection of files
describing that parallel port.

The ``/proc/sys/dev/parport`` directory tree looks like::

	parport
	|-- default
	|   |-- spintime
	|   `-- timeslice
	|-- parport0
	|   |-- autoprobe
	|   |-- autoprobe0
	|   |-- autoprobe1
	|   |-- autoprobe2
	|   |-- autoprobe3
	|   |-- devices
	|   |   |-- active
	|   |   `-- lp
	|   |       `-- timeslice
	|   |-- base-addr
	|   |-- irq
	|   |-- dma
	|   |-- modes
	|   `-- spintime
	`-- parport1
	|-- autoprobe
	|-- autoprobe0
	|-- autoprobe1
	|-- autoprobe2
	|-- autoprobe3
	|-- devices
	|   |-- active
	|   `-- ppa
	|       `-- timeslice
	|-- base-addr
	|-- irq
	|-- dma
	|-- modes
	`-- spintime

.. tabularcolumns:: |p{4.0cm}|p{13.5cm}|

=======================	=======================================================
File			Contents
=======================	=======================================================
``devices/active``	A list of the device drivers using that port.  A "+"
			will appear by the name of the device currently using
			the port (it might not appear against any).  The
			string "none" means that there are no device drivers
			using that port.

``base-addr``		Parallel port's base address, or addresses if the port
			has more than one in which case they are separated
			with tabs.  These values might not have any sensible
			meaning for some ports.

``irq``			Parallel port's IRQ, or -1 if none is being used.

``dma``			Parallel port's DMA channel, or -1 if none is being
			used.

``modes``		Parallel port's hardware modes, comma-separated,
			meaning:

			- PCSPP
				PC-style SPP registers are available.

			- TRISTATE
				Port is bidirectional.

			- COMPAT
				Hardware acceleration for printers is
				available and will be used.

			- EPP
				Hardware acceleration for EPP protocol
				is available and will be used.

			- ECP
				Hardware acceleration for ECP protocol
				is available and will be used.

			- DMA
				DMA is available and will be used.

			Note that the current implementation will only take
			advantage of COMPAT and ECP modes if it has an IRQ
			line to use.

``autoprobe``		Any IEEE-1284 device ID information that has been
			acquired from the (non-IEEE 1284.3) device.

``autoprobe[0-3]``	IEEE 1284 device ID information retrieved from
			daisy-chain devices that conform to IEEE 1284.3.

``spintime``		The number of microseconds to busy-loop while waiting
			for the peripheral to respond.  You might find that
			adjusting this improves performance, depending on your
			peripherals.  This is a port-wide setting, i.e. it
			applies to all devices on a particular port.

``timeslice``		The number of milliseconds that a device driver is
			allowed to keep a port claimed for.  This is advisory,
			and driver can ignore it if it must.

``default/*``		The defaults for spintime and timeslice. When a new
			port is	registered, it picks up the default spintime.
			When a new device is registered, it picks up the
			default timeslice.
=======================	=======================================================

Device drivers
==============

Once the parport code is initialised, you can attach device drivers to
specific ports.  Normally this happens automatically; if the lp driver
is loaded it will create one lp device for each port found.  You can
override this, though, by using parameters either when you load the lp
driver::

	# insmod lp parport=0,2

or on the LILO command line::

	lp=parport0 lp=parport2

Both the above examples would inform lp that you want ``/dev/lp0`` to be
the first parallel port, and /dev/lp1 to be the **third** parallel port,
with no lp device associated with the second port (parport1).  Note
that this is different to the way older kernels worked; there used to
be a static association between the I/O port address and the device
name, so ``/dev/lp0`` was always the port at 0x3bc.  This is no longer the
case - if you only have one port, it will default to being ``/dev/lp0``,
regardless of base address.

Also:

 * If you selected the IEEE 1284 support at compile time, you can say
   ``lp=auto`` on the kernel command line, and lp will create devices
   only for those ports that seem to have printers attached.

 * If you give PLIP the ``timid`` parameter, either with ``plip=timid`` on
   the command line, or with ``insmod plip timid=1`` when using modules,
   it will avoid any ports that seem to be in use by other devices.

 * IRQ autoprobing works only for a few port types at the moment.

Reporting printer problems with parport
=======================================

If you are having problems printing, please go through these steps to
try to narrow down where the problem area is.

When reporting problems with parport, really you need to give all of
the messages that ``parport_pc`` spits out when it initialises.  There are
several code paths:

- polling
- interrupt-driven, protocol in software
- interrupt-driven, protocol in hardware using PIO
- interrupt-driven, protocol in hardware using DMA

The kernel messages that ``parport_pc`` logs give an indication of which
code path is being used. (They could be a lot better actually..)

For normal printer protocol, having IEEE 1284 modes enabled or not
should not make a difference.

To turn off the 'protocol in hardware' code paths, disable
``CONFIG_PARPORT_PC_FIFO``.  Note that when they are enabled they are not
necessarily **used**; it depends on whether the hardware is available,
enabled by the BIOS, and detected by the driver.

So, to start with, disable ``CONFIG_PARPORT_PC_FIFO``, and load ``parport_pc``
with ``irq=none``. See if printing works then.  It really should,
because this is the simplest code path.

If that works fine, try with ``io=0x378 irq=7`` (adjust for your
hardware), to make it use interrupt-driven in-software protocol.

If **that** works fine, then one of the hardware modes isn't working
right.  Enable ``CONFIG_FIFO`` (no, it isn't a module option,
and yes, it should be), set the port to ECP mode in the BIOS and note
the DMA channel, and try with::

    io=0x378 irq=7 dma=none (for PIO)
    io=0x378 irq=7 dma=3 (for DMA)

----------

philb@gnu.org
tim@cyberelk.net