code:
.. _unevictable_lru:

==============================
Unevictable LRU Infrastructure
==============================

.. contents:: :local:

Introduction
============

This document describes the Linux memory manager's "Unevictable LRU"
infrastructure and the use of this to manage several types of "unevictable"
pages.

The document attempts to provide the overall rationale behind this mechanism
and the rationale for some of the design decisions that drove the
implementation.  The latter design rationale is discussed in the context of an
implementation description.  Admittedly, one can obtain the implementation
details - the "what does it do?" - by reading the code.  One hopes that the
descriptions below add value by provide the answer to "why does it do that?".

The Unevictable LRU
===================

The Unevictable LRU facility adds an additional LRU list to track unevictable
pages and to hide these pages from vmscan.  This mechanism is based on a patch
by Larry Woodman of Red Hat to address several scalability problems with page
reclaim in Linux.  The problems have been observed at customer sites on large
memory x86_64 systems.

To illustrate this with an example, a non-NUMA x86_64 platform with 128GB of
main memory will have over 32 million 4k pages in a single node.  When a large
fraction of these pages are not evictable for any reason [see below], vmscan
will spend a lot of time scanning the LRU lists looking for the small fraction
of pages that are evictable.  This can result in a situation where all CPUs are
spending 100% of their time in vmscan for hours or days on end, with the system
completely unresponsive.

The unevictable list addresses the following classes of unevictable pages:

 * Those owned by ramfs.

 * Those mapped into SHM_LOCK'd shared memory regions.

 * Those mapped into VM_LOCKED [mlock()ed] VMAs.

The infrastructure may also be able to handle other conditions that make pages
unevictable, either by definition or by circumstance, in the future.

The Unevictable LRU Page List
-----------------------------

The Unevictable LRU page list is a lie.  It was never an LRU-ordered list, but a
companion to the LRU-ordered anonymous and file, active and inactive page lists;
and now it is not even a page list.  But following familiar convention, here in
this document and in the source, we often imagine it as a fifth LRU page list.

The Unevictable LRU infrastructure consists of an additional, per-node, LRU list
called the "unevictable" list and an associated page flag, PG_unevictable, to
indicate that the page is being managed on the unevictable list.

The PG_unevictable flag is analogous to, and mutually exclusive with, the
PG_active flag in that it indicates on which LRU list a page resides when
PG_lru is set.

The Unevictable LRU infrastructure maintains unevictable pages as if they were
on an additional LRU list for a few reasons:

 (1) We get to "treat unevictable pages just like we treat other pages in the
     system - which means we get to use the same code to manipulate them, the
     same code to isolate them (for migrate, etc.), the same code to keep track
     of the statistics, etc..." [Rik van Riel]

 (2) We want to be able to migrate unevictable pages between nodes for memory
     defragmentation, workload management and memory hotplug.  The Linux kernel
     can only migrate pages that it can successfully isolate from the LRU
     lists (or "Movable" pages: outside of consideration here).  If we were to
     maintain pages elsewhere than on an LRU-like list, where they can be
     detected by isolate_lru_page(), we would prevent their migration.

The unevictable list does not differentiate between file-backed and anonymous,
swap-backed pages.  This differentiation is only important while the pages are,
in fact, evictable.

The unevictable list benefits from the "arrayification" of the per-node LRU
lists and statistics originally proposed and posted by Christoph Lameter.

Memory Control Group Interaction
--------------------------------

The unevictable LRU facility interacts with the memory control group [aka
memory controller; see Documentation/admin-guide/cgroup-v1/memory.rst] by
extending the lru_list enum.

The memory controller data structure automatically gets a per-node unevictable
list as a result of the "arrayification" of the per-node LRU lists (one per
lru_list enum element).  The memory controller tracks the movement of pages to
and from the unevictable list.

When a memory control group comes under memory pressure, the controller will
not attempt to reclaim pages on the unevictable list.  This has a couple of
effects:

 (1) Because the pages are "hidden" from reclaim on the unevictable list, the
     reclaim process can be more efficient, dealing only with pages that have a
     chance of being reclaimed.

 (2) On the other hand, if too many of the pages charged to the control group
     are unevictable, the evictable portion of the working set of the tasks in
     the control group may not fit into the available memory.  This can cause
     the control group to thrash or to OOM-kill tasks.

.. _mark_addr_space_unevict:

Marking Address Spaces Unevictable
----------------------------------

For facilities such as ramfs none of the pages attached to the address space
may be evicted.  To prevent eviction of any such pages, the AS_UNEVICTABLE
address space flag is provided, and this can be manipulated by a filesystem
using a number of wrapper functions:

 * ``void mapping_set_unevictable(struct address_space *mapping);``

	Mark the address space as being completely unevictable.

 * ``void mapping_clear_unevictable(struct address_space *mapping);``

	Mark the address space as being evictable.

 * ``int mapping_unevictable(struct address_space *mapping);``

	Query the address space, and return true if it is completely
	unevictable.

These are currently used in three places in the kernel:

 (1) By ramfs to mark the address spaces of its inodes when they are created,
     and this mark remains for the life of the inode.

 (2) By SYSV SHM to mark SHM_LOCK'd address spaces until SHM_UNLOCK is called.
     Note that SHM_LOCK is not required to page in the locked pages if they're
     swapped out; the application must touch the pages manually if it wants to
     ensure they're in memory.

 (3) By the i915 driver to mark pinned address space until it's unpinned. The
     amount of unevictable memory marked by i915 driver is roughly the bounded
     object size in debugfs/dri/0/i915_gem_objects.

Detecting Unevictable Pages
---------------------------

The function page_evictable() in mm/internal.h determines whether a page is
evictable or not using the query function outlined above [see section
:ref:`Marking address spaces unevictable <mark_addr_space_unevict>`]
to check the AS_UNEVICTABLE flag.

For address spaces that are so marked after being populated (as SHM regions
might be), the lock action (e.g. SHM_LOCK) can be lazy, and need not populate
the page tables for the region as does, for example, mlock(), nor need it make
any special effort to push any pages in the SHM_LOCK'd area to the unevictable
list.  Instead, vmscan will do this if and when it encounters the pages during
a reclamation scan.

On an unlock action (such as SHM_UNLOCK), the unlocker (e.g. shmctl()) must scan
the pages in the region and "rescue" them from the unevictable list if no other
condition is keeping them unevictable.  If an unevictable region is destroyed,
the pages are also "rescued" from the unevictable list in the process of
freeing them.

page_evictable() also checks for mlocked pages by testing an additional page
flag, PG_mlocked (as wrapped by PageMlocked()), which is set when a page is
faulted into a VM_LOCKED VMA, or found in a VMA being VM_LOCKED.

Vmscan's Handling of Unevictable Pages
--------------------------------------

If unevictable pages are culled in the fault path, or moved to the unevictable
list at mlock() or mmap() time, vmscan will not encounter the pages until they
have become evictable again (via munlock() for example) and have been "rescued"
from the unevictable list.  However, there may be situations where we decide,
for the sake of expediency, to leave an unevictable page on one of the regular
active/inactive LRU lists for vmscan to deal with.  vmscan checks for such
pages in all of the shrink_{active|inactive|page}_list() functions and will
"cull" such pages that it encounters: that is, it diverts those pages to the
unevictable list for the memory cgroup and node being scanned.

There may be situations where a page is mapped into a VM_LOCKED VMA, but the
page is not marked as PG_mlocked.  Such pages will make it all the way to
shrink_active_list() or shrink_page_list() where they will be detected when
vmscan walks the reverse map in folio_referenced() or try_to_unmap().  The page
is culled to the unevictable list when it is released by the shrinker.

To "cull" an unevictable page, vmscan simply puts the page back on the LRU list
using putback_lru_page() - the inverse operation to isolate_lru_page() - after
dropping the page lock.  Because the condition which makes the page unevictable
may change once the page is unlocked, __pagevec_lru_add_fn() will recheck the
unevictable state of a page before placing it on the unevictable list.

MLOCKED Pages
=============

The unevictable page list is also useful for mlock(), in addition to ramfs and
SYSV SHM.  Note that mlock() is only available in CONFIG_MMU=y situations; in
NOMMU situations, all mappings are effectively mlocked.

History
-------

The "Unevictable mlocked Pages" infrastructure is based on work originally
posted by Nick Piggin in an RFC patch entitled "mm: mlocked pages off LRU".
Nick posted his patch as an alternative to a patch posted by Christoph Lameter
to achieve the same objective: hiding mlocked pages from vmscan.

In Nick's patch, he used one of the struct page LRU list link fields as a count
of VM_LOCKED VMAs that map the page (Rik van Riel had the same idea three years
earlier).  But this use of the link field for a count prevented the management
of the pages on an LRU list, and thus mlocked pages were not migratable as
isolate_lru_page() could not detect them, and the LRU list link field was not
available to the migration subsystem.

Nick resolved this by putting mlocked pages back on the LRU list before
attempting to isolate them, thus abandoning the count of VM_LOCKED VMAs.  When
Nick's patch was integrated with the Unevictable LRU work, the count was
replaced by walking the reverse map when munlocking, to determine whether any
other VM_LOCKED VMAs still mapped the page.

However, walking the reverse map for each page when munlocking was ugly and
inefficient, and could lead to catastrophic contention on a file's rmap lock,
when many processes which had it mlocked were trying to exit.  In 5.18, the
idea of keeping mlock_count in Unevictable LRU list link field was revived and
put to work, without preventing the migration of mlocked pages.  This is why
the "Unevictable LRU list" cannot be a linked list of pages now; but there was
no use for that linked list anyway - though its size is maintained for meminfo.

Basic Management
----------------

mlocked pages - pages mapped into a VM_LOCKED VMA - are a class of unevictable
pages.  When such a page has been "noticed" by the memory management subsystem,
the page is marked with the PG_mlocked flag.  This can be manipulated using the
PageMlocked() functions.

A PG_mlocked page will be placed on the unevictable list when it is added to
the LRU.  Such pages can be "noticed" by memory management in several places:

 (1) in the mlock()/mlock2()/mlockall() system call handlers;

 (2) in the mmap() system call handler when mmapping a region with the
     MAP_LOCKED flag;

 (3) mmapping a region in a task that has called mlockall() with the MCL_FUTURE
     flag;

 (4) in the fault path and when a VM_LOCKED stack segment is expanded; or

 (5) as mentioned above, in vmscan:shrink_page_list() when attempting to
     reclaim a page in a VM_LOCKED VMA by folio_referenced() or try_to_unmap().

mlocked pages become unlocked and rescued from the unevictable list when:

 (1) mapped in a range unlocked via the munlock()/munlockall() system calls;

 (2) munmap()'d out of the last VM_LOCKED VMA that maps the page, including
     unmapping at task exit;

 (3) when the page is truncated from the last VM_LOCKED VMA of an mmapped file;
     or

 (4) before a page is COW'd in a VM_LOCKED VMA.

mlock()/mlock2()/mlockall() System Call Handling
------------------------------------------------

mlock(), mlock2() and mlockall() system call handlers proceed to mlock_fixup()
for each VMA in the range specified by the call.  In the case of mlockall(),
this is the entire active address space of the task.  Note that mlock_fixup()
is used for both mlocking and munlocking a range of memory.  A call to mlock()
an already VM_LOCKED VMA, or to munlock() a VMA that is not VM_LOCKED, is
treated as a no-op and mlock_fixup() simply returns.

If the VMA passes some filtering as described in "Filtering Special VMAs"
below, mlock_fixup() will attempt to merge the VMA with its neighbors or split
off a subset of the VMA if the range does not cover the entire VMA.  Any pages
already present in the VMA are then marked as mlocked by mlock_page() via
mlock_pte_range() via walk_page_range() via mlock_vma_pages_range().

Before returning from the system call, do_mlock() or mlockall() will call
__mm_populate() to fault in the remaining pages via get_user_pages() and to
mark those pages as mlocked as they are faulted.

Note that the VMA being mlocked might be mapped with PROT_NONE.  In this case,
get_user_pages() will be unable to fault in the pages.  That's okay.  If pages
do end up getting faulted into this VM_LOCKED VMA, they will be handled in the
fault path - which is also how mlock2()'s MLOCK_ONFAULT areas are handled.

For each PTE (or PMD) being faulted into a VMA, the page add rmap function
calls mlock_vma_page(), which calls mlock_page() when the VMA is VM_LOCKED
(unless it is a PTE mapping of a part of a transparent huge page).  Or when
it is a newly allocated anonymous page, lru_cache_add_inactive_or_unevictable()
calls mlock_new_page() instead: similar to mlock_page(), but can make better
judgments, since this page is held exclusively and known not to be on LRU yet.

mlock_page() sets PageMlocked immediately, then places the page on the CPU's
mlock pagevec, to batch up the rest of the work to be done under lru_lock by
__mlock_page().  __mlock_page() sets PageUnevictable, initializes mlock_count
and moves the page to unevictable state ("the unevictable LRU", but with
mlock_count in place of LRU threading).  Or if the page was already PageLRU
and PageUnevictable and PageMlocked, it simply increments the mlock_count.

But in practice that may not work ideally: the page may not yet be on an LRU, or
it may have been temporarily isolated from LRU.  In such cases the mlock_count
field cannot be touched, but will be set to 0 later when __pagevec_lru_add_fn()
returns the page to "LRU".  Races prohibit mlock_count from being set to 1 then:
rather than risk stranding a page indefinitely as unevictable, always err with
mlock_count on the low side, so that when munlocked the page will be rescued to
an evictable LRU, then perhaps be mlocked again later if vmscan finds it in a
VM_LOCKED VMA.

Filtering Special VMAs
----------------------

mlock_fixup() filters several classes of "special" VMAs:

1) VMAs with VM_IO or VM_PFNMAP set are skipped entirely.  The pages behind
   these mappings are inherently pinned, so we don't need to mark them as
   mlocked.  In any case, most of the pages have no struct page in which to so
   mark the page.  Because of this, get_user_pages() will fail for these VMAs,
   so there is no sense in attempting to visit them.

2) VMAs mapping hugetlbfs page are already effectively pinned into memory.  We
   neither need nor want to mlock() these pages.  But __mm_populate() includes
   hugetlbfs ranges, allocating the huge pages and populating the PTEs.

3) VMAs with VM_DONTEXPAND are generally userspace mappings of kernel pages,
   such as the VDSO page, relay channel pages, etc.  These pages are inherently
   unevictable and are not managed on the LRU lists.  __mm_populate() includes
   these ranges, populating the PTEs if not already populated.

4) VMAs with VM_MIXEDMAP set are not marked VM_LOCKED, but __mm_populate()
   includes these ranges, populating the PTEs if not already populated.

Note that for all of these special VMAs, mlock_fixup() does not set the
VM_LOCKED flag.  Therefore, we won't have to deal with them later during
munlock(), munmap() or task exit.  Neither does mlock_fixup() account these
VMAs against the task's "locked_vm".

munlock()/munlockall() System Call Handling
-------------------------------------------

The munlock() and munlockall() system calls are handled by the same
mlock_fixup() function as mlock(), mlock2() and mlockall() system calls are.
If called to munlock an already munlocked VMA, mlock_fixup() simply returns.
Because of the VMA filtering discussed above, VM_LOCKED will not be set in
any "special" VMAs.  So, those VMAs will be ignored for munlock.

If the VMA is VM_LOCKED, mlock_fixup() again attempts to merge or split off the
specified range.  All pages in the VMA are then munlocked by munlock_page() via
mlock_pte_range() via walk_page_range() via mlock_vma_pages_range() - the same
function used when mlocking a VMA range, with new flags for the VMA indicating
that it is munlock() being performed.

munlock_page() uses the mlock pagevec to batch up work to be done under
lru_lock by  __munlock_page().  __munlock_page() decrements the page's
mlock_count, and when that reaches 0 it clears PageMlocked and clears
PageUnevictable, moving the page from unevictable state to inactive LRU.

But in practice that may not work ideally: the page may not yet have reached
"the unevictable LRU", or it may have been temporarily isolated from it.  In
those cases its mlock_count field is unusable and must be assumed to be 0: so
that the page will be rescued to an evictable LRU, then perhaps be mlocked
again later if vmscan finds it in a VM_LOCKED VMA.

Migrating MLOCKED Pages
-----------------------

A page that is being migrated has been isolated from the LRU lists and is held
locked across unmapping of the page, updating the page's address space entry
and copying the contents and state, until the page table entry has been
replaced with an entry that refers to the new page.  Linux supports migration
of mlocked pages and other unevictable pages.  PG_mlocked is cleared from the
the old page when it is unmapped from the last VM_LOCKED VMA, and set when the
new page is mapped in place of migration entry in a VM_LOCKED VMA.  If the page
was unevictable because mlocked, PG_unevictable follows PG_mlocked; but if the
page was unevictable for other reasons, PG_unevictable is copied explicitly.

Note that page migration can race with mlocking or munlocking of the same page.
There is mostly no problem since page migration requires unmapping all PTEs of
the old page (including munlock where VM_LOCKED), then mapping in the new page
(including mlock where VM_LOCKED).  The page table locks provide sufficient
synchronization.

However, since mlock_vma_pages_range() starts by setting VM_LOCKED on a VMA,
before mlocking any pages already present, if one of those pages were migrated
before mlock_pte_range() reached it, it would get counted twice in mlock_count.
To prevent that, mlock_vma_pages_range() temporarily marks the VMA as VM_IO,
so that mlock_vma_page() will skip it.

To complete page migration, we place the old and new pages back onto the LRU
afterwards.  The "unneeded" page - old page on success, new page on failure -
is freed when the reference count held by the migration process is released.

Compacting MLOCKED Pages
------------------------

The memory map can be scanned for compactable regions and the default behavior
is to let unevictable pages be moved.  /proc/sys/vm/compact_unevictable_allowed
controls this behavior (see Documentation/admin-guide/sysctl/vm.rst).  The work
of compaction is mostly handled by the page migration code and the same work
flow as described in Migrating MLOCKED Pages will apply.

MLOCKING Transparent Huge Pages
-------------------------------

A transparent huge page is represented by a single entry on an LRU list.
Therefore, we can only make unevictable an entire compound page, not
individual subpages.

If a user tries to mlock() part of a huge page, and no user mlock()s the
whole of the huge page, we want the rest of the page to be reclaimable.

We cannot just split the page on partial mlock() as split_huge_page() can
fail and a new intermittent failure mode for the syscall is undesirable.

We handle this by keeping PTE-mlocked huge pages on evictable LRU lists:
the PMD on the border of a VM_LOCKED VMA will be split into a PTE table.

This way the huge page is accessible for vmscan.  Under memory pressure the
page will be split, subpages which belong to VM_LOCKED VMAs will be moved
to the unevictable LRU and the rest can be reclaimed.

/proc/meminfo's Unevictable and Mlocked amounts do not include those parts
of a transparent huge page which are mapped only by PTEs in VM_LOCKED VMAs.

mmap(MAP_LOCKED) System Call Handling
-------------------------------------

In addition to the mlock(), mlock2() and mlockall() system calls, an application
can request that a region of memory be mlocked by supplying the MAP_LOCKED flag
to the mmap() call.  There is one important and subtle difference here, though.
mmap() + mlock() will fail if the range cannot be faulted in (e.g. because
mm_populate fails) and returns with ENOMEM while mmap(MAP_LOCKED) will not fail.
The mmaped area will still have properties of the locked area - pages will not
get swapped out - but major page faults to fault memory in might still happen.

Furthermore, any mmap() call or brk() call that expands the heap by a task
that has previously called mlockall() with the MCL_FUTURE flag will result
in the newly mapped memory being mlocked.  Before the unevictable/mlock
changes, the kernel simply called make_pages_present() to allocate pages
and populate the page table.

To mlock a range of memory under the unevictable/mlock infrastructure,
the mmap() handler and task address space expansion functions call
populate_vma_page_range() specifying the vma and the address range to mlock.

munmap()/exit()/exec() System Call Handling
-------------------------------------------

When unmapping an mlocked region of memory, whether by an explicit call to
munmap() or via an internal unmap from exit() or exec() processing, we must
munlock the pages if we're removing the last VM_LOCKED VMA that maps the pages.
Before the unevictable/mlock changes, mlocking did not mark the pages in any
way, so unmapping them required no processing.

For each PTE (or PMD) being unmapped from a VMA, page_remove_rmap() calls
munlock_vma_page(), which calls munlock_page() when the VMA is VM_LOCKED
(unless it was a PTE mapping of a part of a transparent huge page).

munlock_page() uses the mlock pagevec to batch up work to be done under
lru_lock by  __munlock_page().  __munlock_page() decrements the page's
mlock_count, and when that reaches 0 it clears PageMlocked and clears
PageUnevictable, moving the page from unevictable state to inactive LRU.

But in practice that may not work ideally: the page may not yet have reached
"the unevictable LRU", or it may have been temporarily isolated from it.  In
those cases its mlock_count field is unusable and must be assumed to be 0: so
that the page will be rescued to an evictable LRU, then perhaps be mlocked
again later if vmscan finds it in a VM_LOCKED VMA.

Truncating MLOCKED Pages
------------------------

File truncation or hole punching forcibly unmaps the deleted pages from
userspace; truncation even unmaps and deletes any private anonymous pages
which had been Copied-On-Write from the file pages now being truncated.

Mlocked pages can be munlocked and deleted in this way: like with munmap(),
for each PTE (or PMD) being unmapped from a VMA, page_remove_rmap() calls
munlock_vma_page(), which calls munlock_page() when the VMA is VM_LOCKED
(unless it was a PTE mapping of a part of a transparent huge page).

However, if there is a racing munlock(), since mlock_vma_pages_range() starts
munlocking by clearing VM_LOCKED from a VMA, before munlocking all the pages
present, if one of those pages were unmapped by truncation or hole punch before
mlock_pte_range() reached it, it would not be recognized as mlocked by this VMA,
and would not be counted out of mlock_count.  In this rare case, a page may
still appear as PageMlocked after it has been fully unmapped: and it is left to
release_pages() (or __page_cache_release()) to clear it and update statistics
before freeing (this event is counted in /proc/vmstat unevictable_pgs_cleared,
which is usually 0).

Page Reclaim in shrink_*_list()
-------------------------------

vmscan's shrink_active_list() culls any obviously unevictable pages -
i.e. !page_evictable(page) pages - diverting those to the unevictable list.
However, shrink_active_list() only sees unevictable pages that made it onto the
active/inactive LRU lists.  Note that these pages do not have PageUnevictable
set - otherwise they would be on the unevictable list and shrink_active_list()
would never see them.

Some examples of these unevictable pages on the LRU lists are:

 (1) ramfs pages that have been placed on the LRU lists when first allocated.

 (2) SHM_LOCK'd shared memory pages.  shmctl(SHM_LOCK) does not attempt to
     allocate or fault in the pages in the shared memory region.  This happens
     when an application accesses the page the first time after SHM_LOCK'ing
     the segment.

 (3) pages still mapped into VM_LOCKED VMAs, which should be marked mlocked,
     but events left mlock_count too low, so they were munlocked too early.

vmscan's shrink_inactive_list() and shrink_page_list() also divert obviously
unevictable pages found on the inactive lists to the appropriate memory cgroup
and node unevictable list.

rmap's folio_referenced_one(), called via vmscan's shrink_active_list() or
shrink_page_list(), and rmap's try_to_unmap_one() called via shrink_page_list(),
check for (3) pages still mapped into VM_LOCKED VMAs, and call mlock_vma_page()
to correct them.  Such pages are culled to the unevictable list when released
by the shrinker.


code:
.. SPDX-License-Identifier: GPL-2.0

=====================================
Virtually Mapped Kernel Stack Support
=====================================

:Author: Shuah Khan <skhan@linuxfoundation.org>

.. contents:: :local:

Overview
--------

This is a compilation of information from the code and original patch
series that introduced the `Virtually Mapped Kernel Stacks feature
<https://lwn.net/Articles/694348/>`

Introduction
------------

Kernel stack overflows are often hard to debug and make the kernel
susceptible to exploits. Problems could show up at a later time making
it difficult to isolate and root-cause.

Virtually-mapped kernel stacks with guard pages causes kernel stack
overflows to be caught immediately rather than causing difficult to
diagnose corruptions.

HAVE_ARCH_VMAP_STACK and VMAP_STACK configuration options enable
support for virtually mapped stacks with guard pages. This feature
causes reliable faults when the stack overflows. The usability of
the stack trace after overflow and response to the overflow itself
is architecture dependent.

.. note::
        As of this writing, arm64, powerpc, riscv, s390, um, and x86 have
        support for VMAP_STACK.

HAVE_ARCH_VMAP_STACK
--------------------

Architectures that can support Virtually Mapped Kernel Stacks should
enable this bool configuration option. The requirements are:

- vmalloc space must be large enough to hold many kernel stacks. This
  may rule out many 32-bit architectures.
- Stacks in vmalloc space need to work reliably.  For example, if
  vmap page tables are created on demand, either this mechanism
  needs to work while the stack points to a virtual address with
  unpopulated page tables or arch code (switch_to() and switch_mm(),
  most likely) needs to ensure that the stack's page table entries
  are populated before running on a possibly unpopulated stack.
- If the stack overflows into a guard page, something reasonable
  should happen. The definition of "reasonable" is flexible, but
  instantly rebooting without logging anything would be unfriendly.

VMAP_STACK
----------

VMAP_STACK bool configuration option when enabled allocates virtually
mapped task stacks. This option depends on HAVE_ARCH_VMAP_STACK.

- Enable this if you want the use virtually-mapped kernel stacks
  with guard pages. This causes kernel stack overflows to be caught
  immediately rather than causing difficult-to-diagnose corruption.

.. note::

        Using this feature with KASAN requires architecture support
        for backing virtual mappings with real shadow memory, and
        KASAN_VMALLOC must be enabled.

.. note::

        VMAP_STACK is enabled, it is not possible to run DMA on stack
        allocated data.

Kernel configuration options and dependencies keep changing. Refer to
the latest code base:

`Kconfig <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/Kconfig>`

Allocation
-----------

When a new kernel thread is created, thread stack is allocated from
virtually contiguous memory pages from the page level allocator. These
pages are mapped into contiguous kernel virtual space with PAGE_KERNEL
protections.

alloc_thread_stack_node() calls __vmalloc_node_range() to allocate stack
with PAGE_KERNEL protections.

- Allocated stacks are cached and later reused by new threads, so memcg
  accounting is performed manually on assigning/releasing stacks to tasks.
  Hence, __vmalloc_node_range is called without __GFP_ACCOUNT.
- vm_struct is cached to be able to find when thread free is initiated
  in interrupt context. free_thread_stack() can be called in interrupt
  context.
- On arm64, all VMAP's stacks need to have the same alignment to ensure
  that VMAP'd stack overflow detection works correctly. Arch specific
  vmap stack allocator takes care of this detail.
- This does not address interrupt stacks - according to the original patch

Thread stack allocation is initiated from clone(), fork(), vfork(),
kernel_thread() via kernel_clone(). Leaving a few hints for searching
the code base to understand when and how thread stack is allocated.

Bulk of the code is in:
`kernel/fork.c <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/fork.c>`.

stack_vm_area pointer in task_struct keeps track of the virtually allocated
stack and a non-null stack_vm_area pointer serves as a indication that the
virtually mapped kernel stacks are enabled.

::

        struct vm_struct *stack_vm_area;

Stack overflow handling
-----------------------

Leading and trailing guard pages help detect stack overflows. When stack
overflows into the guard pages, handlers have to be careful not overflow
the stack again. When handlers are called, it is likely that very little
stack space is left.

On x86, this is done by handling the page fault indicating the kernel
stack overflow on the double-fault stack.

Testing VMAP allocation with guard pages
----------------------------------------

How do we ensure that VMAP_STACK is actually allocating with a leading
and trailing guard page? The following lkdtm tests can help detect any
regressions.

::

        void lkdtm_STACK_GUARD_PAGE_LEADING()
        void lkdtm_STACK_GUARD_PAGE_TRAILING()

Conclusions
-----------

- A percpu cache of vmalloced stacks appears to be a bit faster than a
  high-order stack allocation, at least when the cache hits.
- THREAD_INFO_IN_TASK gets rid of arch-specific thread_info entirely and
  simply embed the thread_info (containing only flags) and 'int cpu' into
  task_struct.
- The thread stack can be free'ed as soon as the task is dead (without
  waiting for RCU) and then, if vmapped stacks are in use, cache the
  entire stack for reuse on the same cpu.

repo_name: angery/linux


code:
.. SPDX-License-Identifier: GPL-2.0

=========================================
A vmemmap diet for HugeTLB and Device DAX
=========================================

HugeTLB
=======

This section is to explain how HugeTLB Vmemmap Optimization (HVO) works.

The ``struct page`` structures are used to describe a physical page frame. By
default, there is a one-to-one mapping from a page frame to it's corresponding
``struct page``.

HugeTLB pages consist of multiple base page size pages and is supported by many
architectures. See Documentation/admin-guide/mm/hugetlbpage.rst for more
details. On the x86-64 architecture, HugeTLB pages of size 2MB and 1GB are
currently supported. Since the base page size on x86 is 4KB, a 2MB HugeTLB page
consists of 512 base pages and a 1GB HugeTLB page consists of 4096 base pages.
For each base page, there is a corresponding ``struct page``.

Within the HugeTLB subsystem, only the first 4 ``struct page`` are used to
contain unique information about a HugeTLB page. ``__NR_USED_SUBPAGE`` provides
this upper limit. The only 'useful' information in the remaining ``struct page``
is the compound_head field, and this field is the same for all tail pages.

By removing redundant ``struct page`` for HugeTLB pages, memory can be returned
to the buddy allocator for other uses.

Different architectures support different HugeTLB pages. For example, the
following table is the HugeTLB page size supported by x86 and arm64
architectures. Because arm64 supports 4k, 16k, and 64k base pages and
supports contiguous entries, so it supports many kinds of sizes of HugeTLB
page.

+--------------+-----------+-----------------------------------------------+
| Architecture | Page Size |                HugeTLB Page Size              |
+--------------+-----------+-----------+-----------+-----------+-----------+
|    x86-64    |    4KB    |    2MB    |    1GB    |           |           |
+--------------+-----------+-----------+-----------+-----------+-----------+
|              |    4KB    |   64KB    |    2MB    |    32MB   |    1GB    |
|              +-----------+-----------+-----------+-----------+-----------+
|    arm64     |   16KB    |    2MB    |   32MB    |     1GB   |           |
|              +-----------+-----------+-----------+-----------+-----------+
|              |   64KB    |    2MB    |  512MB    |    16GB   |           |
+--------------+-----------+-----------+-----------+-----------+-----------+

When the system boot up, every HugeTLB page has more than one ``struct page``
structs which size is (unit: pages)::

   struct_size = HugeTLB_Size / PAGE_SIZE * sizeof(struct page) / PAGE_SIZE

Where HugeTLB_Size is the size of the HugeTLB page. We know that the size
of the HugeTLB page is always n times PAGE_SIZE. So we can get the following
relationship::

   HugeTLB_Size = n * PAGE_SIZE

Then::

   struct_size = n * PAGE_SIZE / PAGE_SIZE * sizeof(struct page) / PAGE_SIZE
               = n * sizeof(struct page) / PAGE_SIZE

We can use huge mapping at the pud/pmd level for the HugeTLB page.

For the HugeTLB page of the pmd level mapping, then::

   struct_size = n * sizeof(struct page) / PAGE_SIZE
               = PAGE_SIZE / sizeof(pte_t) * sizeof(struct page) / PAGE_SIZE
               = sizeof(struct page) / sizeof(pte_t)
               = 64 / 8
               = 8 (pages)

Where n is how many pte entries which one page can contains. So the value of
n is (PAGE_SIZE / sizeof(pte_t)).

This optimization only supports 64-bit system, so the value of sizeof(pte_t)
is 8. And this optimization also applicable only when the size of ``struct page``
is a power of two. In most cases, the size of ``struct page`` is 64 bytes (e.g.
x86-64 and arm64). So if we use pmd level mapping for a HugeTLB page, the
size of ``struct page`` structs of it is 8 page frames which size depends on the
size of the base page.

For the HugeTLB page of the pud level mapping, then::

   struct_size = PAGE_SIZE / sizeof(pmd_t) * struct_size(pmd)
               = PAGE_SIZE / 8 * 8 (pages)
               = PAGE_SIZE (pages)

Where the struct_size(pmd) is the size of the ``struct page`` structs of a
HugeTLB page of the pmd level mapping.

E.g.: A 2MB HugeTLB page on x86_64 consists in 8 page frames while 1GB
HugeTLB page consists in 4096.

Next, we take the pmd level mapping of the HugeTLB page as an example to
show the internal implementation of this optimization. There are 8 pages
``struct page`` structs associated with a HugeTLB page which is pmd mapped.

Here is how things look before optimization::

    HugeTLB                  struct pages(8 pages)         page frame(8 pages)
 +-----------+ ---virt_to_page---> +-----------+   mapping to   +-----------+
 |           |                     |     0     | -------------> |     0     |
 |           |                     +-----------+                +-----------+
 |           |                     |     1     | -------------> |     1     |
 |           |                     +-----------+                +-----------+
 |           |                     |     2     | -------------> |     2     |
 |           |                     +-----------+                +-----------+
 |           |                     |     3     | -------------> |     3     |
 |           |                     +-----------+                +-----------+
 |           |                     |     4     | -------------> |     4     |
 |    PMD    |                     +-----------+                +-----------+
 |   level   |                     |     5     | -------------> |     5     |
 |  mapping  |                     +-----------+                +-----------+
 |           |                     |     6     | -------------> |     6     |
 |           |                     +-----------+                +-----------+
 |           |                     |     7     | -------------> |     7     |
 |           |                     +-----------+                +-----------+
 |           |
 |           |
 |           |
 +-----------+

The value of page->compound_head is the same for all tail pages. The first
page of ``struct page`` (page 0) associated with the HugeTLB page contains the 4
``struct page`` necessary to describe the HugeTLB. The only use of the remaining
pages of ``struct page`` (page 1 to page 7) is to point to page->compound_head.
Therefore, we can remap pages 1 to 7 to page 0. Only 1 page of ``struct page``
will be used for each HugeTLB page. This will allow us to free the remaining
7 pages to the buddy allocator.

Here is how things look after remapping::

    HugeTLB                  struct pages(8 pages)         page frame(8 pages)
 +-----------+ ---virt_to_page---> +-----------+   mapping to   +-----------+
 |           |                     |     0     | -------------> |     0     |
 |           |                     +-----------+                +-----------+
 |           |                     |     1     | ---------------^ ^ ^ ^ ^ ^ ^
 |           |                     +-----------+                  | | | | | |
 |           |                     |     2     | -----------------+ | | | | |
 |           |                     +-----------+                    | | | | |
 |           |                     |     3     | -------------------+ | | | |
 |           |                     +-----------+                      | | | |
 |           |                     |     4     | ---------------------+ | | |
 |    PMD    |                     +-----------+                        | | |
 |   level   |                     |     5     | -----------------------+ | |
 |  mapping  |                     +-----------+                          | |
 |           |                     |     6     | -------------------------+ |
 |           |                     +-----------+                            |
 |           |                     |     7     | ---------------------------+
 |           |                     +-----------+
 |           |
 |           |
 |           |
 +-----------+

When a HugeTLB is freed to the buddy system, we should allocate 7 pages for
vmemmap pages and restore the previous mapping relationship.

For the HugeTLB page of the pud level mapping. It is similar to the former.
We also can use this approach to free (PAGE_SIZE - 1) vmemmap pages.

Apart from the HugeTLB page of the pmd/pud level mapping, some architectures
(e.g. aarch64) provides a contiguous bit in the translation table entries
that hints to the MMU to indicate that it is one of a contiguous set of
entries that can be cached in a single TLB entry.

The contiguous bit is used to increase the mapping size at the pmd and pte
(last) level. So this type of HugeTLB page can be optimized only when its
size of the ``struct page`` structs is greater than **1** page.

Notice: The head vmemmap page is not freed to the buddy allocator and all
tail vmemmap pages are mapped to the head vmemmap page frame. So we can see
more than one ``struct page`` struct with ``PG_head`` (e.g. 8 per 2 MB HugeTLB
page) associated with each HugeTLB page. The ``compound_head()`` can handle
this correctly. There is only **one** head ``struct page``, the tail
``struct page`` with ``PG_head`` are fake head ``struct page``.  We need an
approach to distinguish between those two different types of ``struct page`` so
that ``compound_head()`` can return the real head ``struct page`` when the
parameter is the tail ``struct page`` but with ``PG_head``. The following code
snippet describes how to distinguish between real and fake head ``struct page``.

.. code-block:: c

	if (test_bit(PG_head, &page->flags)) {
		unsigned long head = READ_ONCE(page[1].compound_head);

		if (head & 1) {
			if (head == (unsigned long)page + 1)
				/* head struct page */
			else
				/* tail struct page */
		} else {
			/* head struct page */
		}
	}

We can safely access the field of the **page[1]** with ``PG_head`` because the
page is a compound page composed with at least two contiguous pages.
The implementation refers to ``page_fixed_fake_head()``.

Device DAX
==========

The device-dax interface uses the same tail deduplication technique explained
in the previous chapter, except when used with the vmemmap in
the device (altmap).

The following page sizes are supported in DAX: PAGE_SIZE (4K on x86_64),
PMD_SIZE (2M on x86_64) and PUD_SIZE (1G on x86_64).

The differences with HugeTLB are relatively minor.

It only use 3 ``struct page`` for storing all information as opposed
to 4 on HugeTLB pages.

There's no remapping of vmemmap given that device-dax memory is not part of
System RAM ranges initialized at boot. Thus the tail page deduplication
happens at a later stage when we populate the sections. HugeTLB reuses the
the head vmemmap page representing, whereas device-dax reuses the tail
vmemmap page. This results in only half of the savings compared to HugeTLB.

Deduplicated tail pages are not mapped read-only.

Here's how things look like on device-dax after the sections are populated::

 +-----------+ ---virt_to_page---> +-----------+   mapping to   +-----------+
 |           |                     |     0     | -------------> |     0     |
 |           |                     +-----------+                +-----------+
 |           |                     |     1     | -------------> |     1     |
 |           |                     +-----------+                +-----------+
 |           |                     |     2     | ----------------^ ^ ^ ^ ^ ^
 |           |                     +-----------+                   | | | | |
 |           |                     |     3     | ------------------+ | | | |
 |           |                     +-----------+                     | | | |
 |           |                     |     4     | --------------------+ | | |
 |    PMD    |                     +-----------+                       | | |
 |   level   |                     |     5     | ----------------------+ | |
 |  mapping  |                     +-----------+                         | |
 |           |                     |     6     | ------------------------+ |
 |           |                     +-----------+                           |
 |           |                     |     7     | --------------------------+
 |           |                     +-----------+
 |           |
 |           |
 |           |
 +-----------+


code:
.. _z3fold:

======
z3fold
======

z3fold is a special purpose allocator for storing compressed pages.
It is designed to store up to three compressed pages per physical page.
It is a zbud derivative which allows for higher compression
ratio keeping the simplicity and determinism of its predecessor.

The main differences between z3fold and zbud are:

* unlike zbud, z3fold allows for up to PAGE_SIZE allocations
* z3fold can hold up to 3 compressed pages in its page
* z3fold doesn't export any API itself and is thus intended to be used
  via the zpool API.

To keep the determinism and simplicity, z3fold, just like zbud, always
stores an integral number of compressed pages per page, but it can store
up to 3 pages unlike zbud which can store at most 2. Therefore the
compression ratio goes to around 2.7x while zbud's one is around 1.7x.

Unlike zbud (but like zsmalloc for that matter) z3fold_alloc() does not
return a dereferenceable pointer. Instead, it returns an unsigned long
handle which encodes actual location of the allocated object.

Keeping effective compression ratio close to zsmalloc's, z3fold doesn't
depend on MMU enabled and provides more predictable reclaim behavior
which makes it a better fit for small and response-critical systems.


code:
.. _zsmalloc:

========
zsmalloc
========

This allocator is designed for use with zram. Thus, the allocator is
supposed to work well under low memory conditions. In particular, it
never attempts higher order page allocation which is very likely to
fail under memory pressure. On the other hand, if we just use single
(0-order) pages, it would suffer from very high fragmentation --
any object of size PAGE_SIZE/2 or larger would occupy an entire page.
This was one of the major issues with its predecessor (xvmalloc).

To overcome these issues, zsmalloc allocates a bunch of 0-order pages
and links them together using various 'struct page' fields. These linked
pages act as a single higher-order page i.e. an object can span 0-order
page boundaries. The code refers to these linked pages as a single entity
called zspage.

For simplicity, zsmalloc can only allocate objects of size up to PAGE_SIZE
since this satisfies the requirements of all its current users (in the
worst case, page is incompressible and is thus stored "as-is" i.e. in
uncompressed form). For allocation requests larger than this size, failure
is returned (see zs_malloc).

Additionally, zs_malloc() does not return a dereferenceable pointer.
Instead, it returns an opaque handle (unsigned long) which encodes actual
location of the allocated object. The reason for this indirection is that
zsmalloc does not keep zspages permanently mapped since that would cause
issues on 32-bit systems where the VA region for kernel space mappings
is very small. So, before using the allocating memory, the object has to
be mapped using zs_map_object() to get a usable pointer and subsequently
unmapped using zs_unmap_object().

stat
====

With CONFIG_ZSMALLOC_STAT, we could see zsmalloc internal information via
``/sys/kernel/debug/zsmalloc/<user name>``. Here is a sample of stat output::

 # cat /sys/kernel/debug/zsmalloc/zram0/classes

 class  size almost_full almost_empty obj_allocated   obj_used pages_used pages_per_zspage
    ...
    ...
     9   176           0            1           186        129          8                4
    10   192           1            0          2880       2872        135                3
    11   208           0            1           819        795         42                2
    12   224           0            1           219        159         12                4
    ...
    ...

class
	index
size
	object size zspage stores
almost_empty
	the number of ZS_ALMOST_EMPTY zspages(see below)
almost_full
	the number of ZS_ALMOST_FULL zspages(see below)
obj_allocated
	the number of objects allocated
obj_used
	the number of objects allocated to the user
pages_used
	the number of pages allocated for the class
pages_per_zspage
	the number of 0-order pages to make a zspage

We assign a zspage to ZS_ALMOST_EMPTY fullness group when n <= N / f, where

* n = number of allocated objects
* N = total number of objects zspage can store
* f = fullness_threshold_frac(ie, 4 at the moment)

Similarly, we assign zspage to:

* ZS_ALMOST_FULL  when n > N / f
* ZS_EMPTY        when n == 0
* ZS_FULL         when n == N


code:
===================================
NetLabel CIPSO/IPv4 Protocol Engine
===================================

Paul Moore, paul.moore@hp.com

May 17, 2006

Overview
========

The NetLabel CIPSO/IPv4 protocol engine is based on the IETF Commercial
IP Security Option (CIPSO) draft from July 16, 1992.  A copy of this
draft can be found in this directory
(draft-ietf-cipso-ipsecurity-01.txt).  While the IETF draft never made
it to an RFC standard it has become a de-facto standard for labeled
networking and is used in many trusted operating systems.

Outbound Packet Processing
==========================

The CIPSO/IPv4 protocol engine applies the CIPSO IP option to packets by
adding the CIPSO label to the socket.  This causes all packets leaving the
system through the socket to have the CIPSO IP option applied.  The socket's
CIPSO label can be changed at any point in time, however, it is recommended
that it is set upon the socket's creation.  The LSM can set the socket's CIPSO
label by using the NetLabel security module API; if the NetLabel "domain" is
configured to use CIPSO for packet labeling then a CIPSO IP option will be
generated and attached to the socket.

Inbound Packet Processing
=========================

The CIPSO/IPv4 protocol engine validates every CIPSO IP option it finds at the
IP layer without any special handling required by the LSM.  However, in order
to decode and translate the CIPSO label on the packet the LSM must use the
NetLabel security module API to extract the security attributes of the packet.
This is typically done at the socket layer using the 'socket_sock_rcv_skb()'
LSM hook.

Label Translation
=================

The CIPSO/IPv4 protocol engine contains a mechanism to translate CIPSO security
attributes such as sensitivity level and category to values which are
appropriate for the host.  These mappings are defined as part of a CIPSO
Domain Of Interpretation (DOI) definition and are configured through the
NetLabel user space communication layer.  Each DOI definition can have a
different security attribute mapping table.

Label Translation Cache
=======================

The NetLabel system provides a framework for caching security attribute
mappings from the network labels to the corresponding LSM identifiers.  The
CIPSO/IPv4 protocol engine supports this caching mechanism.


code:
IETF CIPSO Working Group
16 July, 1992

                 COMMERCIAL IP SECURITY OPTION (CIPSO 2.2)

1.    Status

This Internet Draft provides the high level specification for a Commercial
IP Security Option (CIPSO).  This draft reflects the version as approved by
the CIPSO IETF Working Group.  Distribution of this memo is unlimited.

This document is an Internet Draft.  Internet Drafts are working documents
of the Internet Engineering Task Force (IETF), its Areas, and its Working
Groups. Note that other groups may also distribute working documents as
Internet Drafts.

Internet Drafts are draft documents valid for a maximum of six months.
Internet Drafts may be updated, replaced, or obsoleted by other documents
at any time.  It is not appropriate to use Internet Drafts as reference
material or to cite them other than as a "working draft" or "work in
progress."

Please check the I-D abstract listing contained in each Internet Draft
directory to learn the current status of this or any other Internet Draft.

2.    Background

Currently the Internet Protocol includes two security options.  One of
these options is the DoD Basic Security Option (BSO) (Type 130) which allows
IP datagrams to be labeled with security classifications.  This option
provides sixteen security classifications and a variable number of handling
restrictions.  To handle additional security information, such as security
categories or compartments, another security option (Type 133) exists and
is referred to as the DoD Extended Security Option (ESO).  The values for
the fixed fields within these two options are administered by the Defense
Information Systems Agency (DISA).

Computer vendors are now building commercial operating systems with
mandatory access controls and multi-level security.  These systems are
no longer built specifically for a particular group in the defense or
intelligence communities.  They are generally available commercial systems
for use in a variety of government and civil sector environments.

The small number of ESO format codes can not support all the possible
applications of a commercial security option.  The BSO and ESO were
designed to only support the United States DoD.  CIPSO has been designed
to support multiple security policies.  This Internet Draft provides the
format and procedures required to support a Mandatory Access Control
security policy.  Support for additional security policies shall be
defined in future RFCs.

Internet Draft, Expires 15 Jan 93                                 [PAGE 1]

CIPSO INTERNET DRAFT                                         16 July, 1992

3.    CIPSO Format

Option type: 134 (Class 0, Number 6, Copy on Fragmentation)
Option length: Variable

This option permits security related information to be passed between
systems within a single Domain of Interpretation (DOI).  A DOI is a
collection of systems which agree on the meaning of particular values
in the security option.  An authority that has been assigned a DOI
identifier will define a mapping between appropriate CIPSO field values
and their human readable equivalent.  This authority will distribute that
mapping to hosts within the authority's domain.  These mappings may be
sensitive, therefore a DOI authority is not required to make these
mappings available to anyone other than the systems that are included in
the DOI.

This option MUST be copied on fragmentation.  This option appears at most
once in a datagram.  All multi-octet fields in the option are defined to be
transmitted in network byte order.  The format of this option is as follows:

+----------+----------+------//------+-----------//---------+
| 10000110 | LLLLLLLL | DDDDDDDDDDDD | TTTTTTTTTTTTTTTTTTTT |
+----------+----------+------//------+-----------//---------+

  TYPE=134    OPTION    DOMAIN OF               TAGS
              LENGTH    INTERPRETATION

                Figure 1. CIPSO Format

3.1    Type

This field is 1 octet in length.  Its value is 134.

3.2    Length

This field is 1 octet in length.  It is the total length of the option
including the type and length fields.  With the current IP header length
restriction of 40 octets the value of this field MUST not exceed 40.

3.3    Domain of Interpretation Identifier

This field is an unsigned 32 bit integer.  The value 0 is reserved and MUST
not appear as the DOI identifier in any CIPSO option.  Implementations
should assume that the DOI identifier field is not aligned on any particular
byte boundary.

To conserve space in the protocol, security levels and categories are
represented by numbers rather than their ASCII equivalent.  This requires
a mapping table within CIPSO hosts to map these numbers to their
corresponding ASCII representations.  Non-related groups of systems may

Internet Draft, Expires 15 Jan 93                                 [PAGE 2]

CIPSO INTERNET DRAFT                                         16 July, 1992

have their own unique mappings.  For example, one group of systems may
use the number 5 to represent Unclassified while another group may use the
number 1 to represent that same security level.  The DOI identifier is used
to identify which mapping was used for the values within the option.

3.4    Tag Types

A common format for passing security related information is necessary
for interoperability.  CIPSO uses sets of "tags" to contain the security
information relevant to the data in the IP packet.  Each tag begins with
a tag type identifier followed by the length of the tag and ends with the
actual security information to be passed.  All multi-octet fields in a tag
are defined to be transmitted in network byte order.  Like the DOI
identifier field in the CIPSO header, implementations should assume that
all tags, as well as fields within a tag, are not aligned on any particular
octet boundary.   The tag types defined in this document contain alignment
bytes to assist alignment of some information, however alignment can not
be guaranteed if CIPSO is not the first IP option.

CIPSO tag types 0 through 127 are reserved for defining standard tag
formats.  Their definitions will be published in RFCs.  Tag types whose
identifiers are greater than 127 are defined by the DOI authority and may
only be meaningful in certain Domains of Interpretation.  For these tag
types, implementations will require the DOI identifier as well as the tag
number to determine the security policy and the format associated with the
tag.  Use of tag types above 127 are restricted to closed networks where
interoperability with other networks will not be an issue.  Implementations
that support a tag type greater than 127 MUST support at least one DOI that
requires only tag types 1 to 127.

Tag type 0 is reserved. Tag types 1, 2, and 5 are defined in this
Internet Draft.  Types 3 and 4 are reserved for work in progress.
The standard format for all current and future CIPSO tags is shown below:

+----------+----------+--------//--------+
| TTTTTTTT | LLLLLLLL | IIIIIIIIIIIIIIII |
+----------+----------+--------//--------+
    TAG       TAG         TAG
    TYPE      LENGTH      INFORMATION

    Figure 2:  Standard Tag Format

In the three tag types described in this document, the length and count
restrictions are based on the current IP limitation of 40 octets for all
IP options.  If the IP header is later expanded, then the length and count
restrictions specified in this document may increase to use the full area
provided for IP options.

3.4.1    Tag Type Classes

Tag classes consist of tag types that have common processing requirements
and support the same security policy.  The three tags defined in this
Internet Draft belong to the Mandatory Access Control (MAC) Sensitivity

Internet Draft, Expires 15 Jan 93                                 [PAGE 3]

CIPSO INTERNET DRAFT                                         16 July, 1992

class and support the MAC Sensitivity security policy.

3.4.2    Tag Type 1

This is referred to as the "bit-mapped" tag type.  Tag type 1 is included
in the MAC Sensitivity tag type class.  The format of this tag type is as
follows:

+----------+----------+----------+----------+--------//---------+
| 00000001 | LLLLLLLL | 00000000 | LLLLLLLL | CCCCCCCCCCCCCCCCC |
+----------+----------+----------+----------+--------//---------+

    TAG       TAG      ALIGNMENT  SENSITIVITY    BIT MAP OF
    TYPE      LENGTH   OCTET      LEVEL          CATEGORIES

            Figure 3. Tag Type 1 Format

3.4.2.1    Tag Type

This field is 1 octet in length and has a value of 1.

3.4.2.2    Tag Length

This field is 1 octet in length.  It is the total length of the tag type
including the type and length fields.  With the current IP header length
restriction of 40 bytes the value within this field is between 4 and 34.

3.4.2.3    Alignment Octet

This field is 1 octet in length and always has the value of 0.  Its purpose
is to align the category bitmap field on an even octet boundary.  This will
speed many implementations including router implementations.

3.4.2.4    Sensitivity Level

This field is 1 octet in length.  Its value is from 0 to 255.  The values
are ordered with 0 being the minimum value and 255 representing the maximum
value.

3.4.2.5    Bit Map of Categories

The length of this field is variable and ranges from 0 to 30 octets.  This
provides representation of categories 0 to 239.  The ordering of the bits
is left to right or MSB to LSB.  For example category 0 is represented by
the most significant bit of the first byte and category 15 is represented
by the least significant bit of the second byte.  Figure 4 graphically
shows this ordering.  Bit N is binary 1 if category N is part of the label
for the datagram, and bit N is binary 0 if category N is not part of the
label.  Except for the optimized tag 1 format described in the next section,

Internet Draft, Expires 15 Jan 93                                 [PAGE 4]

CIPSO INTERNET DRAFT                                         16 July, 1992

minimal encoding SHOULD be used resulting in no trailing zero octets in the
category bitmap.

        octet 0  octet 1  octet 2  octet 3  octet 4  octet 5
        XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX . . .
bit     01234567 89111111 11112222 22222233 33333333 44444444
number             012345 67890123 45678901 23456789 01234567

            Figure 4. Ordering of Bits in Tag 1 Bit Map

3.4.2.6    Optimized Tag 1 Format

Routers work most efficiently when processing fixed length fields.  To
support these routers there is an optimized form of tag type 1.  The format
does not change.  The only change is to the category bitmap which is set to
a constant length of 10 octets.  Trailing octets required to fill out the 10
octets are zero filled.  Ten octets, allowing for 80 categories, was chosen
because it makes the total length of the CIPSO option 20 octets.  If CIPSO
is the only option then the option will be full word aligned and additional
filler octets will not be required.

3.4.3    Tag Type 2

This is referred to as the "enumerated" tag type.  It is used to describe
large but sparsely populated sets of categories.  Tag type 2 is in the MAC
Sensitivity tag type class.  The format of this tag type is as follows:

+----------+----------+----------+----------+-------------//-------------+
| 00000010 | LLLLLLLL | 00000000 | LLLLLLLL | CCCCCCCCCCCCCCCCCCCCCCCCCC |
+----------+----------+----------+----------+-------------//-------------+

    TAG       TAG      ALIGNMENT  SENSITIVITY         ENUMERATED
    TYPE      LENGTH   OCTET      LEVEL               CATEGORIES

                Figure 5. Tag Type 2 Format

3.4.3.1     Tag Type

This field is one octet in length and has a value of 2.

3.4.3.2    Tag Length

This field is 1 octet in length. It is the total length of the tag type
including the type and length fields.  With the current IP header length
restriction of 40 bytes the value within this field is between 4 and 34.

3.4.3.3    Alignment Octet

This field is 1 octet in length and always has the value of 0.  Its purpose
is to align the category field on an even octet boundary.  This will

Internet Draft, Expires 15 Jan 93                                 [PAGE 5]

CIPSO INTERNET DRAFT                                         16 July, 1992

speed many implementations including router implementations.

3.4.3.4    Sensitivity Level

This field is 1 octet in length. Its value is from 0 to 255.  The values
are ordered with 0 being the minimum value and 255 representing the
maximum value.

3.4.3.5    Enumerated Categories

In this tag, categories are represented by their actual value rather than
by their position within a bit field.  The length of each category is 2
octets.  Up to 15 categories may be represented by this tag.  Valid values
for categories are 0 to 65534.  Category 65535 is not a valid category
value.  The categories MUST be listed in ascending order within the tag.

3.4.4    Tag Type 5

This is referred to as the "range" tag type.  It is used to represent
labels where all categories in a range, or set of ranges, are included
in the sensitivity label.  Tag type 5 is in the MAC Sensitivity tag type
class.  The format of this tag type is as follows:

+----------+----------+----------+----------+------------//-------------+
| 00000101 | LLLLLLLL | 00000000 | LLLLLLLL |  Top/Bottom | Top/Bottom  |
+----------+----------+----------+----------+------------//-------------+

    TAG       TAG      ALIGNMENT  SENSITIVITY        CATEGORY RANGES
    TYPE      LENGTH   OCTET      LEVEL

                     Figure 6. Tag Type 5 Format

3.4.4.1     Tag Type

This field is one octet in length and has a value of 5.

3.4.4.2    Tag Length

This field is 1 octet in length. It is the total length of the tag type
including the type and length fields.  With the current IP header length
restriction of 40 bytes the value within this field is between 4 and 34.

3.4.4.3    Alignment Octet

This field is 1 octet in length and always has the value of 0.  Its purpose
is to align the category range field on an even octet boundary.  This will
speed many implementations including router implementations.

Internet Draft, Expires 15 Jan 93                                 [PAGE 6]

CIPSO INTERNET DRAFT                                         16 July, 1992

3.4.4.4    Sensitivity Level

This field is 1 octet in length. Its value is from 0 to 255.  The values
are ordered with 0 being the minimum value and 255 representing the maximum
value.

3.4.4.5    Category Ranges

A category range is a 4 octet field comprised of the 2 octet index of the
highest numbered category followed by the 2 octet index of the lowest
numbered category.  These range endpoints are inclusive within the range of
categories.  All categories within a range are included in the sensitivity
label.  This tag may contain a maximum of 7 category pairs.  The bottom
category endpoint for the last pair in the tag MAY be omitted and SHOULD be
assumed to be 0.  The ranges MUST be non-overlapping and be listed in
descending order.  Valid values for categories are 0 to 65534.  Category
65535 is not a valid category value.

3.4.5     Minimum Requirements

A CIPSO implementation MUST be capable of generating at least tag type 1 in
the non-optimized form.  In addition, a CIPSO implementation MUST be able
to receive any valid tag type 1 even those using the optimized tag type 1
format.

4.    Configuration Parameters

The configuration parameters defined below are required for all CIPSO hosts,
gateways, and routers that support multiple sensitivity labels.  A CIPSO
host is defined to be the origination or destination system for an IP
datagram.  A CIPSO gateway provides IP routing services between two or more
IP networks and may be required to perform label translations between
networks.  A CIPSO gateway may be an enhanced CIPSO host or it may just
provide gateway services with no end system CIPSO capabilities.  A CIPSO
router is a dedicated IP router that routes IP datagrams between two or more
IP networks.

An implementation of CIPSO on a host MUST have the capability to reject a
datagram for reasons that the information contained can not be adequately
protected by the receiving host or if acceptance may result in violation of
the host or network security policy.  In addition, a CIPSO gateway or router
MUST be able to reject datagrams going to networks that can not provide
adequate protection or may violate the network's security policy.  To
provide this capability the following minimal set of configuration
parameters are required for CIPSO implementations:

HOST_LABEL_MAX - This parameter contains the maximum sensitivity label that
a CIPSO host is authorized to handle.  All datagrams that have a label
greater than this maximum MUST be rejected by the CIPSO host.  This
parameter does not apply to CIPSO gateways or routers.  This parameter need
not be defined explicitly as it can be implicitly derived from the
PORT_LABEL_MAX parameters for the associated interfaces.

Internet Draft, Expires 15 Jan 93                                 [PAGE 7]

CIPSO INTERNET DRAFT                                         16 July, 1992

HOST_LABEL_MIN - This parameter contains the minimum sensitivity label that
a CIPSO host is authorized to handle.  All datagrams that have a label less
than this minimum MUST be rejected by the CIPSO host.  This parameter does
not apply to CIPSO gateways or routers.  This parameter need not be defined
explicitly as it can be implicitly derived from the PORT_LABEL_MIN
parameters for the associated interfaces.

PORT_LABEL_MAX - This parameter contains the maximum sensitivity label for
all datagrams that may exit a particular network interface port.  All
outgoing datagrams that have a label greater than this maximum MUST be
rejected by the CIPSO system.  The label within this parameter MUST be
less than or equal to the label within the HOST_LABEL_MAX parameter.  This
parameter does not apply to CIPSO hosts that support only one network port.

PORT_LABEL_MIN - This parameter contains the minimum sensitivity label for
all datagrams that may exit a particular network interface port.  All
outgoing datagrams that have a label less than this minimum MUST be
rejected by the CIPSO system.  The label within this parameter MUST be
greater than or equal to the label within the HOST_LABEL_MIN parameter.
This parameter does not apply to CIPSO hosts that support only one network
port.

PORT_DOI - This parameter is used to assign a DOI identifier value to a
particular network interface port.  All CIPSO labels within datagrams
going out this port MUST use the specified DOI identifier.  All CIPSO
hosts and gateways MUST support either this parameter, the NET_DOI
parameter, or the HOST_DOI parameter.

NET_DOI - This parameter is used to assign a DOI identifier value to a
particular IP network address.  All CIPSO labels within datagrams destined
for the particular IP network MUST use the specified DOI identifier.  All
CIPSO hosts and gateways MUST support either this parameter, the PORT_DOI
parameter, or the HOST_DOI parameter.

HOST_DOI - This parameter is used to assign a DOI identifier value to a
particular IP host address.  All CIPSO labels within datagrams destined for
the particular IP host will use the specified DOI identifier.  All CIPSO
hosts and gateways MUST support either this parameter, the PORT_DOI
parameter, or the NET_DOI parameter.

This list represents the minimal set of configuration parameters required
to be compliant.  Implementors are encouraged to add to this list to
provide enhanced functionality and control.  For example, many security
policies may require both incoming and outgoing datagrams be checked against
the port and host label ranges.

4.1    Port Range Parameters

The labels represented by the PORT_LABEL_MAX and PORT_LABEL_MIN parameters
MAY be in CIPSO or local format.  Some CIPSO systems, such as routers, may
want to have the range parameters expressed in CIPSO format so that incoming
labels do not have to be converted to a local format before being compared
against the range.  If multiple DOIs are supported by one of these CIPSO

Internet Draft, Expires 15 Jan 93                                 [PAGE 8]

CIPSO INTERNET DRAFT                                         16 July, 1992

systems then multiple port range parameters would be needed, one set for
each DOI supported on a particular port.

The port range will usually represent the total set of labels that may
exist on the logical network accessed through the corresponding network
interface.  It may, however, represent a subset of these labels that are
allowed to enter the CIPSO system.

4.2    Single Label CIPSO Hosts

CIPSO implementations that support only one label are not required to
support the parameters described above.  These limited implementations are
only required to support a NET_LABEL parameter.  This parameter contains
the CIPSO label that may be inserted in datagrams that exit the host.  In
addition, the host MUST reject any incoming datagram that has a label which
is not equivalent to the NET_LABEL parameter.

5.    Handling Procedures

This section describes the processing requirements for incoming and
outgoing IP datagrams.  Just providing the correct CIPSO label format
is not enough.  Assumptions will be made by one system on how a
receiving system will handle the CIPSO label.  Wrong assumptions may
lead to non-interoperability or even a security incident.  The
requirements described below represent the minimal set needed for
interoperability and that provide users some level of confidence.
Many other requirements could be added to increase user confidence,
however at the risk of restricting creativity and limiting vendor
participation.

5.1    Input Procedures

All datagrams received through a network port MUST have a security label
associated with them, either contained in the datagram or assigned to the
receiving port.  Without this label the host, gateway, or router will not
have the information it needs to make security decisions.  This security
label will be obtained from the CIPSO if the option is present in the
datagram.  See section 4.1.2 for handling procedures for unlabeled
datagrams.  This label will be compared against the PORT (if appropriate)
and HOST configuration parameters defined in section 3.

If any field within the CIPSO option, such as the DOI identifier, is not
recognized the IP datagram is discarded and an ICMP "parameter problem"
(type 12) is generated and returned.  The ICMP code field is set to "bad
parameter" (code 0) and the pointer is set to the start of the CIPSO field
that is unrecognized.

If the contents of the CIPSO are valid but the security label is
outside of the configured host or port label range, the datagram is
discarded and an ICMP "destination unreachable" (type 3) is generated
and returned.  The code field of the ICMP is set to "communication with
destination network administratively prohibited" (code 9) or to

Internet Draft, Expires 15 Jan 93                                 [PAGE 9]

CIPSO INTERNET DRAFT                                         16 July, 1992

"communication with destination host administratively prohibited"
(code 10).  The value of the code field used is dependent upon whether
the originator of the ICMP message is acting as a CIPSO host or a CIPSO
gateway.  The recipient of the ICMP message MUST be able to handle either
value.  The same procedure is performed if a CIPSO can not be added to an
IP packet because it is too large to fit in the IP options area.

If the error is triggered by receipt of an ICMP message, the message
is discarded and no response is permitted (consistent with general ICMP
processing rules).

5.1.1    Unrecognized tag types

The default condition for any CIPSO implementation is that an
unrecognized tag type MUST be treated as a "parameter problem" and
handled as described in section 4.1.  A CIPSO implementation MAY allow
the system administrator to identify tag types that may safely be
ignored.  This capability is an allowable enhancement, not a
requirement.

5.1.2    Unlabeled Packets

A network port may be configured to not require a CIPSO label for all
incoming  datagrams.  For this configuration a CIPSO label must be
assigned to that network port and associated with all unlabeled IP
datagrams.  This capability might be used for single level networks or
networks that have CIPSO and non-CIPSO hosts and the non-CIPSO hosts
all operate at the same label.

If a CIPSO option is required and none is found, the datagram is
discarded and an ICMP "parameter problem" (type 12) is generated and
returned to the originator of the datagram.  The code field of the ICMP
is set to "option missing" (code 1) and the ICMP pointer is set to 134
(the value of the option type for the missing CIPSO option).

5.2    Output Procedures

A CIPSO option MUST appear only once in a datagram.  Only one tag type
from the MAC Sensitivity class MAY be included in a CIPSO option.  Given
the current set of defined tag types, this means that CIPSO labels at
first will contain only one tag.

All datagrams leaving a CIPSO system MUST meet the following condition:

        PORT_LABEL_MIN <= CIPSO label <= PORT_LABEL_MAX

If this condition is not satisfied the datagram MUST be discarded.
If the CIPSO system only supports one port, the HOST_LABEL_MIN and the
HOST_LABEL_MAX parameters MAY be substituted for the PORT parameters in
the above condition.

The DOI identifier to be used for all outgoing datagrams is configured by

Internet Draft, Expires 15 Jan 93                                 [PAGE 10]

CIPSO INTERNET DRAFT                                         16 July, 1992

the administrator.  If port level DOI identifier assignment is used, then
the PORT_DOI configuration parameter MUST contain the DOI identifier to
use.  If network level DOI assignment is used, then the NET_DOI parameter
MUST contain the DOI identifier to use.  And if host level DOI assignment
is employed, then the HOST_DOI parameter MUST contain the DOI identifier
to use.  A CIPSO implementation need only support one level of DOI
assignment.

5.3    DOI Processing Requirements

A CIPSO implementation MUST support at least one DOI and SHOULD support
multiple DOIs.  System and network administrators are cautioned to
ensure that at least one DOI is common within an IP network to allow for
broadcasting of IP datagrams.

CIPSO gateways MUST be capable of translating a CIPSO option from one
DOI to another when forwarding datagrams between networks.  For
efficiency purposes this capability is only a desired feature for CIPSO
routers.

5.4    Label of ICMP Messages

The CIPSO label to be used on all outgoing ICMP messages MUST be equivalent
to the label of the datagram that caused the ICMP message.  If the ICMP was
generated due to a problem associated with the original CIPSO label then the
following responses are allowed:

  a.  Use the CIPSO label of the original IP datagram
  b.  Drop the original datagram with no return message generated

In most cases these options will have the same effect.  If you can not
interpret the label or if it is outside the label range of your host or
interface then an ICMP message with the same label will probably not be
able to exit the system.

6.    Assignment of DOI Identifier Numbers                                   =

Requests for assignment of a DOI identifier number should be addressed to
the Internet Assigned Numbers Authority (IANA).

7.    Acknowledgements

Much of the material in this RFC is based on (and copied from) work
done by Gary Winiger of Sun Microsystems and published as Commercial
IP Security Option at the INTEROP 89, Commercial IPSO Workshop.

8.    Author's Address

To submit mail for distribution to members of the IETF CIPSO Working
Group, send mail to: cipso@wdl1.wdl.loral.com.

Internet Draft, Expires 15 Jan 93                                 [PAGE 11]

CIPSO INTERNET DRAFT                                         16 July, 1992

To be added to or deleted from this distribution, send mail to:
cipso-request@wdl1.wdl.loral.com.

9.    References

RFC 1038, "Draft Revised IP Security Option", M. St. Johns, IETF, January
1988.

RFC 1108, "U.S. Department of Defense Security Options
for the Internet Protocol", Stephen Kent, IAB, 1 March, 1991.

Internet Draft, Expires 15 Jan 93                                 [PAGE 12]


code:
Draft IETF CIPSO IP Security
----------------------------

 .. include:: draft-ietf-cipso-ipsecurity-01.txt
    :literal:


code:
.. SPDX-License-Identifier: GPL-2.0

========
NetLabel
========

.. toctree::
    :maxdepth: 1

    introduction
    cipso_ipv4
    lsm_interface

    draft_ietf

.. only::  subproject and html

   Indices
   =======

   * :ref:`genindex`


code:
=====================
NetLabel Introduction
=====================

Paul Moore, paul.moore@hp.com

August 2, 2006

Overview
========

NetLabel is a mechanism which can be used by kernel security modules to attach
security attributes to outgoing network packets generated from user space
applications and read security attributes from incoming network packets.  It
is composed of three main components, the protocol engines, the communication
layer, and the kernel security module API.

Protocol Engines
================

The protocol engines are responsible for both applying and retrieving the
network packet's security attributes.  If any translation between the network
security attributes and those on the host are required then the protocol
engine will handle those tasks as well.  Other kernel subsystems should
refrain from calling the protocol engines directly, instead they should use
the NetLabel kernel security module API described below.

Detailed information about each NetLabel protocol engine can be found in this
directory.

Communication Layer
===================

The communication layer exists to allow NetLabel configuration and monitoring
from user space.  The NetLabel communication layer uses a message based
protocol built on top of the Generic NETLINK transport mechanism.  The exact
formatting of these NetLabel messages as well as the Generic NETLINK family
names can be found in the 'net/netlabel/' directory as comments in the
header files as well as in 'include/net/netlabel.h'.

Security Module API
===================

The purpose of the NetLabel security module API is to provide a protocol
independent interface to the underlying NetLabel protocol engines.  In addition
to protocol independence, the security module API is designed to be completely
LSM independent which should allow multiple LSMs to leverage the same code
base.

Detailed information about the NetLabel security module API can be found in the
'include/net/netlabel.h' header file as well as the 'lsm_interface.txt' file
found in this directory.


code:
========================================
NetLabel Linux Security Module Interface
========================================

Paul Moore, paul.moore@hp.com

May 17, 2006

Overview
========

NetLabel is a mechanism which can set and retrieve security attributes from
network packets.  It is intended to be used by LSM developers who want to make
use of a common code base for several different packet labeling protocols.
The NetLabel security module API is defined in 'include/net/netlabel.h' but a
brief overview is given below.

NetLabel Security Attributes
============================

Since NetLabel supports multiple different packet labeling protocols and LSMs
it uses the concept of security attributes to refer to the packet's security
labels.  The NetLabel security attributes are defined by the
'netlbl_lsm_secattr' structure in the NetLabel header file.  Internally the
NetLabel subsystem converts the security attributes to and from the correct
low-level packet label depending on the NetLabel build time and run time
configuration.  It is up to the LSM developer to translate the NetLabel
security attributes into whatever security identifiers are in use for their
particular LSM.

NetLabel LSM Protocol Operations
================================

These are the functions which allow the LSM developer to manipulate the labels
on outgoing packets as well as read the labels on incoming packets.  Functions
exist to operate both on sockets as well as the sk_buffs directly.  These high
level functions are translated into low level protocol operations based on how
the administrator has configured the NetLabel subsystem.

NetLabel Label Mapping Cache Operations
=======================================

Depending on the exact configuration, translation between the network packet
label and the internal LSM security identifier can be time consuming.  The
NetLabel label mapping cache is a caching mechanism which can be used to
sidestep much of this overhead once a mapping has been established.  Once the
LSM has received a packet, used NetLabel to decode its security attributes,
and translated the security attributes into a LSM internal identifier the LSM
can use the NetLabel caching functions to associate the LSM internal
identifier with the network packet's label.  This means that in the future
when a incoming packet matches a cached value not only are the internal
NetLabel translation mechanisms bypassed but the LSM translation mechanisms are
bypassed as well which should result in a significant reduction in overhead.


code:
.. SPDX-License-Identifier: GPL-2.0

==============================================
Netdev private dataroom for 6lowpan interfaces
==============================================

All 6lowpan able net devices, means all interfaces with ARPHRD_6LOWPAN,
must have "struct lowpan_priv" placed at beginning of netdev_priv.

The priv_size of each interface should be calculate by::

 dev->priv_size = LOWPAN_PRIV_SIZE(LL_6LOWPAN_PRIV_DATA);

Where LL_PRIV_6LOWPAN_DATA is sizeof linklayer 6lowpan private data struct.
To access the LL_PRIV_6LOWPAN_DATA structure you can cast::

 lowpan_priv(dev)-priv;

to your LL_6LOWPAN_PRIV_DATA structure.

Before registering the lowpan netdev interface you must run::

 lowpan_netdev_setup(dev, LOWPAN_LLTYPE_FOOBAR);

wheres LOWPAN_LLTYPE_FOOBAR is a define for your 6LoWPAN linklayer type of
enum lowpan_lltypes.

Example to evaluate the private usually you can do::

 static inline struct lowpan_priv_foobar *
 lowpan_foobar_priv(struct net_device *dev)
 {
	return (struct lowpan_priv_foobar *)lowpan_priv(dev)->priv;
 }

 switch (dev->type) {
 case ARPHRD_6LOWPAN:
	lowpan_priv = lowpan_priv(dev);
	/* do great stuff which is ARPHRD_6LOWPAN related */
	switch (lowpan_priv->lltype) {
	case LOWPAN_LLTYPE_FOOBAR:
		/* do 802.15.4 6LoWPAN handling here */
		lowpan_foobar_priv(dev)->bar = foo;
		break;
	...
	}
	break;
 ...
 }

In case of generic 6lowpan branch ("net/6lowpan") you can remove the check
on ARPHRD_6LOWPAN, because you can be sure that these function are called
by ARPHRD_6LOWPAN interfaces.


code:
.. SPDX-License-Identifier: GPL-2.0

==============
6pack Protocol
==============

This is the 6pack-mini-HOWTO, written by

Andreas Könsgen DG3KQ

:Internet: ajk@comnets.uni-bremen.de
:AMPR-net: dg3kq@db0pra.ampr.org
:AX.25:    dg3kq@db0ach.#nrw.deu.eu

Last update: April 7, 1998

1. What is 6pack, and what are the advantages to KISS?
======================================================

6pack is a transmission protocol for data exchange between the PC and
the TNC over a serial line. It can be used as an alternative to KISS.

6pack has two major advantages:

- The PC is given full control over the radio
  channel. Special control data is exchanged between the PC and the TNC so
  that the PC knows at any time if the TNC is receiving data, if a TNC
  buffer underrun or overrun has occurred, if the PTT is
  set and so on. This control data is processed at a higher priority than
  normal data, so a data stream can be interrupted at any time to issue an
  important event. This helps to improve the channel access and timing
  algorithms as everything is computed in the PC. It would even be possible
  to experiment with something completely different from the known CSMA and
  DAMA channel access methods.
  This kind of real-time control is especially important to supply several
  TNCs that are connected between each other and the PC by a daisy chain
  (however, this feature is not supported yet by the Linux 6pack driver).

- Each packet transferred over the serial line is supplied with a checksum,
  so it is easy to detect errors due to problems on the serial line.
  Received packets that are corrupt are not passed on to the AX.25 layer.
  Damaged packets that the TNC has received from the PC are not transmitted.

More details about 6pack are described in the file 6pack.ps that is located
in the doc directory of the AX.25 utilities package.

2. Who has developed the 6pack protocol?
========================================

The 6pack protocol has been developed by Ekki Plicht DF4OR, Henning Rech
DF9IC and Gunter Jost DK7WJ. A driver for 6pack, written by Gunter Jost and
Matthias Welwarsky DG2FEF, comes along with the PC version of FlexNet.
They have also written a firmware for TNCs to perform the 6pack
protocol (see section 4 below).

3. Where can I get the latest version of 6pack for LinuX?
=========================================================

At the moment, the 6pack stuff can obtained via anonymous ftp from
db0bm.automation.fh-aachen.de. In the directory /incoming/dg3kq,
there is a file named 6pack.tgz.

4. Preparing the TNC for 6pack operation
========================================

To be able to use 6pack, a special firmware for the TNC is needed. The EPROM
of a newly bought TNC does not contain 6pack, so you will have to
program an EPROM yourself. The image file for 6pack EPROMs should be
available on any packet radio box where PC/FlexNet can be found. The name of
the file is 6pack.bin. This file is copyrighted and maintained by the FlexNet
team. It can be used under the terms of the license that comes along
with PC/FlexNet. Please do not ask me about the internals of this file as I
don't know anything about it. I used a textual description of the 6pack
protocol to program the Linux driver.

TNCs contain a 64kByte EPROM, the lower half of which is used for
the firmware/KISS. The upper half is either empty or is sometimes
programmed with software called TAPR. In the latter case, the TNC
is supplied with a DIP switch so you can easily change between the
two systems. When programming a new EPROM, one of the systems is replaced
by 6pack. It is useful to replace TAPR, as this software is rarely used
nowadays. If your TNC is not equipped with the switch mentioned above, you
can build in one yourself that switches over the highest address pin
of the EPROM between HIGH and LOW level. After having inserted the new EPROM
and switched to 6pack, apply power to the TNC for a first test. The connect
and the status LED are lit for about a second if the firmware initialises
the TNC correctly.

5. Building and installing the 6pack driver
===========================================

The driver has been tested with kernel version 2.1.90. Use with older
kernels may lead to a compilation error because the interface to a kernel
function has been changed in the 2.1.8x kernels.

How to turn on 6pack support:
=============================

- In the linux kernel configuration program, select the code maturity level
  options menu and turn on the prompting for development drivers.

- Select the amateur radio support menu and turn on the serial port 6pack
  driver.

- Compile and install the kernel and the modules.

To use the driver, the kissattach program delivered with the AX.25 utilities
has to be modified.

- Do a cd to the directory that holds the kissattach sources. Edit the
  kissattach.c file. At the top, insert the following lines::

    #ifndef N_6PACK
    #define N_6PACK (N_AX25+1)
    #endif

  Then find the line:

    int disc = N_AX25;

  and replace N_AX25 by N_6PACK.

- Recompile kissattach. Rename it to spattach to avoid confusions.

Installing the driver:
----------------------

- Do an insmod 6pack. Look at your /var/log/messages file to check if the
  module has printed its initialization message.

- Do a spattach as you would launch kissattach when starting a KISS port.
  Check if the kernel prints the message '6pack: TNC found'.

- From here, everything should work as if you were setting up a KISS port.
  The only difference is that the network device that represents
  the 6pack port is called sp instead of sl or ax. So, sp0 would be the
  first 6pack port.

Although the driver has been tested on various platforms, I still declare it
ALPHA. BE CAREFUL! Sync your disks before insmoding the 6pack module
and spattaching. Watch out if your computer behaves strangely. Read section
6 of this file about known problems.

Note that the connect and status LEDs of the TNC are controlled in a
different way than they are when the TNC is used with PC/FlexNet. When using
FlexNet, the connect LED is on if there is a connection; the status LED is
on if there is data in the buffer of the PC's AX.25 engine that has to be
transmitted. Under Linux, the 6pack layer is beyond the AX.25 layer,
so the 6pack driver doesn't know anything about connects or data that
has not yet been transmitted. Therefore the LEDs are controlled
as they are in KISS mode: The connect LED is turned on if data is transferred
from the PC to the TNC over the serial line, the status LED if data is
sent to the PC.

6. Known problems
=================

When testing the driver with 2.0.3x kernels and
operating with data rates on the radio channel of 9600 Baud or higher,
the driver may, on certain systems, sometimes print the message '6pack:
bad checksum', which is due to data loss if the other station sends two
or more subsequent packets. I have been told that this is due to a problem
with the serial driver of 2.0.3x kernels. I don't know yet if the problem
still exists with 2.1.x kernels, as I have heard that the serial driver
code has been changed with 2.1.x.

When shutting down the sp interface with ifconfig, the kernel crashes if
there is still an AX.25 connection left over which an IP connection was
running, even if that IP connection is already closed. The problem does not
occur when there is a bare AX.25 connection still running. I don't know if
this is a problem of the 6pack driver or something else in the kernel.

The driver has been tested as a module, not yet as a kernel-builtin driver.

The 6pack protocol supports daisy-chaining of TNCs in a token ring, which is
connected to one serial port of the PC. This feature is not implemented
and at least at the moment I won't be able to do it because I do not have
the opportunity to build a TNC daisy-chain and test it.

Some of the comments in the source code are inaccurate. They are left from
the SLIP/KISS driver, from which the 6pack driver has been derived.
I haven't modified or removed them yet -- sorry! The code itself needs
some cleaning and optimizing. This will be done in a later release.

If you encounter a bug or if you have a question or suggestion concerning the
driver, feel free to mail me, using the addresses given at the beginning of
this file.

Have fun!

Andreas