code:
.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>
.. _switchdev:

===============================================
Ethernet switch device driver model (switchdev)
===============================================

Copyright |copy| 2014 Jiri Pirko <jiri@resnulli.us>

Copyright |copy| 2014-2015 Scott Feldman <sfeldma@gmail.com>

The Ethernet switch device driver model (switchdev) is an in-kernel driver
model for switch devices which offload the forwarding (data) plane from the
kernel.

Figure 1 is a block diagram showing the components of the switchdev model for
an example setup using a data-center-class switch ASIC chip.  Other setups
with SR-IOV or soft switches, such as OVS, are possible.

::

			     User-space tools

       user space                   |
      +-------------------------------------------------------------------+
       kernel                       | Netlink
				    |
		     +--------------+-------------------------------+
		     |         Network stack                        |
		     |           (Linux)                            |
		     |                                              |
		     +----------------------------------------------+

			   sw1p2     sw1p4     sw1p6
		      sw1p1  +  sw1p3  +  sw1p5  +          eth1
			+    |    +    |    +    |            +
			|    |    |    |    |    |            |
		     +--+----+----+----+----+----+---+  +-----+-----+
		     |         Switch driver         |  |    mgmt   |
		     |        (this document)        |  |   driver  |
		     |                               |  |           |
		     +--------------+----------------+  +-----------+
				    |
       kernel                       | HW bus (eg PCI)
      +-------------------------------------------------------------------+
       hardware                     |
		     +--------------+----------------+
		     |         Switch device (sw1)   |
		     |  +----+                       +--------+
		     |  |    v offloaded data path   | mgmt port
		     |  |    |                       |
		     +--|----|----+----+----+----+---+
			|    |    |    |    |    |
			+    +    +    +    +    +
		       p1   p2   p3   p4   p5   p6

			     front-panel ports

				    Fig 1.

Include Files
-------------

::

    #include <linux/netdevice.h>
    #include <net/switchdev.h>

Configuration
-------------

Use "depends NET_SWITCHDEV" in driver's Kconfig to ensure switchdev model
support is built for driver.

Switch Ports
------------

On switchdev driver initialization, the driver will allocate and register a
struct net_device (using register_netdev()) for each enumerated physical switch
port, called the port netdev.  A port netdev is the software representation of
the physical port and provides a conduit for control traffic to/from the
controller (the kernel) and the network, as well as an anchor point for higher
level constructs such as bridges, bonds, VLANs, tunnels, and L3 routers.  Using
standard netdev tools (iproute2, ethtool, etc), the port netdev can also
provide to the user access to the physical properties of the switch port such
as PHY link state and I/O statistics.

There is (currently) no higher-level kernel object for the switch beyond the
port netdevs.  All of the switchdev driver ops are netdev ops or switchdev ops.

A switch management port is outside the scope of the switchdev driver model.
Typically, the management port is not participating in offloaded data plane and
is loaded with a different driver, such as a NIC driver, on the management port
device.

Switch ID
^^^^^^^^^

The switchdev driver must implement the net_device operation
ndo_get_port_parent_id for each port netdev, returning the same physical ID for
each port of a switch. The ID must be unique between switches on the same
system. The ID does not need to be unique between switches on different
systems.

The switch ID is used to locate ports on a switch and to know if aggregated
ports belong to the same switch.

Port Netdev Naming
^^^^^^^^^^^^^^^^^^

Udev rules should be used for port netdev naming, using some unique attribute
of the port as a key, for example the port MAC address or the port PHYS name.
Hard-coding of kernel netdev names within the driver is discouraged; let the
kernel pick the default netdev name, and let udev set the final name based on a
port attribute.

Using port PHYS name (ndo_get_phys_port_name) for the key is particularly
useful for dynamically-named ports where the device names its ports based on
external configuration.  For example, if a physical 40G port is split logically
into 4 10G ports, resulting in 4 port netdevs, the device can give a unique
name for each port using port PHYS name.  The udev rule would be::

    SUBSYSTEM=="net", ACTION=="add", ATTR{phys_switch_id}=="<phys_switch_id>", \
	    ATTR{phys_port_name}!="", NAME="swX$attr{phys_port_name}"

Suggested naming convention is "swXpYsZ", where X is the switch name or ID, Y
is the port name or ID, and Z is the sub-port name or ID.  For example, sw1p1s0
would be sub-port 0 on port 1 on switch 1.

Port Features
^^^^^^^^^^^^^

NETIF_F_NETNS_LOCAL

If the switchdev driver (and device) only supports offloading of the default
network namespace (netns), the driver should set this feature flag to prevent
the port netdev from being moved out of the default netns.  A netns-aware
driver/device would not set this flag and be responsible for partitioning
hardware to preserve netns containment.  This means hardware cannot forward
traffic from a port in one namespace to another port in another namespace.

Port Topology
^^^^^^^^^^^^^

The port netdevs representing the physical switch ports can be organized into
higher-level switching constructs.  The default construct is a standalone
router port, used to offload L3 forwarding.  Two or more ports can be bonded
together to form a LAG.  Two or more ports (or LAGs) can be bridged to bridge
L2 networks.  VLANs can be applied to sub-divide L2 networks.  L2-over-L3
tunnels can be built on ports.  These constructs are built using standard Linux
tools such as the bridge driver, the bonding/team drivers, and netlink-based
tools such as iproute2.

The switchdev driver can know a particular port's position in the topology by
monitoring NETDEV_CHANGEUPPER notifications.  For example, a port moved into a
bond will see its upper master change.  If that bond is moved into a bridge,
the bond's upper master will change.  And so on.  The driver will track such
movements to know what position a port is in in the overall topology by
registering for netdevice events and acting on NETDEV_CHANGEUPPER.

L2 Forwarding Offload
---------------------

The idea is to offload the L2 data forwarding (switching) path from the kernel
to the switchdev device by mirroring bridge FDB entries down to the device.  An
FDB entry is the {port, MAC, VLAN} tuple forwarding destination.

To offloading L2 bridging, the switchdev driver/device should support:

	- Static FDB entries installed on a bridge port
	- Notification of learned/forgotten src mac/vlans from device
	- STP state changes on the port
	- VLAN flooding of multicast/broadcast and unknown unicast packets

Static FDB Entries
^^^^^^^^^^^^^^^^^^

A driver which implements the ``ndo_fdb_add``, ``ndo_fdb_del`` and
``ndo_fdb_dump`` operations is able to support the command below, which adds a
static bridge FDB entry::

        bridge fdb add dev DEV ADDRESS [vlan VID] [self] static

(the "static" keyword is non-optional: if not specified, the entry defaults to
being "local", which means that it should not be forwarded)

The "self" keyword (optional because it is implicit) has the role of
instructing the kernel to fulfill the operation through the ``ndo_fdb_add``
implementation of the ``DEV`` device itself. If ``DEV`` is a bridge port, this
will bypass the bridge and therefore leave the software database out of sync
with the hardware one.

To avoid this, the "master" keyword can be used::

        bridge fdb add dev DEV ADDRESS [vlan VID] master static

The above command instructs the kernel to search for a master interface of
``DEV`` and fulfill the operation through the ``ndo_fdb_add`` method of that.
This time, the bridge generates a ``SWITCHDEV_FDB_ADD_TO_DEVICE`` notification
which the port driver can handle and use it to program its hardware table. This
way, the software and the hardware database will both contain this static FDB
entry.

Note: for new switchdev drivers that offload the Linux bridge, implementing the
``ndo_fdb_add`` and ``ndo_fdb_del`` bridge bypass methods is strongly
discouraged: all static FDB entries should be added on a bridge port using the
"master" flag. The ``ndo_fdb_dump`` is an exception and can be implemented to
visualize the hardware tables, if the device does not have an interrupt for
notifying the operating system of newly learned/forgotten dynamic FDB
addresses. In that case, the hardware FDB might end up having entries that the
software FDB does not, and implementing ``ndo_fdb_dump`` is the only way to see
them.

Note: by default, the bridge does not filter on VLAN and only bridges untagged
traffic.  To enable VLAN support, turn on VLAN filtering::

	echo 1 >/sys/class/net/<bridge>/bridge/vlan_filtering

Notification of Learned/Forgotten Source MAC/VLANs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The switch device will learn/forget source MAC address/VLAN on ingress packets
and notify the switch driver of the mac/vlan/port tuples.  The switch driver,
in turn, will notify the bridge driver using the switchdev notifier call::

	err = call_switchdev_notifiers(val, dev, info, extack);

Where val is SWITCHDEV_FDB_ADD when learning and SWITCHDEV_FDB_DEL when
forgetting, and info points to a struct switchdev_notifier_fdb_info.  On
SWITCHDEV_FDB_ADD, the bridge driver will install the FDB entry into the
bridge's FDB and mark the entry as NTF_EXT_LEARNED.  The iproute2 bridge
command will label these entries "offload"::

	$ bridge fdb
	52:54:00:12:35:01 dev sw1p1 master br0 permanent
	00:02:00:00:02:00 dev sw1p1 master br0 offload
	00:02:00:00:02:00 dev sw1p1 self
	52:54:00:12:35:02 dev sw1p2 master br0 permanent
	00:02:00:00:03:00 dev sw1p2 master br0 offload
	00:02:00:00:03:00 dev sw1p2 self
	33:33:00:00:00:01 dev eth0 self permanent
	01:00:5e:00:00:01 dev eth0 self permanent
	33:33:ff:00:00:00 dev eth0 self permanent
	01:80:c2:00:00:0e dev eth0 self permanent
	33:33:00:00:00:01 dev br0 self permanent
	01:00:5e:00:00:01 dev br0 self permanent
	33:33:ff:12:35:01 dev br0 self permanent

Learning on the port should be disabled on the bridge using the bridge command::

	bridge link set dev DEV learning off

Learning on the device port should be enabled, as well as learning_sync::

	bridge link set dev DEV learning on self
	bridge link set dev DEV learning_sync on self

Learning_sync attribute enables syncing of the learned/forgotten FDB entry to
the bridge's FDB.  It's possible, but not optimal, to enable learning on the
device port and on the bridge port, and disable learning_sync.

To support learning, the driver implements switchdev op
switchdev_port_attr_set for SWITCHDEV_ATTR_PORT_ID_{PRE}_BRIDGE_FLAGS.

FDB Ageing
^^^^^^^^^^

The bridge will skip ageing FDB entries marked with NTF_EXT_LEARNED and it is
the responsibility of the port driver/device to age out these entries.  If the
port device supports ageing, when the FDB entry expires, it will notify the
driver which in turn will notify the bridge with SWITCHDEV_FDB_DEL.  If the
device does not support ageing, the driver can simulate ageing using a
garbage collection timer to monitor FDB entries.  Expired entries will be
notified to the bridge using SWITCHDEV_FDB_DEL.  See rocker driver for
example of driver running ageing timer.

To keep an NTF_EXT_LEARNED entry "alive", the driver should refresh the FDB
entry by calling call_switchdev_notifiers(SWITCHDEV_FDB_ADD, ...).  The
notification will reset the FDB entry's last-used time to now.  The driver
should rate limit refresh notifications, for example, no more than once a
second.  (The last-used time is visible using the bridge -s fdb option).

STP State Change on Port
^^^^^^^^^^^^^^^^^^^^^^^^

Internally or with a third-party STP protocol implementation (e.g. mstpd), the
bridge driver maintains the STP state for ports, and will notify the switch
driver of STP state change on a port using the switchdev op
switchdev_attr_port_set for SWITCHDEV_ATTR_PORT_ID_STP_UPDATE.

State is one of BR_STATE_*.  The switch driver can use STP state updates to
update ingress packet filter list for the port.  For example, if port is
DISABLED, no packets should pass, but if port moves to BLOCKED, then STP BPDUs
and other IEEE 01:80:c2:xx:xx:xx link-local multicast packets can pass.

Note that STP BDPUs are untagged and STP state applies to all VLANs on the port
so packet filters should be applied consistently across untagged and tagged
VLANs on the port.

Flooding L2 domain
^^^^^^^^^^^^^^^^^^

For a given L2 VLAN domain, the switch device should flood multicast/broadcast
and unknown unicast packets to all ports in domain, if allowed by port's
current STP state.  The switch driver, knowing which ports are within which
vlan L2 domain, can program the switch device for flooding.  The packet may
be sent to the port netdev for processing by the bridge driver.  The
bridge should not reflood the packet to the same ports the device flooded,
otherwise there will be duplicate packets on the wire.

To avoid duplicate packets, the switch driver should mark a packet as already
forwarded by setting the skb->offload_fwd_mark bit. The bridge driver will mark
the skb using the ingress bridge port's mark and prevent it from being forwarded
through any bridge port with the same mark.

It is possible for the switch device to not handle flooding and push the
packets up to the bridge driver for flooding.  This is not ideal as the number
of ports scale in the L2 domain as the device is much more efficient at
flooding packets that software.

If supported by the device, flood control can be offloaded to it, preventing
certain netdevs from flooding unicast traffic for which there is no FDB entry.

IGMP Snooping
^^^^^^^^^^^^^

In order to support IGMP snooping, the port netdevs should trap to the bridge
driver all IGMP join and leave messages.
The bridge multicast module will notify port netdevs on every multicast group
changed whether it is static configured or dynamically joined/leave.
The hardware implementation should be forwarding all registered multicast
traffic groups only to the configured ports.

L3 Routing Offload
------------------

Offloading L3 routing requires that device be programmed with FIB entries from
the kernel, with the device doing the FIB lookup and forwarding.  The device
does a longest prefix match (LPM) on FIB entries matching route prefix and
forwards the packet to the matching FIB entry's nexthop(s) egress ports.

To program the device, the driver has to register a FIB notifier handler
using register_fib_notifier. The following events are available:

===================  ===================================================
FIB_EVENT_ENTRY_ADD  used for both adding a new FIB entry to the device,
		     or modifying an existing entry on the device.
FIB_EVENT_ENTRY_DEL  used for removing a FIB entry
FIB_EVENT_RULE_ADD,
FIB_EVENT_RULE_DEL   used to propagate FIB rule changes
===================  ===================================================

FIB_EVENT_ENTRY_ADD and FIB_EVENT_ENTRY_DEL events pass::

	struct fib_entry_notifier_info {
		struct fib_notifier_info info; /* must be first */
		u32 dst;
		int dst_len;
		struct fib_info *fi;
		u8 tos;
		u8 type;
		u32 tb_id;
		u32 nlflags;
	};

to add/modify/delete IPv4 dst/dest_len prefix on table tb_id.  The ``*fi``
structure holds details on the route and route's nexthops.  ``*dev`` is one
of the port netdevs mentioned in the route's next hop list.

Routes offloaded to the device are labeled with "offload" in the ip route
listing::

	$ ip route show
	default via 192.168.0.2 dev eth0
	11.0.0.0/30 dev sw1p1  proto kernel  scope link  src 11.0.0.2 offload
	11.0.0.4/30 via 11.0.0.1 dev sw1p1  proto zebra  metric 20 offload
	11.0.0.8/30 dev sw1p2  proto kernel  scope link  src 11.0.0.10 offload
	11.0.0.12/30 via 11.0.0.9 dev sw1p2  proto zebra  metric 20 offload
	12.0.0.2  proto zebra  metric 30 offload
		nexthop via 11.0.0.1  dev sw1p1 weight 1
		nexthop via 11.0.0.9  dev sw1p2 weight 1
	12.0.0.3 via 11.0.0.1 dev sw1p1  proto zebra  metric 20 offload
	12.0.0.4 via 11.0.0.9 dev sw1p2  proto zebra  metric 20 offload
	192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.15

The "offload" flag is set in case at least one device offloads the FIB entry.

XXX: add/mod/del IPv6 FIB API

Nexthop Resolution
^^^^^^^^^^^^^^^^^^

The FIB entry's nexthop list contains the nexthop tuple (gateway, dev), but for
the switch device to forward the packet with the correct dst mac address, the
nexthop gateways must be resolved to the neighbor's mac address.  Neighbor mac
address discovery comes via the ARP (or ND) process and is available via the
arp_tbl neighbor table.  To resolve the routes nexthop gateways, the driver
should trigger the kernel's neighbor resolution process.  See the rocker
driver's rocker_port_ipv4_resolve() for an example.

The driver can monitor for updates to arp_tbl using the netevent notifier
NETEVENT_NEIGH_UPDATE.  The device can be programmed with resolved nexthops
for the routes as arp_tbl updates.  The driver implements ndo_neigh_destroy
to know when arp_tbl neighbor entries are purged from the port.

Device driver expected behavior
-------------------------------

Below is a set of defined behavior that switchdev enabled network devices must
adhere to.

Configuration-less state
^^^^^^^^^^^^^^^^^^^^^^^^

Upon driver bring up, the network devices must be fully operational, and the
backing driver must configure the network device such that it is possible to
send and receive traffic to this network device and it is properly separated
from other network devices/ports (e.g.: as is frequent with a switch ASIC). How
this is achieved is heavily hardware dependent, but a simple solution can be to
use per-port VLAN identifiers unless a better mechanism is available
(proprietary metadata for each network port for instance).

The network device must be capable of running a full IP protocol stack
including multicast, DHCP, IPv4/6, etc. If necessary, it should program the
appropriate filters for VLAN, multicast, unicast etc. The underlying device
driver must effectively be configured in a similar fashion to what it would do
when IGMP snooping is enabled for IP multicast over these switchdev network
devices and unsolicited multicast must be filtered as early as possible in
the hardware.

When configuring VLANs on top of the network device, all VLANs must be working,
irrespective of the state of other network devices (e.g.: other ports being part
of a VLAN-aware bridge doing ingress VID checking). See below for details.

If the device implements e.g.: VLAN filtering, putting the interface in
promiscuous mode should allow the reception of all VLAN tags (including those
not present in the filter(s)).

Bridged switch ports
^^^^^^^^^^^^^^^^^^^^

When a switchdev enabled network device is added as a bridge member, it should
not disrupt any functionality of non-bridged network devices and they
should continue to behave as normal network devices. Depending on the bridge
configuration knobs below, the expected behavior is documented.

Bridge VLAN filtering
^^^^^^^^^^^^^^^^^^^^^

The Linux bridge allows the configuration of a VLAN filtering mode (statically,
at device creation time, and dynamically, during run time) which must be
observed by the underlying switchdev network device/hardware:

- with VLAN filtering turned off: the bridge is strictly VLAN unaware and its
  data path will process all Ethernet frames as if they are VLAN-untagged.
  The bridge VLAN database can still be modified, but the modifications should
  have no effect while VLAN filtering is turned off. Frames ingressing the
  device with a VID that is not programmed into the bridge/switch's VLAN table
  must be forwarded and may be processed using a VLAN device (see below).

- with VLAN filtering turned on: the bridge is VLAN-aware and frames ingressing
  the device with a VID that is not programmed into the bridges/switch's VLAN
  table must be dropped (strict VID checking).

When there is a VLAN device (e.g: sw0p1.100) configured on top of a switchdev
network device which is a bridge port member, the behavior of the software
network stack must be preserved, or the configuration must be refused if that
is not possible.

- with VLAN filtering turned off, the bridge will process all ingress traffic
  for the port, except for the traffic tagged with a VLAN ID destined for a
  VLAN upper. The VLAN upper interface (which consumes the VLAN tag) can even
  be added to a second bridge, which includes other switch ports or software
  interfaces. Some approaches to ensure that the forwarding domain for traffic
  belonging to the VLAN upper interfaces are managed properly:

    * If forwarding destinations can be managed per VLAN, the hardware could be
      configured to map all traffic, except the packets tagged with a VID
      belonging to a VLAN upper interface, to an internal VID corresponding to
      untagged packets. This internal VID spans all ports of the VLAN-unaware
      bridge. The VID corresponding to the VLAN upper interface spans the
      physical port of that VLAN interface, as well as the other ports that
      might be bridged with it.
    * Treat bridge ports with VLAN upper interfaces as standalone, and let
      forwarding be handled in the software data path.

- with VLAN filtering turned on, these VLAN devices can be created as long as
  the bridge does not have an existing VLAN entry with the same VID on any
  bridge port. These VLAN devices cannot be enslaved into the bridge since they
  duplicate functionality/use case with the bridge's VLAN data path processing.

Non-bridged network ports of the same switch fabric must not be disturbed in any
way by the enabling of VLAN filtering on the bridge device(s). If the VLAN
filtering setting is global to the entire chip, then the standalone ports
should indicate to the network stack that VLAN filtering is required by setting
'rx-vlan-filter: on [fixed]' in the ethtool features.

Because VLAN filtering can be turned on/off at runtime, the switchdev driver
must be able to reconfigure the underlying hardware on the fly to honor the
toggling of that option and behave appropriately. If that is not possible, the
switchdev driver can also refuse to support dynamic toggling of the VLAN
filtering knob at runtime and require a destruction of the bridge device(s) and
creation of new bridge device(s) with a different VLAN filtering value to
ensure VLAN awareness is pushed down to the hardware.

Even when VLAN filtering in the bridge is turned off, the underlying switch
hardware and driver may still configure itself in a VLAN-aware mode provided
that the behavior described above is observed.

The VLAN protocol of the bridge plays a role in deciding whether a packet is
treated as tagged or not: a bridge using the 802.1ad protocol must treat both
VLAN-untagged packets, as well as packets tagged with 802.1Q headers, as
untagged.

The 802.1p (VID 0) tagged packets must be treated in the same way by the device
as untagged packets, since the bridge device does not allow the manipulation of
VID 0 in its database.

When the bridge has VLAN filtering enabled and a PVID is not configured on the
ingress port, untagged and 802.1p tagged packets must be dropped. When the bridge
has VLAN filtering enabled and a PVID exists on the ingress port, untagged and
priority-tagged packets must be accepted and forwarded according to the
bridge's port membership of the PVID VLAN. When the bridge has VLAN filtering
disabled, the presence/lack of a PVID should not influence the packet
forwarding decision.

Bridge IGMP snooping
^^^^^^^^^^^^^^^^^^^^

The Linux bridge allows the configuration of IGMP snooping (statically, at
interface creation time, or dynamically, during runtime) which must be observed
by the underlying switchdev network device/hardware in the following way:

- when IGMP snooping is turned off, multicast traffic must be flooded to all
  ports within the same bridge that have mcast_flood=true. The CPU/management
  port should ideally not be flooded (unless the ingress interface has
  IFF_ALLMULTI or IFF_PROMISC) and continue to learn multicast traffic through
  the network stack notifications. If the hardware is not capable of doing that
  then the CPU/management port must also be flooded and multicast filtering
  happens in software.

- when IGMP snooping is turned on, multicast traffic must selectively flow
  to the appropriate network ports (including CPU/management port). Flooding of
  unknown multicast should be only towards the ports connected to a multicast
  router (the local device may also act as a multicast router).

The switch must adhere to RFC 4541 and flood multicast traffic accordingly
since that is what the Linux bridge implementation does.

Because IGMP snooping can be turned on/off at runtime, the switchdev driver
must be able to reconfigure the underlying hardware on the fly to honor the
toggling of that option and behave appropriately.

A switchdev driver can also refuse to support dynamic toggling of the multicast
snooping knob at runtime and require the destruction of the bridge device(s)
and creation of a new bridge device(s) with a different multicast snooping
value.


code:
.. SPDX-License-Identifier: GPL-2.0

=============
Sysfs tagging
=============

(Taken almost verbatim from Eric Biederman's netns tagging patch
commit msg)

The problem.  Network devices show up in sysfs and with the network
namespace active multiple devices with the same name can show up in
the same directory, ouch!

To avoid that problem and allow existing applications in network
namespaces to see the same interface that is currently presented in
sysfs, sysfs now has tagging directory support.

By using the network namespace pointers as tags to separate out
the sysfs directory entries we ensure that we don't have conflicts
in the directories and applications only see a limited set of
the network devices.

Each sysfs directory entry may be tagged with a namespace via the
``void *ns member`` of its ``kernfs_node``.  If a directory entry is tagged,
then ``kernfs_node->flags`` will have a flag between KOBJ_NS_TYPE_NONE
and KOBJ_NS_TYPES, and ns will point to the namespace to which it
belongs.

Each sysfs superblock's kernfs_super_info contains an array
``void *ns[KOBJ_NS_TYPES]``.  When a task in a tagging namespace
kobj_nstype first mounts sysfs, a new superblock is created.  It
will be differentiated from other sysfs mounts by having its
``s_fs_info->ns[kobj_nstype]`` set to the new namespace.  Note that
through bind mounting and mounts propagation, a task can easily view
the contents of other namespaces' sysfs mounts.  Therefore, when a
namespace exits, it will call kobj_ns_exit() to invalidate any
kernfs_node->ns pointers pointing to it.

Users of this interface:

- define a type in the ``kobj_ns_type`` enumeration.
- call kobj_ns_type_register() with its ``kobj_ns_type_operations`` which has

  - current_ns() which returns current's namespace
  - netlink_ns() which returns a socket's namespace
  - initial_ns() which returns the initial namesapce

- call kobj_ns_exit() when an individual tag is no longer valid

repo_name: angery/linux


code:
.. SPDX-License-Identifier: GPL-2.0

================================
TC Actions - Environmental Rules
================================

The "environmental" rules for authors of any new tc actions are:

1) If you stealeth or borroweth any packet thou shalt be branching
   from the righteous path and thou shalt cloneth.

   For example if your action queues a packet to be processed later,
   or intentionally branches by redirecting a packet, then you need to
   clone the packet.

2) If you munge any packet thou shalt call pskb_expand_head in the case
   someone else is referencing the skb. After that you "own" the skb.

3) Dropping packets you don't own is a no-no. You simply return
   TC_ACT_SHOT to the caller and they will drop it.

The "environmental" rules for callers of actions (qdiscs etc) are:

#) Thou art responsible for freeing anything returned as being
   TC_ACT_SHOT/STOLEN/QUEUED. If none of TC_ACT_SHOT/STOLEN/QUEUED is
   returned, then all is great and you don't need to do anything.

Post on netdev if something is unclear.


code:
.. SPDX-License-Identifier: GPL-2.0

=========================
TC queue based filtering
=========================

TC can be used for directing traffic to either a set of queues or
to a single queue on both the transmit and receive side.

On the transmit side:

1) TC filter directing traffic to a set of queues is achieved
   using the action skbedit priority for Tx priority selection,
   the priority maps to a traffic class (set of queues) when
   the queue-sets are configured using mqprio.

2) TC filter directs traffic to a transmit queue with the action
   skbedit queue_mapping $tx_qid. The action skbedit queue_mapping
   for transmit queue is executed in software only and cannot be
   offloaded.

Likewise, on the receive side, the two filters for selecting set of
queues and/or a single queue are supported as below:

1) TC flower filter directs incoming traffic to a set of queues using
   the 'hw_tc' option.
   hw_tc $TCID - Specify a hardware traffic class to pass matching
   packets on to. TCID is in the range 0 through 15.

2) TC filter with action skbedit queue_mapping $rx_qid selects a
   receive queue. The action skbedit queue_mapping for receive queue
   is supported only in hardware. Multiple filters may compete in
   the hardware for queue selection. In such case, the hardware
   pipeline resolves conflicts based on priority. On Intel E810
   devices, TC filter directing traffic to a queue have higher
   priority over flow director filter assigning a queue. The hash
   filter has lowest priority.


code:
.. SPDX-License-Identifier: GPL-2.0

====================
Thin-streams and TCP
====================

A wide range of Internet-based services that use reliable transport
protocols display what we call thin-stream properties. This means
that the application sends data with such a low rate that the
retransmission mechanisms of the transport protocol are not fully
effective. In time-dependent scenarios (like online games, control
systems, stock trading etc.) where the user experience depends
on the data delivery latency, packet loss can be devastating for
the service quality. Extreme latencies are caused by TCP's
dependency on the arrival of new data from the application to trigger
retransmissions effectively through fast retransmit instead of
waiting for long timeouts.

After analysing a large number of time-dependent interactive
applications, we have seen that they often produce thin streams
and also stay with this traffic pattern throughout its entire
lifespan. The combination of time-dependency and the fact that the
streams provoke high latencies when using TCP is unfortunate.

In order to reduce application-layer latency when packets are lost,
a set of mechanisms has been made, which address these latency issues
for thin streams. In short, if the kernel detects a thin stream,
the retransmission mechanisms are modified in the following manner:

1) If the stream is thin, fast retransmit on the first dupACK.
2) If the stream is thin, do not apply exponential backoff.

These enhancements are applied only if the stream is detected as
thin. This is accomplished by defining a threshold for the number
of packets in flight. If there are less than 4 packets in flight,
fast retransmissions can not be triggered, and the stream is prone
to experience high retransmission latencies.

Since these mechanisms are targeted at time-dependent applications,
they must be specifically activated by the application using the
TCP_THIN_LINEAR_TIMEOUTS and TCP_THIN_DUPACK IOCTLS or the
tcp_thin_linear_timeouts and tcp_thin_dupack sysctls. Both
modifications are turned off by default.

References
==========
More information on the modifications, as well as a wide range of
experimental data can be found here:

"Improving latency for interactive, thin-stream applications over
reliable transport"
http://simula.no/research/nd/publications/Simula.nd.477/simula_pdf_file


code:
.. SPDX-License-Identifier: GPL-2.0

====
Team
====

Team devices are driven from userspace via libteam library which is here:
	https://github.com/jpirko/libteam


code:
.. SPDX-License-Identifier: GPL-2.0

============
Timestamping
============

1. Control Interfaces
=====================

The interfaces for receiving network packages timestamps are:

SO_TIMESTAMP
  Generates a timestamp for each incoming packet in (not necessarily
  monotonic) system time. Reports the timestamp via recvmsg() in a
  control message in usec resolution.
  SO_TIMESTAMP is defined as SO_TIMESTAMP_NEW or SO_TIMESTAMP_OLD
  based on the architecture type and time_t representation of libc.
  Control message format is in struct __kernel_old_timeval for
  SO_TIMESTAMP_OLD and in struct __kernel_sock_timeval for
  SO_TIMESTAMP_NEW options respectively.

SO_TIMESTAMPNS
  Same timestamping mechanism as SO_TIMESTAMP, but reports the
  timestamp as struct timespec in nsec resolution.
  SO_TIMESTAMPNS is defined as SO_TIMESTAMPNS_NEW or SO_TIMESTAMPNS_OLD
  based on the architecture type and time_t representation of libc.
  Control message format is in struct timespec for SO_TIMESTAMPNS_OLD
  and in struct __kernel_timespec for SO_TIMESTAMPNS_NEW options
  respectively.

IP_MULTICAST_LOOP + SO_TIMESTAMP[NS]
  Only for multicast:approximate transmit timestamp obtained by
  reading the looped packet receive timestamp.

SO_TIMESTAMPING
  Generates timestamps on reception, transmission or both. Supports
  multiple timestamp sources, including hardware. Supports generating
  timestamps for stream sockets.

1.1 SO_TIMESTAMP (also SO_TIMESTAMP_OLD and SO_TIMESTAMP_NEW)
-------------------------------------------------------------

This socket option enables timestamping of datagrams on the reception
path. Because the destination socket, if any, is not known early in
the network stack, the feature has to be enabled for all packets. The
same is true for all early receive timestamp options.

For interface details, see `man 7 socket`.

Always use SO_TIMESTAMP_NEW timestamp to always get timestamp in
struct __kernel_sock_timeval format.

SO_TIMESTAMP_OLD returns incorrect timestamps after the year 2038
on 32 bit machines.

1.2 SO_TIMESTAMPNS (also SO_TIMESTAMPNS_OLD and SO_TIMESTAMPNS_NEW)
-------------------------------------------------------------------

This option is identical to SO_TIMESTAMP except for the returned data type.
Its struct timespec allows for higher resolution (ns) timestamps than the
timeval of SO_TIMESTAMP (ms).

Always use SO_TIMESTAMPNS_NEW timestamp to always get timestamp in
struct __kernel_timespec format.

SO_TIMESTAMPNS_OLD returns incorrect timestamps after the year 2038
on 32 bit machines.

1.3 SO_TIMESTAMPING (also SO_TIMESTAMPING_OLD and SO_TIMESTAMPING_NEW)
----------------------------------------------------------------------

Supports multiple types of timestamp requests. As a result, this
socket option takes a bitmap of flags, not a boolean. In::

  err = setsockopt(fd, SOL_SOCKET, SO_TIMESTAMPING, &val, sizeof(val));

val is an integer with any of the following bits set. Setting other
bit returns EINVAL and does not change the current state.

The socket option configures timestamp generation for individual
sk_buffs (1.3.1), timestamp reporting to the socket's error
queue (1.3.2) and options (1.3.3). Timestamp generation can also
be enabled for individual sendmsg calls using cmsg (1.3.4).

1.3.1 Timestamp Generation
^^^^^^^^^^^^^^^^^^^^^^^^^^

Some bits are requests to the stack to try to generate timestamps. Any
combination of them is valid. Changes to these bits apply to newly
created packets, not to packets already in the stack. As a result, it
is possible to selectively request timestamps for a subset of packets
(e.g., for sampling) by embedding an send() call within two setsockopt
calls, one to enable timestamp generation and one to disable it.
Timestamps may also be generated for reasons other than being
requested by a particular socket, such as when receive timestamping is
enabled system wide, as explained earlier.

SOF_TIMESTAMPING_RX_HARDWARE:
  Request rx timestamps generated by the network adapter.

SOF_TIMESTAMPING_RX_SOFTWARE:
  Request rx timestamps when data enters the kernel. These timestamps
  are generated just after a device driver hands a packet to the
  kernel receive stack.

SOF_TIMESTAMPING_TX_HARDWARE:
  Request tx timestamps generated by the network adapter. This flag
  can be enabled via both socket options and control messages.

SOF_TIMESTAMPING_TX_SOFTWARE:
  Request tx timestamps when data leaves the kernel. These timestamps
  are generated in the device driver as close as possible, but always
  prior to, passing the packet to the network interface. Hence, they
  require driver support and may not be available for all devices.
  This flag can be enabled via both socket options and control messages.

SOF_TIMESTAMPING_TX_SCHED:
  Request tx timestamps prior to entering the packet scheduler. Kernel
  transmit latency is, if long, often dominated by queuing delay. The
  difference between this timestamp and one taken at
  SOF_TIMESTAMPING_TX_SOFTWARE will expose this latency independent
  of protocol processing. The latency incurred in protocol
  processing, if any, can be computed by subtracting a userspace
  timestamp taken immediately before send() from this timestamp. On
  machines with virtual devices where a transmitted packet travels
  through multiple devices and, hence, multiple packet schedulers,
  a timestamp is generated at each layer. This allows for fine
  grained measurement of queuing delay. This flag can be enabled
  via both socket options and control messages.

SOF_TIMESTAMPING_TX_ACK:
  Request tx timestamps when all data in the send buffer has been
  acknowledged. This only makes sense for reliable protocols. It is
  currently only implemented for TCP. For that protocol, it may
  over-report measurement, because the timestamp is generated when all
  data up to and including the buffer at send() was acknowledged: the
  cumulative acknowledgment. The mechanism ignores SACK and FACK.
  This flag can be enabled via both socket options and control messages.

1.3.2 Timestamp Reporting
^^^^^^^^^^^^^^^^^^^^^^^^^

The other three bits control which timestamps will be reported in a
generated control message. Changes to the bits take immediate
effect at the timestamp reporting locations in the stack. Timestamps
are only reported for packets that also have the relevant timestamp
generation request set.

SOF_TIMESTAMPING_SOFTWARE:
  Report any software timestamps when available.

SOF_TIMESTAMPING_SYS_HARDWARE:
  This option is deprecated and ignored.

SOF_TIMESTAMPING_RAW_HARDWARE:
  Report hardware timestamps as generated by
  SOF_TIMESTAMPING_TX_HARDWARE when available.

1.3.3 Timestamp Options
^^^^^^^^^^^^^^^^^^^^^^^

The interface supports the options

SOF_TIMESTAMPING_OPT_ID:
  Generate a unique identifier along with each packet. A process can
  have multiple concurrent timestamping requests outstanding. Packets
  can be reordered in the transmit path, for instance in the packet
  scheduler. In that case timestamps will be queued onto the error
  queue out of order from the original send() calls. It is not always
  possible to uniquely match timestamps to the original send() calls
  based on timestamp order or payload inspection alone, then.

  This option associates each packet at send() with a unique
  identifier and returns that along with the timestamp. The identifier
  is derived from a per-socket u32 counter (that wraps). For datagram
  sockets, the counter increments with each sent packet. For stream
  sockets, it increments with every byte. For stream sockets, also set
  SOF_TIMESTAMPING_OPT_ID_TCP, see the section below.

  The counter starts at zero. It is initialized the first time that
  the socket option is enabled. It is reset each time the option is
  enabled after having been disabled. Resetting the counter does not
  change the identifiers of existing packets in the system.

  This option is implemented only for transmit timestamps. There, the
  timestamp is always looped along with a struct sock_extended_err.
  The option modifies field ee_data to pass an id that is unique
  among all possibly concurrently outstanding timestamp requests for
  that socket.

SOF_TIMESTAMPING_OPT_ID_TCP:
  Pass this modifier along with SOF_TIMESTAMPING_OPT_ID for new TCP
  timestamping applications. SOF_TIMESTAMPING_OPT_ID defines how the
  counter increments for stream sockets, but its starting point is
  not entirely trivial. This option fixes that.

  For stream sockets, if SOF_TIMESTAMPING_OPT_ID is set, this should
  always be set too. On datagram sockets the option has no effect.

  A reasonable expectation is that the counter is reset to zero with
  the system call, so that a subsequent write() of N bytes generates
  a timestamp with counter N-1. SOF_TIMESTAMPING_OPT_ID_TCP
  implements this behavior under all conditions.

  SOF_TIMESTAMPING_OPT_ID without modifier often reports the same,
  especially when the socket option is set when no data is in
  transmission. If data is being transmitted, it may be off by the
  length of the output queue (SIOCOUTQ).

  The difference is due to being based on snd_una versus write_seq.
  snd_una is the offset in the stream acknowledged by the peer. This
  depends on factors outside of process control, such as network RTT.
  write_seq is the last byte written by the process. This offset is
  not affected by external inputs.

  The difference is subtle and unlikely to be noticed when configured
  at initial socket creation, when no data is queued or sent. But
  SOF_TIMESTAMPING_OPT_ID_TCP behavior is more robust regardless of
  when the socket option is set.

SOF_TIMESTAMPING_OPT_CMSG:
  Support recv() cmsg for all timestamped packets. Control messages
  are already supported unconditionally on all packets with receive
  timestamps and on IPv6 packets with transmit timestamp. This option
  extends them to IPv4 packets with transmit timestamp. One use case
  is to correlate packets with their egress device, by enabling socket
  option IP_PKTINFO simultaneously.

SOF_TIMESTAMPING_OPT_TSONLY:
  Applies to transmit timestamps only. Makes the kernel return the
  timestamp as a cmsg alongside an empty packet, as opposed to
  alongside the original packet. This reduces the amount of memory
  charged to the socket's receive budget (SO_RCVBUF) and delivers
  the timestamp even if sysctl net.core.tstamp_allow_data is 0.
  This option disables SOF_TIMESTAMPING_OPT_CMSG.

SOF_TIMESTAMPING_OPT_STATS:
  Optional stats that are obtained along with the transmit timestamps.
  It must be used together with SOF_TIMESTAMPING_OPT_TSONLY. When the
  transmit timestamp is available, the stats are available in a
  separate control message of type SCM_TIMESTAMPING_OPT_STATS, as a
  list of TLVs (struct nlattr) of types. These stats allow the
  application to associate various transport layer stats with
  the transmit timestamps, such as how long a certain block of
  data was limited by peer's receiver window.

SOF_TIMESTAMPING_OPT_PKTINFO:
  Enable the SCM_TIMESTAMPING_PKTINFO control message for incoming
  packets with hardware timestamps. The message contains struct
  scm_ts_pktinfo, which supplies the index of the real interface which
  received the packet and its length at layer 2. A valid (non-zero)
  interface index will be returned only if CONFIG_NET_RX_BUSY_POLL is
  enabled and the driver is using NAPI. The struct contains also two
  other fields, but they are reserved and undefined.

SOF_TIMESTAMPING_OPT_TX_SWHW:
  Request both hardware and software timestamps for outgoing packets
  when SOF_TIMESTAMPING_TX_HARDWARE and SOF_TIMESTAMPING_TX_SOFTWARE
  are enabled at the same time. If both timestamps are generated,
  two separate messages will be looped to the socket's error queue,
  each containing just one timestamp.

New applications are encouraged to pass SOF_TIMESTAMPING_OPT_ID to
disambiguate timestamps and SOF_TIMESTAMPING_OPT_TSONLY to operate
regardless of the setting of sysctl net.core.tstamp_allow_data.

An exception is when a process needs additional cmsg data, for
instance SOL_IP/IP_PKTINFO to detect the egress network interface.
Then pass option SOF_TIMESTAMPING_OPT_CMSG. This option depends on
having access to the contents of the original packet, so cannot be
combined with SOF_TIMESTAMPING_OPT_TSONLY.

1.3.4. Enabling timestamps via control messages
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In addition to socket options, timestamp generation can be requested
per write via cmsg, only for SOF_TIMESTAMPING_TX_* (see Section 1.3.1).
Using this feature, applications can sample timestamps per sendmsg()
without paying the overhead of enabling and disabling timestamps via
setsockopt::

  struct msghdr *msg;
  ...
  cmsg			       = CMSG_FIRSTHDR(msg);
  cmsg->cmsg_level	       = SOL_SOCKET;
  cmsg->cmsg_type	       = SO_TIMESTAMPING;
  cmsg->cmsg_len	       = CMSG_LEN(sizeof(__u32));
  *((__u32 *) CMSG_DATA(cmsg)) = SOF_TIMESTAMPING_TX_SCHED |
				 SOF_TIMESTAMPING_TX_SOFTWARE |
				 SOF_TIMESTAMPING_TX_ACK;
  err = sendmsg(fd, msg, 0);

The SOF_TIMESTAMPING_TX_* flags set via cmsg will override
the SOF_TIMESTAMPING_TX_* flags set via setsockopt.

Moreover, applications must still enable timestamp reporting via
setsockopt to receive timestamps::

  __u32 val = SOF_TIMESTAMPING_SOFTWARE |
	      SOF_TIMESTAMPING_OPT_ID /* or any other flag */;
  err = setsockopt(fd, SOL_SOCKET, SO_TIMESTAMPING, &val, sizeof(val));

1.4 Bytestream Timestamps
-------------------------

The SO_TIMESTAMPING interface supports timestamping of bytes in a
bytestream. Each request is interpreted as a request for when the
entire contents of the buffer has passed a timestamping point. That
is, for streams option SOF_TIMESTAMPING_TX_SOFTWARE will record
when all bytes have reached the device driver, regardless of how
many packets the data has been converted into.

In general, bytestreams have no natural delimiters and therefore
correlating a timestamp with data is non-trivial. A range of bytes
may be split across segments, any segments may be merged (possibly
coalescing sections of previously segmented buffers associated with
independent send() calls). Segments can be reordered and the same
byte range can coexist in multiple segments for protocols that
implement retransmissions.

It is essential that all timestamps implement the same semantics,
regardless of these possible transformations, as otherwise they are
incomparable. Handling "rare" corner cases differently from the
simple case (a 1:1 mapping from buffer to skb) is insufficient
because performance debugging often needs to focus on such outliers.

In practice, timestamps can be correlated with segments of a
bytestream consistently, if both semantics of the timestamp and the
timing of measurement are chosen correctly. This challenge is no
different from deciding on a strategy for IP fragmentation. There, the
definition is that only the first fragment is timestamped. For
bytestreams, we chose that a timestamp is generated only when all
bytes have passed a point. SOF_TIMESTAMPING_TX_ACK as defined is easy to
implement and reason about. An implementation that has to take into
account SACK would be more complex due to possible transmission holes
and out of order arrival.

On the host, TCP can also break the simple 1:1 mapping from buffer to
skbuff as a result of Nagle, cork, autocork, segmentation and GSO. The
implementation ensures correctness in all cases by tracking the
individual last byte passed to send(), even if it is no longer the
last byte after an skbuff extend or merge operation. It stores the
relevant sequence number in skb_shinfo(skb)->tskey. Because an skbuff
has only one such field, only one timestamp can be generated.

In rare cases, a timestamp request can be missed if two requests are
collapsed onto the same skb. A process can detect this situation by
enabling SOF_TIMESTAMPING_OPT_ID and comparing the byte offset at
send time with the value returned for each timestamp. It can prevent
the situation by always flushing the TCP stack in between requests,
for instance by enabling TCP_NODELAY and disabling TCP_CORK and
autocork.

These precautions ensure that the timestamp is generated only when all
bytes have passed a timestamp point, assuming that the network stack
itself does not reorder the segments. The stack indeed tries to avoid
reordering. The one exception is under administrator control: it is
possible to construct a packet scheduler configuration that delays
segments from the same stream differently. Such a setup would be
unusual.

2 Data Interfaces
==================

Timestamps are read using the ancillary data feature of recvmsg().
See `man 3 cmsg` for details of this interface. The socket manual
page (`man 7 socket`) describes how timestamps generated with
SO_TIMESTAMP and SO_TIMESTAMPNS records can be retrieved.

2.1 SCM_TIMESTAMPING records
----------------------------

These timestamps are returned in a control message with cmsg_level
SOL_SOCKET, cmsg_type SCM_TIMESTAMPING, and payload of type

For SO_TIMESTAMPING_OLD::

	struct scm_timestamping {
		struct timespec ts[3];
	};

For SO_TIMESTAMPING_NEW::

	struct scm_timestamping64 {
		struct __kernel_timespec ts[3];

Always use SO_TIMESTAMPING_NEW timestamp to always get timestamp in
struct scm_timestamping64 format.

SO_TIMESTAMPING_OLD returns incorrect timestamps after the year 2038
on 32 bit machines.

The structure can return up to three timestamps. This is a legacy
feature. At least one field is non-zero at any time. Most timestamps
are passed in ts[0]. Hardware timestamps are passed in ts[2].

ts[1] used to hold hardware timestamps converted to system time.
Instead, expose the hardware clock device on the NIC directly as
a HW PTP clock source, to allow time conversion in userspace and
optionally synchronize system time with a userspace PTP stack such
as linuxptp. For the PTP clock API, see Documentation/driver-api/ptp.rst.

Note that if the SO_TIMESTAMP or SO_TIMESTAMPNS option is enabled
together with SO_TIMESTAMPING using SOF_TIMESTAMPING_SOFTWARE, a false
software timestamp will be generated in the recvmsg() call and passed
in ts[0] when a real software timestamp is missing. This happens also
on hardware transmit timestamps.

2.1.1 Transmit timestamps with MSG_ERRQUEUE
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For transmit timestamps the outgoing packet is looped back to the
socket's error queue with the send timestamp(s) attached. A process
receives the timestamps by calling recvmsg() with flag MSG_ERRQUEUE
set and with a msg_control buffer sufficiently large to receive the
relevant metadata structures. The recvmsg call returns the original
outgoing data packet with two ancillary messages attached.

A message of cm_level SOL_IP(V6) and cm_type IP(V6)_RECVERR
embeds a struct sock_extended_err. This defines the error type. For
timestamps, the ee_errno field is ENOMSG. The other ancillary message
will have cm_level SOL_SOCKET and cm_type SCM_TIMESTAMPING. This
embeds the struct scm_timestamping.

2.1.1.2 Timestamp types
~~~~~~~~~~~~~~~~~~~~~~~

The semantics of the three struct timespec are defined by field
ee_info in the extended error structure. It contains a value of
type SCM_TSTAMP_* to define the actual timestamp passed in
scm_timestamping.

The SCM_TSTAMP_* types are 1:1 matches to the SOF_TIMESTAMPING_*
control fields discussed previously, with one exception. For legacy
reasons, SCM_TSTAMP_SND is equal to zero and can be set for both
SOF_TIMESTAMPING_TX_HARDWARE and SOF_TIMESTAMPING_TX_SOFTWARE. It
is the first if ts[2] is non-zero, the second otherwise, in which
case the timestamp is stored in ts[0].

2.1.1.3 Fragmentation
~~~~~~~~~~~~~~~~~~~~~

Fragmentation of outgoing datagrams is rare, but is possible, e.g., by
explicitly disabling PMTU discovery. If an outgoing packet is fragmented,
then only the first fragment is timestamped and returned to the sending
socket.

2.1.1.4 Packet Payload
~~~~~~~~~~~~~~~~~~~~~~

The calling application is often not interested in receiving the whole
packet payload that it passed to the stack originally: the socket
error queue mechanism is just a method to piggyback the timestamp on.
In this case, the application can choose to read datagrams with a
smaller buffer, possibly even of length 0. The payload is truncated
accordingly. Until the process calls recvmsg() on the error queue,
however, the full packet is queued, taking up budget from SO_RCVBUF.

2.1.1.5 Blocking Read
~~~~~~~~~~~~~~~~~~~~~

Reading from the error queue is always a non-blocking operation. To
block waiting on a timestamp, use poll or select. poll() will return
POLLERR in pollfd.revents if any data is ready on the error queue.
There is no need to pass this flag in pollfd.events. This flag is
ignored on request. See also `man 2 poll`.

2.1.2 Receive timestamps
^^^^^^^^^^^^^^^^^^^^^^^^

On reception, there is no reason to read from the socket error queue.
The SCM_TIMESTAMPING ancillary data is sent along with the packet data
on a normal recvmsg(). Since this is not a socket error, it is not
accompanied by a message SOL_IP(V6)/IP(V6)_RECVERROR. In this case,
the meaning of the three fields in struct scm_timestamping is
implicitly defined. ts[0] holds a software timestamp if set, ts[1]
is again deprecated and ts[2] holds a hardware timestamp if set.

3. Hardware Timestamping configuration: SIOCSHWTSTAMP and SIOCGHWTSTAMP
=======================================================================

Hardware time stamping must also be initialized for each device driver
that is expected to do hardware time stamping. The parameter is defined in
include/uapi/linux/net_tstamp.h as::

	struct hwtstamp_config {
		int flags;	/* no flags defined right now, must be zero */
		int tx_type;	/* HWTSTAMP_TX_* */
		int rx_filter;	/* HWTSTAMP_FILTER_* */
	};

Desired behavior is passed into the kernel and to a specific device by
calling ioctl(SIOCSHWTSTAMP) with a pointer to a struct ifreq whose
ifr_data points to a struct hwtstamp_config. The tx_type and
rx_filter are hints to the driver what it is expected to do. If
the requested fine-grained filtering for incoming packets is not
supported, the driver may time stamp more than just the requested types
of packets.

Drivers are free to use a more permissive configuration than the requested
configuration. It is expected that drivers should only implement directly the
most generic mode that can be supported. For example if the hardware can
support HWTSTAMP_FILTER_PTP_V2_EVENT, then it should generally always upscale
HWTSTAMP_FILTER_PTP_V2_L2_SYNC, and so forth, as HWTSTAMP_FILTER_PTP_V2_EVENT
is more generic (and more useful to applications).

A driver which supports hardware time stamping shall update the struct
with the actual, possibly more permissive configuration. If the
requested packets cannot be time stamped, then nothing should be
changed and ERANGE shall be returned (in contrast to EINVAL, which
indicates that SIOCSHWTSTAMP is not supported at all).

Only a processes with admin rights may change the configuration. User
space is responsible to ensure that multiple processes don't interfere
with each other and that the settings are reset.

Any process can read the actual configuration by passing this
structure to ioctl(SIOCGHWTSTAMP) in the same way.  However, this has
not been implemented in all drivers.

::

    /* possible values for hwtstamp_config->tx_type */
    enum {
	    /*
	    * no outgoing packet will need hardware time stamping;
	    * should a packet arrive which asks for it, no hardware
	    * time stamping will be done
	    */
	    HWTSTAMP_TX_OFF,

	    /*
	    * enables hardware time stamping for outgoing packets;
	    * the sender of the packet decides which are to be
	    * time stamped by setting SOF_TIMESTAMPING_TX_SOFTWARE
	    * before sending the packet
	    */
	    HWTSTAMP_TX_ON,
    };

    /* possible values for hwtstamp_config->rx_filter */
    enum {
	    /* time stamp no incoming packet at all */
	    HWTSTAMP_FILTER_NONE,

	    /* time stamp any incoming packet */
	    HWTSTAMP_FILTER_ALL,

	    /* return value: time stamp all packets requested plus some others */
	    HWTSTAMP_FILTER_SOME,

	    /* PTP v1, UDP, any kind of event packet */
	    HWTSTAMP_FILTER_PTP_V1_L4_EVENT,

	    /* for the complete list of values, please check
	    * the include file include/uapi/linux/net_tstamp.h
	    */
    };

3.1 Hardware Timestamping Implementation: Device Drivers
--------------------------------------------------------

A driver which supports hardware time stamping must support the
SIOCSHWTSTAMP ioctl and update the supplied struct hwtstamp_config with
the actual values as described in the section on SIOCSHWTSTAMP.  It
should also support SIOCGHWTSTAMP.

Time stamps for received packets must be stored in the skb. To get a pointer
to the shared time stamp structure of the skb call skb_hwtstamps(). Then
set the time stamps in the structure::

    struct skb_shared_hwtstamps {
	    /* hardware time stamp transformed into duration
	    * since arbitrary point in time
	    */
	    ktime_t	hwtstamp;
    };

Time stamps for outgoing packets are to be generated as follows:

- In hard_start_xmit(), check if (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)
  is set no-zero. If yes, then the driver is expected to do hardware time
  stamping.
- If this is possible for the skb and requested, then declare
  that the driver is doing the time stamping by setting the flag
  SKBTX_IN_PROGRESS in skb_shinfo(skb)->tx_flags , e.g. with::

      skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;

  You might want to keep a pointer to the associated skb for the next step
  and not free the skb. A driver not supporting hardware time stamping doesn't
  do that. A driver must never touch sk_buff::tstamp! It is used to store
  software generated time stamps by the network subsystem.
- Driver should call skb_tx_timestamp() as close to passing sk_buff to hardware
  as possible. skb_tx_timestamp() provides a software time stamp if requested
  and hardware timestamping is not possible (SKBTX_IN_PROGRESS not set).
- As soon as the driver has sent the packet and/or obtained a
  hardware time stamp for it, it passes the time stamp back by
  calling skb_tstamp_tx() with the original skb, the raw
  hardware time stamp. skb_tstamp_tx() clones the original skb and
  adds the timestamps, therefore the original skb has to be freed now.
  If obtaining the hardware time stamp somehow fails, then the driver
  should not fall back to software time stamping. The rationale is that
  this would occur at a later time in the processing pipeline than other
  software time stamping and therefore could lead to unexpected deltas
  between time stamps.

3.2 Special considerations for stacked PTP Hardware Clocks
----------------------------------------------------------

There are situations when there may be more than one PHC (PTP Hardware Clock)
in the data path of a packet. The kernel has no explicit mechanism to allow the
user to select which PHC to use for timestamping Ethernet frames. Instead, the
assumption is that the outermost PHC is always the most preferable, and that
kernel drivers collaborate towards achieving that goal. Currently there are 3
cases of stacked PHCs, detailed below:

3.2.1 DSA (Distributed Switch Architecture) switches
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

These are Ethernet switches which have one of their ports connected to an
(otherwise completely unaware) host Ethernet interface, and perform the role of
a port multiplier with optional forwarding acceleration features.  Each DSA
switch port is visible to the user as a standalone (virtual) network interface,
and its network I/O is performed, under the hood, indirectly through the host
interface (redirecting to the host port on TX, and intercepting frames on RX).

When a DSA switch is attached to a host port, PTP synchronization has to
suffer, since the switch's variable queuing delay introduces a path delay
jitter between the host port and its PTP partner. For this reason, some DSA
switches include a timestamping clock of their own, and have the ability to
perform network timestamping on their own MAC, such that path delays only
measure wire and PHY propagation latencies. Timestamping DSA switches are
supported in Linux and expose the same ABI as any other network interface (save
for the fact that the DSA interfaces are in fact virtual in terms of network
I/O, they do have their own PHC).  It is typical, but not mandatory, for all
interfaces of a DSA switch to share the same PHC.

By design, PTP timestamping with a DSA switch does not need any special
handling in the driver for the host port it is attached to.  However, when the
host port also supports PTP timestamping, DSA will take care of intercepting
the ``.ndo_eth_ioctl`` calls towards the host port, and block attempts to enable
hardware timestamping on it. This is because the SO_TIMESTAMPING API does not
allow the delivery of multiple hardware timestamps for the same packet, so
anybody else except for the DSA switch port must be prevented from doing so.

In the generic layer, DSA provides the following infrastructure for PTP
timestamping:

- ``.port_txtstamp()``: a hook called prior to the transmission of
  packets with a hardware TX timestamping request from user space.
  This is required for two-step timestamping, since the hardware
  timestamp becomes available after the actual MAC transmission, so the
  driver must be prepared to correlate the timestamp with the original
  packet so that it can re-enqueue the packet back into the socket's
  error queue. To save the packet for when the timestamp becomes
  available, the driver can call ``skb_clone_sk`` , save the clone pointer
  in skb->cb and enqueue a tx skb queue. Typically, a switch will have a
  PTP TX timestamp register (or sometimes a FIFO) where the timestamp
  becomes available. In case of a FIFO, the hardware might store
  key-value pairs of PTP sequence ID/message type/domain number and the
  actual timestamp. To perform the correlation correctly between the
  packets in a queue waiting for timestamping and the actual timestamps,
  drivers can use a BPF classifier (``ptp_classify_raw``) to identify
  the PTP transport type, and ``ptp_parse_header`` to interpret the PTP
  header fields. There may be an IRQ that is raised upon this
  timestamp's availability, or the driver might have to poll after
  invoking ``dev_queue_xmit()`` towards the host interface.
  One-step TX timestamping do not require packet cloning, since there is
  no follow-up message required by the PTP protocol (because the
  TX timestamp is embedded into the packet by the MAC), and therefore
  user space does not expect the packet annotated with the TX timestamp
  to be re-enqueued into its socket's error queue.

- ``.port_rxtstamp()``: On RX, the BPF classifier is run by DSA to
  identify PTP event messages (any other packets, including PTP general
  messages, are not timestamped). The original (and only) timestampable
  skb is provided to the driver, for it to annotate it with a timestamp,
  if that is immediately available, or defer to later. On reception,
  timestamps might either be available in-band (through metadata in the
  DSA header, or attached in other ways to the packet), or out-of-band
  (through another RX timestamping FIFO). Deferral on RX is typically
  necessary when retrieving the timestamp needs a sleepable context. In
  that case, it is the responsibility of the DSA driver to call
  ``netif_rx()`` on the freshly timestamped skb.

3.2.2 Ethernet PHYs
^^^^^^^^^^^^^^^^^^^

These are devices that typically fulfill a Layer 1 role in the network stack,
hence they do not have a representation in terms of a network interface as DSA
switches do. However, PHYs may be able to detect and timestamp PTP packets, for
performance reasons: timestamps taken as close as possible to the wire have the
potential to yield a more stable and precise synchronization.

A PHY driver that supports PTP timestamping must create a ``struct
mii_timestamper`` and add a pointer to it in ``phydev->mii_ts``. The presence
of this pointer will be checked by the networking stack.

Since PHYs do not have network interface representations, the timestamping and
ethtool ioctl operations for them need to be mediated by their respective MAC
driver.  Therefore, as opposed to DSA switches, modifications need to be done
to each individual MAC driver for PHY timestamping support. This entails:

- Checking, in ``.ndo_eth_ioctl``, whether ``phy_has_hwtstamp(netdev->phydev)``
  is true or not. If it is, then the MAC driver should not process this request
  but instead pass it on to the PHY using ``phy_mii_ioctl()``.

- On RX, special intervention may or may not be needed, depending on the
  function used to deliver skb's up the network stack. In the case of plain
  ``netif_rx()`` and similar, MAC drivers must check whether
  ``skb_defer_rx_timestamp(skb)`` is necessary or not - and if it is, don't
  call ``netif_rx()`` at all.  If ``CONFIG_NETWORK_PHY_TIMESTAMPING`` is
  enabled, and ``skb->dev->phydev->mii_ts`` exists, its ``.rxtstamp()`` hook
  will be called now, to determine, using logic very similar to DSA, whether
  deferral for RX timestamping is necessary.  Again like DSA, it becomes the
  responsibility of the PHY driver to send the packet up the stack when the
  timestamp is available.

  For other skb receive functions, such as ``napi_gro_receive`` and
  ``netif_receive_skb``, the stack automatically checks whether
  ``skb_defer_rx_timestamp()`` is necessary, so this check is not needed inside
  the driver.

- On TX, again, special intervention might or might not be needed.  The
  function that calls the ``mii_ts->txtstamp()`` hook is named
  ``skb_clone_tx_timestamp()``. This function can either be called directly
  (case in which explicit MAC driver support is indeed needed), but the
  function also piggybacks from the ``skb_tx_timestamp()`` call, which many MAC
  drivers already perform for software timestamping purposes. Therefore, if a
  MAC supports software timestamping, it does not need to do anything further
  at this stage.

3.2.3 MII bus snooping devices
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

These perform the same role as timestamping Ethernet PHYs, save for the fact
that they are discrete devices and can therefore be used in conjunction with
any PHY even if it doesn't support timestamping. In Linux, they are
discoverable and attachable to a ``struct phy_device`` through Device Tree, and
for the rest, they use the same mii_ts infrastructure as those. See
Documentation/devicetree/bindings/ptp/timestamper.txt for more details.

3.2.4 Other caveats for MAC drivers
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Stacked PHCs, especially DSA (but not only) - since that doesn't require any
modification to MAC drivers, so it is more difficult to ensure correctness of
all possible code paths - is that they uncover bugs which were impossible to
trigger before the existence of stacked PTP clocks.  One example has to do with
this line of code, already presented earlier::

      skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;

Any TX timestamping logic, be it a plain MAC driver, a DSA switch driver, a PHY
driver or a MII bus snooping device driver, should set this flag.
But a MAC driver that is unaware of PHC stacking might get tripped up by
somebody other than itself setting this flag, and deliver a duplicate
timestamp.
For example, a typical driver design for TX timestamping might be to split the
transmission part into 2 portions:

1. "TX": checks whether PTP timestamping has been previously enabled through
   the ``.ndo_eth_ioctl`` ("``priv->hwtstamp_tx_enabled == true``") and the
   current skb requires a TX timestamp ("``skb_shinfo(skb)->tx_flags &
   SKBTX_HW_TSTAMP``"). If this is true, it sets the
   "``skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS``" flag. Note: as
   described above, in the case of a stacked PHC system, this condition should
   never trigger, as this MAC is certainly not the outermost PHC. But this is
   not where the typical issue is.  Transmission proceeds with this packet.

2. "TX confirmation": Transmission has finished. The driver checks whether it
   is necessary to collect any TX timestamp for it. Here is where the typical
   issues are: the MAC driver takes a shortcut and only checks whether
   "``skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS``" was set. With a stacked
   PHC system, this is incorrect because this MAC driver is not the only entity
   in the TX data path who could have enabled SKBTX_IN_PROGRESS in the first
   place.

The correct solution for this problem is for MAC drivers to have a compound
check in their "TX confirmation" portion, not only for
"``skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS``", but also for
"``priv->hwtstamp_tx_enabled == true``". Because the rest of the system ensures
that PTP timestamping is not enabled for anything other than the outermost PHC,
this enhanced check will avoid delivering a duplicated TX timestamp to user
space.


code:
.. SPDX-License-Identifier: GPL-2.0

=================
Linux Kernel TIPC
=================

Introduction
============

TIPC (Transparent Inter Process Communication) is a protocol that is specially
designed for intra-cluster communication. It can be configured to transmit
messages either on UDP or directly across Ethernet. Message delivery is
sequence guaranteed, loss free and flow controlled. Latency times are shorter
than with any other known protocol, while maximal throughput is comparable to
that of TCP.

TIPC Features
-------------

- Cluster wide IPC service

  Have you ever wished you had the convenience of Unix Domain Sockets even when
  transmitting data between cluster nodes? Where you yourself determine the
  addresses you want to bind to and use? Where you don't have to perform DNS
  lookups and worry about IP addresses? Where you don't have to start timers
  to monitor the continuous existence of peer sockets? And yet without the
  downsides of that socket type, such as the risk of lingering inodes?

  Welcome to the Transparent Inter Process Communication service, TIPC in short,
  which gives you all of this, and a lot more.

- Service Addressing

  A fundamental concept in TIPC is that of Service Addressing which makes it
  possible for a programmer to chose his own address, bind it to a server
  socket and let client programs use only that address for sending messages.

- Service Tracking

  A client wanting to wait for the availability of a server, uses the Service
  Tracking mechanism to subscribe for binding and unbinding/close events for
  sockets with the associated service address.

  The service tracking mechanism can also be used for Cluster Topology Tracking,
  i.e., subscribing for availability/non-availability of cluster nodes.

  Likewise, the service tracking mechanism can be used for Cluster Connectivity
  Tracking, i.e., subscribing for up/down events for individual links between
  cluster nodes.

- Transmission Modes

  Using a service address, a client can send datagram messages to a server socket.

  Using the same address type, it can establish a connection towards an accepting
  server socket.

  It can also use a service address to create and join a Communication Group,
  which is the TIPC manifestation of a brokerless message bus.

  Multicast with very good performance and scalability is available both in
  datagram mode and in communication group mode.

- Inter Node Links

  Communication between any two nodes in a cluster is maintained by one or two
  Inter Node Links, which both guarantee data traffic integrity and monitor
  the peer node's availability.

- Cluster Scalability

  By applying the Overlapping Ring Monitoring algorithm on the inter node links
  it is possible to scale TIPC clusters up to 1000 nodes with a maintained
  neighbor failure discovery time of 1-2 seconds. For smaller clusters this
  time can be made much shorter.

- Neighbor Discovery

  Neighbor Node Discovery in the cluster is done by Ethernet broadcast or UDP
  multicast, when any of those services are available. If not, configured peer
  IP addresses can be used.

- Configuration

  When running TIPC in single node mode no configuration whatsoever is needed.
  When running in cluster mode TIPC must as a minimum be given a node address
  (before Linux 4.17) and told which interface to attach to. The "tipc"
  configuration tool makes is possible to add and maintain many more
  configuration parameters.

- Performance

  TIPC message transfer latency times are better than in any other known protocol.
  Maximal byte throughput for inter-node connections is still somewhat lower than
  for TCP, while they are superior for intra-node and inter-container throughput
  on the same host.

- Language Support

  The TIPC user API has support for C, Python, Perl, Ruby, D and Go.

More Information
----------------

- How to set up TIPC:

  http://tipc.io/getting_started.html

- How to program with TIPC:

  http://tipc.io/programming.html

- How to contribute to TIPC:

- http://tipc.io/contacts.html

- More details about TIPC specification:

  http://tipc.io/protocol.html

Implementation
==============

TIPC is implemented as a kernel module in net/tipc/ directory.

TIPC Base Types
---------------

.. kernel-doc:: net/tipc/subscr.h
   :internal:

.. kernel-doc:: net/tipc/bearer.h
   :internal:

.. kernel-doc:: net/tipc/name_table.h
   :internal:

.. kernel-doc:: net/tipc/name_distr.h
   :internal:

.. kernel-doc:: net/tipc/bcast.c
   :internal:

TIPC Bearer Interfaces
----------------------

.. kernel-doc:: net/tipc/bearer.c
   :internal:

.. kernel-doc:: net/tipc/udp_media.c
   :internal:

TIPC Crypto Interfaces
----------------------

.. kernel-doc:: net/tipc/crypto.c
   :internal:

TIPC Discoverer Interfaces
--------------------------

.. kernel-doc:: net/tipc/discover.c
   :internal:

TIPC Link Interfaces
--------------------

.. kernel-doc:: net/tipc/link.c
   :internal:

TIPC msg Interfaces
-------------------

.. kernel-doc:: net/tipc/msg.c
   :internal:

TIPC Name Interfaces
--------------------

.. kernel-doc:: net/tipc/name_table.c
   :internal:

.. kernel-doc:: net/tipc/name_distr.c
   :internal:

TIPC Node Management Interfaces
-------------------------------

.. kernel-doc:: net/tipc/node.c
   :internal:

TIPC Socket Interfaces
----------------------

.. kernel-doc:: net/tipc/socket.c
   :internal:

TIPC Network Topology Interfaces
--------------------------------

.. kernel-doc:: net/tipc/subscr.c
   :internal:

TIPC Server Interfaces
----------------------

.. kernel-doc:: net/tipc/topsrv.c
   :internal:

TIPC Trace Interfaces
---------------------

.. kernel-doc:: net/tipc/trace.c
   :internal:


code:
.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)

==================
Kernel TLS offload
==================

Kernel TLS operation
====================

Linux kernel provides TLS connection offload infrastructure. Once a TCP
connection is in ``ESTABLISHED`` state user space can enable the TLS Upper
Layer Protocol (ULP) and install the cryptographic connection state.
For details regarding the user-facing interface refer to the TLS
documentation in :ref:`Documentation/networking/tls.rst <kernel_tls>`.

``ktls`` can operate in three modes:

 * Software crypto mode (``TLS_SW``) - CPU handles the cryptography.
   In most basic cases only crypto operations synchronous with the CPU
   can be used, but depending on calling context CPU may utilize
   asynchronous crypto accelerators. The use of accelerators introduces extra
   latency on socket reads (decryption only starts when a read syscall
   is made) and additional I/O load on the system.
 * Packet-based NIC offload mode (``TLS_HW``) - the NIC handles crypto
   on a packet by packet basis, provided the packets arrive in order.
   This mode integrates best with the kernel stack and is described in detail
   in the remaining part of this document
   (``ethtool`` flags ``tls-hw-tx-offload`` and ``tls-hw-rx-offload``).
 * Full TCP NIC offload mode (``TLS_HW_RECORD``) - mode of operation where
   NIC driver and firmware replace the kernel networking stack
   with its own TCP handling, it is not usable in production environments
   making use of the Linux networking stack for example any firewalling
   abilities or QoS and packet scheduling (``ethtool`` flag ``tls-hw-record``).

The operation mode is selected automatically based on device configuration,
offload opt-in or opt-out on per-connection basis is not currently supported.

TX
--

At a high level user write requests are turned into a scatter list, the TLS ULP
intercepts them, inserts record framing, performs encryption (in ``TLS_SW``
mode) and then hands the modified scatter list to the TCP layer. From this
point on the TCP stack proceeds as normal.

In ``TLS_HW`` mode the encryption is not performed in the TLS ULP.
Instead packets reach a device driver, the driver will mark the packets
for crypto offload based on the socket the packet is attached to,
and send them to the device for encryption and transmission.

RX
--

On the receive side if the device handled decryption and authentication
successfully, the driver will set the decrypted bit in the associated
:c:type:`struct sk_buff <sk_buff>`. The packets reach the TCP stack and
are handled normally. ``ktls`` is informed when data is queued to the socket
and the ``strparser`` mechanism is used to delineate the records. Upon read
request, records are retrieved from the socket and passed to decryption routine.
If device decrypted all the segments of the record the decryption is skipped,
otherwise software path handles decryption.

.. kernel-figure::  tls-offload-layers.svg
   :alt:	TLS offload layers
   :align:	center
   :figwidth:	28em

   Layers of Kernel TLS stack

Device configuration
====================

During driver initialization device sets the ``NETIF_F_HW_TLS_RX`` and
``NETIF_F_HW_TLS_TX`` features and installs its
:c:type:`struct tlsdev_ops <tlsdev_ops>`
pointer in the :c:member:`tlsdev_ops` member of the
:c:type:`struct net_device <net_device>`.

When TLS cryptographic connection state is installed on a ``ktls`` socket
(note that it is done twice, once for RX and once for TX direction,
and the two are completely independent), the kernel checks if the underlying
network device is offload-capable and attempts the offload. In case offload
fails the connection is handled entirely in software using the same mechanism
as if the offload was never tried.

Offload request is performed via the :c:member:`tls_dev_add` callback of
:c:type:`struct tlsdev_ops <tlsdev_ops>`:

.. code-block:: c

	int (*tls_dev_add)(struct net_device *netdev, struct sock *sk,
			   enum tls_offload_ctx_dir direction,
			   struct tls_crypto_info *crypto_info,
			   u32 start_offload_tcp_sn);

``direction`` indicates whether the cryptographic information is for
the received or transmitted packets. Driver uses the ``sk`` parameter
to retrieve the connection 5-tuple and socket family (IPv4 vs IPv6).
Cryptographic information in ``crypto_info`` includes the key, iv, salt
as well as TLS record sequence number. ``start_offload_tcp_sn`` indicates
which TCP sequence number corresponds to the beginning of the record with
sequence number from ``crypto_info``. The driver can add its state
at the end of kernel structures (see :c:member:`driver_state` members
in ``include/net/tls.h``) to avoid additional allocations and pointer
dereferences.

TX
--

After TX state is installed, the stack guarantees that the first segment
of the stream will start exactly at the ``start_offload_tcp_sn`` sequence
number, simplifying TCP sequence number matching.

TX offload being fully initialized does not imply that all segments passing
through the driver and which belong to the offloaded socket will be after
the expected sequence number and will have kernel record information.
In particular, already encrypted data may have been queued to the socket
before installing the connection state in the kernel.

RX
--

In RX direction local networking stack has little control over the segmentation,
so the initial records' TCP sequence number may be anywhere inside the segment.

Normal operation
================

At the minimum the device maintains the following state for each connection, in
each direction:

 * crypto secrets (key, iv, salt)
 * crypto processing state (partial blocks, partial authentication tag, etc.)
 * record metadata (sequence number, processing offset and length)
 * expected TCP sequence number

There are no guarantees on record length or record segmentation. In particular
segments may start at any point of a record and contain any number of records.
Assuming segments are received in order, the device should be able to perform
crypto operations and authentication regardless of segmentation. For this
to be possible device has to keep small amount of segment-to-segment state.
This includes at least:

 * partial headers (if a segment carried only a part of the TLS header)
 * partial data block
 * partial authentication tag (all data had been seen but part of the
   authentication tag has to be written or read from the subsequent segment)

Record reassembly is not necessary for TLS offload. If the packets arrive
in order the device should be able to handle them separately and make
forward progress.

TX
--

The kernel stack performs record framing reserving space for the authentication
tag and populating all other TLS header and tailer fields.

Both the device and the driver maintain expected TCP sequence numbers
due to the possibility of retransmissions and the lack of software fallback
once the packet reaches the device.
For segments passed in order, the driver marks the packets with
a connection identifier (note that a 5-tuple lookup is insufficient to identify
packets requiring HW offload, see the :ref:`5tuple_problems` section)
and hands them to the device. The device identifies the packet as requiring
TLS handling and confirms the sequence number matches its expectation.
The device performs encryption and authentication of the record data.
It replaces the authentication tag and TCP checksum with correct values.

RX
--

Before a packet is DMAed to the host (but after NIC's embedded switching
and packet transformation functions) the device validates the Layer 4
checksum and performs a 5-tuple lookup to find any TLS connection the packet
may belong to (technically a 4-tuple
lookup is sufficient - IP addresses and TCP port numbers, as the protocol
is always TCP). If connection is matched device confirms if the TCP sequence
number is the expected one and proceeds to TLS handling (record delineation,
decryption, authentication for each record in the packet). The device leaves
the record framing unmodified, the stack takes care of record decapsulation.
Device indicates successful handling of TLS offload in the per-packet context
(descriptor) passed to the host.

Upon reception of a TLS offloaded packet, the driver sets
the :c:member:`decrypted` mark in :c:type:`struct sk_buff <sk_buff>`
corresponding to the segment. Networking stack makes sure decrypted
and non-decrypted segments do not get coalesced (e.g. by GRO or socket layer)
and takes care of partial decryption.

Resync handling
===============

In presence of packet drops or network packet reordering, the device may lose
synchronization with the TLS stream, and require a resync with the kernel's
TCP stack.

Note that resync is only attempted for connections which were successfully
added to the device table and are in TLS_HW mode. For example,
if the table was full when cryptographic state was installed in the kernel,
such connection will never get offloaded. Therefore the resync request
does not carry any cryptographic connection state.

TX
--

Segments transmitted from an offloaded socket can get out of sync
in similar ways to the receive side-retransmissions - local drops
are possible, though network reorders are not. There are currently
two mechanisms for dealing with out of order segments.

Crypto state rebuilding
~~~~~~~~~~~~~~~~~~~~~~~

Whenever an out of order segment is transmitted the driver provides
the device with enough information to perform cryptographic operations.
This means most likely that the part of the record preceding the current
segment has to be passed to the device as part of the packet context,
together with its TCP sequence number and TLS record number. The device
can then initialize its crypto state, process and discard the preceding
data (to be able to insert the authentication tag) and move onto handling
the actual packet.

In this mode depending on the implementation the driver can either ask
for a continuation with the crypto state and the new sequence number
(next expected segment is the one after the out of order one), or continue
with the previous stream state - assuming that the out of order segment
was just a retransmission. The former is simpler, and does not require
retransmission detection therefore it is the recommended method until
such time it is proven inefficient.

Next record sync
~~~~~~~~~~~~~~~~

Whenever an out of order segment is detected the driver requests
that the ``ktls`` software fallback code encrypt it. If the segment's
sequence number is lower than expected the driver assumes retransmission
and doesn't change device state. If the segment is in the future, it
may imply a local drop, the driver asks the stack to sync the device
to the next record state and falls back to software.

Resync request is indicated with:

.. code-block:: c

  void tls_offload_tx_resync_request(struct sock *sk, u32 got_seq, u32 exp_seq)

Until resync is complete driver should not access its expected TCP
sequence number (as it will be updated from a different context).
Following helper should be used to test if resync is complete:

.. code-block:: c

  bool tls_offload_tx_resync_pending(struct sock *sk)

Next time ``ktls`` pushes a record it will first send its TCP sequence number
and TLS record number to the driver. Stack will also make sure that
the new record will start on a segment boundary (like it does when
the connection is initially added).

RX
--

A small amount of RX reorder events may not require a full resynchronization.
In particular the device should not lose synchronization
when record boundary can be recovered:

.. kernel-figure::  tls-offload-reorder-good.svg
   :alt:	reorder of non-header segment
   :align:	center

   Reorder of non-header segment

Green segments are successfully decrypted, blue ones are passed
as received on wire, red stripes mark start of new records.

In above case segment 1 is received and decrypted successfully.
Segment 2 was dropped so 3 arrives out of order. The device knows
the next record starts inside 3, based on record length in segment 1.
Segment 3 is passed untouched, because due to lack of data from segment 2
the remainder of the previous record inside segment 3 cannot be handled.
The device can, however, collect the authentication algorithm's state
and partial block from the new record in segment 3 and when 4 and 5
arrive continue decryption. Finally when 2 arrives it's completely outside
of expected window of the device so it's passed as is without special
handling. ``ktls`` software fallback handles the decryption of record
spanning segments 1, 2 and 3. The device did not get out of sync,
even though two segments did not get decrypted.

Kernel synchronization may be necessary if the lost segment contained
a record header and arrived after the next record header has already passed:

.. kernel-figure::  tls-offload-reorder-bad.svg
   :alt:	reorder of header segment
   :align:	center

   Reorder of segment with a TLS header

In this example segment 2 gets dropped, and it contains a record header.
Device can only detect that segment 4 also contains a TLS header
if it knows the length of the previous record from segment 2. In this case
the device will lose synchronization with the stream.

Stream scan resynchronization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When the device gets out of sync and the stream reaches TCP sequence
numbers more than a max size record past the expected TCP sequence number,
the device starts scanning for a known header pattern. For example
for TLS 1.2 and TLS 1.3 subsequent bytes of value ``0x03 0x03`` occur
in the SSL/TLS version field of the header. Once pattern is matched
the device continues attempting parsing headers at expected locations
(based on the length fields at guessed locations).
Whenever the expected location does not contain a valid header the scan
is restarted.

When the header is matched the device sends a confirmation request
to the kernel, asking if the guessed location is correct (if a TLS record
really starts there), and which record sequence number the given header had.
The kernel confirms the guessed location was correct and tells the device
the record sequence number. Meanwhile, the device had been parsing
and counting all records since the just-confirmed one, it adds the number
of records it had seen to the record number provided by the kernel.
At this point the device is in sync and can resume decryption at next
segment boundary.

In a pathological case the device may latch onto a sequence of matching
headers and never hear back from the kernel (there is no negative
confirmation from the kernel). The implementation may choose to periodically
restart scan. Given how unlikely falsely-matching stream is, however,
periodic restart is not deemed necessary.

Special care has to be taken if the confirmation request is passed
asynchronously to the packet stream and record may get processed
by the kernel before the confirmation request.

Stack-driven resynchronization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The driver may also request the stack to perform resynchronization
whenever it sees the records are no longer getting decrypted.
If the connection is configured in this mode the stack automatically
schedules resynchronization after it has received two completely encrypted
records.

The stack waits for the socket to drain and informs the device about
the next expected record number and its TCP sequence number. If the
records continue to be received fully encrypted stack retries the
synchronization with an exponential back off (first after 2 encrypted
records, then after 4 records, after 8, after 16... up until every
128 records).

Error handling
==============

TX
--

Packets may be redirected or rerouted by the stack to a different
device than the selected TLS offload device. The stack will handle
such condition using the :c:func:`sk_validate_xmit_skb` helper
(TLS offload code installs :c:func:`tls_validate_xmit_skb` at this hook).
Offload maintains information about all records until the data is
fully acknowledged, so if skbs reach the wrong device they can be handled
by software fallback.

Any device TLS offload handling error on the transmission side must result
in the packet being dropped. For example if a packet got out of order
due to a bug in the stack or the device, reached the device and can't
be encrypted such packet must be dropped.

RX
--

If the device encounters any problems with TLS offload on the receive
side it should pass the packet to the host's networking stack as it was
received on the wire.

For example authentication failure for any record in the segment should
result in passing the unmodified packet to the software fallback. This means
packets should not be modified "in place". Splitting segments to handle partial
decryption is not advised. In other words either all records in the packet
had been handled successfully and authenticated or the packet has to be passed
to the host's stack as it was on the wire (recovering original packet in the
driver if device provides precise error is sufficient).

The Linux networking stack does not provide a way of reporting per-packet
decryption and authentication errors, packets with errors must simply not
have the :c:member:`decrypted` mark set.

A packet should also not be handled by the TLS offload if it contains
incorrect checksums.

Performance metrics
===================

TLS offload can be characterized by the following basic metrics:

 * max connection count
 * connection installation rate
 * connection installation latency
 * total cryptographic performance

Note that each TCP connection requires a TLS session in both directions,
the performance may be reported treating each direction separately.

Max connection count
--------------------

The number of connections device can support can be exposed via
``devlink resource`` API.

Total cryptographic performance
-------------------------------

Offload performance may depend on segment and record size.

Overload of the cryptographic subsystem of the device should not have
significant performance impact on non-offloaded streams.

Statistics
==========

Following minimum set of TLS-related statistics should be reported
by the driver:

 * ``rx_tls_decrypted_packets`` - number of successfully decrypted RX packets
   which were part of a TLS stream.
 * ``rx_tls_decrypted_bytes`` - number of TLS payload bytes in RX packets
   which were successfully decrypted.
 * ``rx_tls_ctx`` - number of TLS RX HW offload contexts added to device for
   decryption.
 * ``rx_tls_del`` - number of TLS RX HW offload contexts deleted from device
   (connection has finished).
 * ``rx_tls_resync_req_pkt`` - number of received TLS packets with a resync
    request.
 * ``rx_tls_resync_req_start`` - number of times the TLS async resync request
    was started.
 * ``rx_tls_resync_req_end`` - number of times the TLS async resync request
    properly ended with providing the HW tracked tcp-seq.
 * ``rx_tls_resync_req_skip`` - number of times the TLS async resync request
    procedure was started by not properly ended.
 * ``rx_tls_resync_res_ok`` - number of times the TLS resync response call to
    the driver was successfully handled.
 * ``rx_tls_resync_res_skip`` - number of times the TLS resync response call to
    the driver was terminated unsuccessfully.
 * ``rx_tls_err`` - number of RX packets which were part of a TLS stream
   but were not decrypted due to unexpected error in the state machine.
 * ``tx_tls_encrypted_packets`` - number of TX packets passed to the device
   for encryption of their TLS payload.
 * ``tx_tls_encrypted_bytes`` - number of TLS payload bytes in TX packets
   passed to the device for encryption.
 * ``tx_tls_ctx`` - number of TLS TX HW offload contexts added to device for
   encryption.
 * ``tx_tls_ooo`` - number of TX packets which were part of a TLS stream
   but did not arrive in the expected order.
 * ``tx_tls_skip_no_sync_data`` - number of TX packets which were part of
   a TLS stream and arrived out-of-order, but skipped the HW offload routine
   and went to the regular transmit flow as they were retransmissions of the
   connection handshake.
 * ``tx_tls_drop_no_sync_data`` - number of TX packets which were part of
   a TLS stream dropped, because they arrived out of order and associated
   record could not be found.
 * ``tx_tls_drop_bypass_req`` - number of TX packets which were part of a TLS
   stream dropped, because they contain both data that has been encrypted by
   software and data that expects hardware crypto offload.

Notable corner cases, exceptions and additional requirements
============================================================

.. _5tuple_problems:

5-tuple matching limitations
----------------------------

The device can only recognize received packets based on the 5-tuple
of the socket. Current ``ktls`` implementation will not offload sockets
routed through software interfaces such as those used for tunneling
or virtual networking. However, many packet transformations performed
by the networking stack (most notably any BPF logic) do not require
any intermediate software device, therefore a 5-tuple match may
consistently miss at the device level. In such cases the device
should still be able to perform TX offload (encryption) and should
fallback cleanly to software decryption (RX).

Out of order
------------

Introducing extra processing in NICs should not cause packets to be
transmitted or received out of order, for example pure ACK packets
should not be reordered with respect to data segments.

Ingress reorder
---------------

A device is permitted to perform packet reordering for consecutive
TCP segments (i.e. placing packets in the correct order) but any form
of additional buffering is disallowed.

Coexistence with standard networking offload features
-----------------------------------------------------

Offloaded ``ktls`` sockets should support standard TCP stack features
transparently. Enabling device TLS offload should not cause any difference
in packets as seen on the wire.

Transport layer transparency
----------------------------

The device should not modify any packet headers for the purpose
of the simplifying TLS offload.

The device should not depend on any packet headers beyond what is strictly
necessary for TLS offload.

Segment drops
-------------

Dropping packets is acceptable only in the event of catastrophic
system errors and should never be used as an error handling mechanism
in cases arising from normal operation. In other words, reliance
on TCP retransmissions to handle corner cases is not acceptable.

TLS device features
-------------------

Drivers should ignore the changes to the TLS device feature flags.
These flags will be acted upon accordingly by the core ``ktls`` code.
TLS device feature flags only control adding of new TLS connection
offloads, old connections will remain active after flags are cleared.

TLS encryption cannot be offloaded to devices without checksum calculation
offload. Hence, TLS TX device feature flag requires TX csum offload being set.
Disabling the latter implies clearing the former. Disabling TX checksum offload
should not affect old connections, and drivers should make sure checksum
calculation does not break for them.
Similarly, device-offloaded TLS decryption implies doing RXCSUM. If the user
does not want to enable RX csum offload, TLS RX device feature is disabled
as well.