code:
.. SPDX-License-Identifier: GPL-2.0

==================================
IP dynamic address hack-port v0.03
==================================

This stuff allows diald ONESHOT connections to get established by
dynamically changing packet source address (and socket's if local procs).
It is implemented for TCP diald-box connections(1) and IP_MASQuerading(2).

If enabled\ [#]_ and forwarding interface has changed:

  1)  Socket (and packet) source address is rewritten ON RETRANSMISSIONS
      while in SYN_SENT state (diald-box processes).
  2)  Out-bounded MASQueraded source address changes ON OUTPUT (when
      internal host does retransmission) until a packet from outside is
      received by the tunnel.

This is specially helpful for auto dialup links (diald), where the
``actual`` outgoing address is unknown at the moment the link is
going up. So, the *same* (local AND masqueraded) connections requests that
bring the link up will be able to get established.

.. [#] At boot, by default no address rewriting is attempted.

  To enable::

     # echo 1 > /proc/sys/net/ipv4/ip_dynaddr

  To enable verbose mode::

    # echo 2 > /proc/sys/net/ipv4/ip_dynaddr

  To disable (default)::

     # echo 0 > /proc/sys/net/ipv4/ip_dynaddr

Enjoy!

Juanjo  <jjciarla@raiz.uncu.edu.ar>

repo_name: angery/linux


code:
.. SPDX-License-Identifier: GPL-2.0

=========================================================
AppleTalk-IP Decapsulation and AppleTalk-IP Encapsulation
=========================================================

Documentation ipddp.c

This file is written by Jay Schulist <jschlst@samba.org>

Introduction
------------

AppleTalk-IP (IPDDP) is the method computers connected to AppleTalk
networks can use to communicate via IP. AppleTalk-IP is simply IP datagrams
inside AppleTalk packets.

Through this driver you can either allow your Linux box to communicate
IP over an AppleTalk network or you can provide IP gatewaying functions
for your AppleTalk users.

You can currently encapsulate or decapsulate AppleTalk-IP on LocalTalk,
EtherTalk and PPPTalk. The only limit on the protocol is that of what
kernel AppleTalk layer and drivers are available.

Each mode requires its own user space software.

Compiling AppleTalk-IP Decapsulation/Encapsulation
==================================================

AppleTalk-IP decapsulation needs to be compiled into your kernel. You
will need to turn on AppleTalk-IP driver support. Then you will need to
select ONE of the two options; IP to AppleTalk-IP encapsulation support or
AppleTalk-IP to IP decapsulation support. If you compile the driver
statically you will only be able to use the driver for the function you have
enabled in the kernel. If you compile the driver as a module you can
select what mode you want it to run in via a module loading param.
ipddp_mode=1 for AppleTalk-IP encapsulation and ipddp_mode=2 for
AppleTalk-IP to IP decapsulation.

Basic instructions for user space tools
=======================================

I will briefly describe the operation of the tools, but you will
need to consult the supporting documentation for each set of tools.

Decapsulation - You will need to download a software package called
MacGate. In this distribution there will be a tool called MacRoute
which enables you to add routes to the kernel for your Macs by hand.
Also the tool MacRegGateWay is included to register the
proper IP Gateway and IP addresses for your machine. Included in this
distribution is a patch to netatalk-1.4b2+asun2.0a17.2 (available from
ftp.u.washington.edu/pub/user-supported/asun/) this patch is optional
but it allows automatic adding and deleting of routes for Macs. (Handy
for locations with large Mac installations)

Encapsulation - You will need to download a software daemon called ipddpd.
This software expects there to be an AppleTalk-IP gateway on the network.
You will also need to add the proper routes to route your Linux box's IP
traffic out the ipddp interface.

Common Uses of ipddp.c
----------------------
Of course AppleTalk-IP decapsulation and encapsulation, but specifically
decapsulation is being used most for connecting LocalTalk networks to
IP networks. Although it has been used on EtherTalk networks to allow
Macs that are only able to tunnel IP over EtherTalk.

Encapsulation has been used to allow a Linux box stuck on a LocalTalk
network to use IP. It should work equally well if you are stuck on an
EtherTalk only network.

Further Assistance
-------------------
You can contact me (Jay Schulist <jschlst@samba.org>) with any
questions regarding decapsulation or encapsulation. Bradford W. Johnson
<johns393@maroon.tc.umn.edu> originally wrote the ipddp.c driver for IP
encapsulation in AppleTalk.


code:
.. SPDX-License-Identifier: GPL-2.0

=====
IPsec
=====

Here documents known IPsec corner cases which need to be keep in mind when
deploy various IPsec configuration in real world production environment.

1. IPcomp:
	   Small IP packet won't get compressed at sender, and failed on
	   policy check on receiver.

Quote from RFC3173::

  2.2. Non-Expansion Policy

   If the total size of a compressed payload and the IPComp header, as
   defined in section 3, is not smaller than the size of the original
   payload, the IP datagram MUST be sent in the original non-compressed
   form.  To clarify: If an IP datagram is sent non-compressed, no

   IPComp header is added to the datagram.  This policy ensures saving
   the decompression processing cycles and avoiding incurring IP
   datagram fragmentation when the expanded datagram is larger than the
   MTU.

   Small IP datagrams are likely to expand as a result of compression.
   Therefore, a numeric threshold should be applied before compression,
   where IP datagrams of size smaller than the threshold are sent in the
   original form without attempting compression.  The numeric threshold
   is implementation dependent.

Current IPComp implementation is indeed by the book, while as in practice
when sending non-compressed packet to the peer (whether or not packet len
is smaller than the threshold or the compressed len is larger than original
packet len), the packet is dropped when checking the policy as this packet
matches the selector but not coming from any XFRM layer, i.e., with no
security path. Such naked packet will not eventually make it to upper layer.
The result is much more wired to the user when ping peer with different
payload length.

One workaround is try to set "level use" for each policy if user observed
above scenario. The consequence of doing so is small packet(uncompressed)
will skip policy checking on receiver side.


code:
.. SPDX-License-Identifier: GPL-2.0

====
IPv6
====

Options for the ipv6 module are supplied as parameters at load time.

Module options may be given as command line arguments to the insmod
or modprobe command, but are usually specified in either
``/etc/modules.d/*.conf`` configuration files, or in a distro-specific
configuration file.

The available ipv6 module parameters are listed below.  If a parameter
is not specified the default value is used.

The parameters are as follows:

disable

	Specifies whether to load the IPv6 module, but disable all
	its functionality.  This might be used when another module
	has a dependency on the IPv6 module being loaded, but no
	IPv6 addresses or operations are desired.

	The possible values and their effects are:

	0
		IPv6 is enabled.

		This is the default value.

	1
		IPv6 is disabled.

		No IPv6 addresses will be added to interfaces, and
		it will not be possible to open an IPv6 socket.

		A reboot is required to enable IPv6.

autoconf

	Specifies whether to enable IPv6 address autoconfiguration
	on all interfaces.  This might be used when one does not wish
	for addresses to be automatically generated from prefixes
	received in Router Advertisements.

	The possible values and their effects are:

	0
		IPv6 address autoconfiguration is disabled on all interfaces.

		Only the IPv6 loopback address (::1) and link-local addresses
		will be added to interfaces.

	1
		IPv6 address autoconfiguration is enabled on all interfaces.

		This is the default value.

disable_ipv6

	Specifies whether to disable IPv6 on all interfaces.
	This might be used when no IPv6 addresses are desired.

	The possible values and their effects are:

	0
		IPv6 is enabled on all interfaces.

		This is the default value.

	1
		IPv6 is disabled on all interfaces.

		No IPv6 addresses will be added to interfaces.


code:
.. SPDX-License-Identifier: GPL-2.0

===================
IPVLAN Driver HOWTO
===================

Initial Release:
	Mahesh Bandewar <maheshb AT google.com>

1. Introduction:
================
This is conceptually very similar to the macvlan driver with one major
exception of using L3 for mux-ing /demux-ing among slaves. This property makes
the master device share the L2 with its slave devices. I have developed this
driver in conjunction with network namespaces and not sure if there is use case
outside of it.

2. Building and Installation:
=============================

In order to build the driver, please select the config item CONFIG_IPVLAN.
The driver can be built into the kernel (CONFIG_IPVLAN=y) or as a module
(CONFIG_IPVLAN=m).

3. Configuration:
=================

There are no module parameters for this driver and it can be configured
using IProute2/ip utility.
::

    ip link add link <master> name <slave> type ipvlan [ mode MODE ] [ FLAGS ]
       where
	 MODE: l3 (default) | l3s | l2
	 FLAGS: bridge (default) | private | vepa

e.g.

    (a) Following will create IPvlan link with eth0 as master in
	L3 bridge mode::

	  bash# ip link add link eth0 name ipvl0 type ipvlan
    (b) This command will create IPvlan link in L2 bridge mode::

	  bash# ip link add link eth0 name ipvl0 type ipvlan mode l2 bridge

    (c) This command will create an IPvlan device in L2 private mode::

	  bash# ip link add link eth0 name ipvlan type ipvlan mode l2 private

    (d) This command will create an IPvlan device in L2 vepa mode::

	  bash# ip link add link eth0 name ipvlan type ipvlan mode l2 vepa

4. Operating modes:
===================

IPvlan has two modes of operation - L2 and L3. For a given master device,
you can select one of these two modes and all slaves on that master will
operate in the same (selected) mode. The RX mode is almost identical except
that in L3 mode the slaves wont receive any multicast / broadcast traffic.
L3 mode is more restrictive since routing is controlled from the other (mostly)
default namespace.

4.1 L2 mode:
------------

In this mode TX processing happens on the stack instance attached to the
slave device and packets are switched and queued to the master device to send
out. In this mode the slaves will RX/TX multicast and broadcast (if applicable)
as well.

4.2 L3 mode:
------------

In this mode TX processing up to L3 happens on the stack instance attached
to the slave device and packets are switched to the stack instance of the
master device for the L2 processing and routing from that instance will be
used before packets are queued on the outbound device. In this mode the slaves
will not receive nor can send multicast / broadcast traffic.

4.3 L3S mode:
-------------

This is very similar to the L3 mode except that iptables (conn-tracking)
works in this mode and hence it is L3-symmetric (L3s). This will have slightly less
performance but that shouldn't matter since you are choosing this mode over plain-L3
mode to make conn-tracking work.

5. Mode flags:
==============

At this time following mode flags are available

5.1 bridge:
-----------
This is the default option. To configure the IPvlan port in this mode,
user can choose to either add this option on the command-line or don't specify
anything. This is the traditional mode where slaves can cross-talk among
themselves apart from talking through the master device.

5.2 private:
------------
If this option is added to the command-line, the port is set in private
mode. i.e. port won't allow cross communication between slaves.

5.3 vepa:
---------
If this is added to the command-line, the port is set in VEPA mode.
i.e. port will offload switching functionality to the external entity as
described in 802.1Qbg
Note: VEPA mode in IPvlan has limitations. IPvlan uses the mac-address of the
master-device, so the packets which are emitted in this mode for the adjacent
neighbor will have source and destination mac same. This will make the switch /
router send the redirect message.

6. What to choose (macvlan vs. ipvlan)?
=======================================

These two devices are very similar in many regards and the specific use
case could very well define which device to choose. if one of the following
situations defines your use case then you can choose to use ipvlan:

(a) The Linux host that is connected to the external switch / router has
    policy configured that allows only one mac per port.
(b) No of virtual devices created on a master exceed the mac capacity and
    puts the NIC in promiscuous mode and degraded performance is a concern.
(c) If the slave device is to be put into the hostile / untrusted network
    namespace where L2 on the slave could be changed / misused.

6. Example configuration:
=========================

::

  +=============================================================+
  |  Host: host1                                                |
  |                                                             |
  |   +----------------------+      +----------------------+    |
  |   |   NS:ns0             |      |  NS:ns1              |    |
  |   |                      |      |                      |    |
  |   |                      |      |                      |    |
  |   |        ipvl0         |      |         ipvl1        |    |
  |   +----------#-----------+      +-----------#----------+    |
  |              #                              #               |
  |              ################################               |
  |                              # eth0                         |
  +==============================#==============================+

(a) Create two network namespaces - ns0, ns1::

	ip netns add ns0
	ip netns add ns1

(b) Create two ipvlan slaves on eth0 (master device)::

	ip link add link eth0 ipvl0 type ipvlan mode l2
	ip link add link eth0 ipvl1 type ipvlan mode l2

(c) Assign slaves to the respective network namespaces::

	ip link set dev ipvl0 netns ns0
	ip link set dev ipvl1 netns ns1

(d) Now switch to the namespace (ns0 or ns1) to configure the slave devices

	- For ns0::

		(1) ip netns exec ns0 bash
		(2) ip link set dev ipvl0 up
		(3) ip link set dev lo up
		(4) ip -4 addr add 127.0.0.1 dev lo
		(5) ip -4 addr add $IPADDR dev ipvl0
		(6) ip -4 route add default via $ROUTER dev ipvl0

	- For ns1::

		(1) ip netns exec ns1 bash
		(2) ip link set dev ipvl1 up
		(3) ip link set dev lo up
		(4) ip -4 addr add 127.0.0.1 dev lo
		(5) ip -4 addr add $IPADDR dev ipvl1
		(6) ip -4 route add default via $ROUTER dev ipvl1


code:
.. SPDX-License-Identifier: GPL-2.0

===========
IPvs-sysctl
===========

/proc/sys/net/ipv4/vs/* Variables:
==================================

am_droprate - INTEGER
	default 10

	It sets the always mode drop rate, which is used in the mode 3
	of the drop_rate defense.

amemthresh - INTEGER
	default 1024

	It sets the available memory threshold (in pages), which is
	used in the automatic modes of defense. When there is no
	enough available memory, the respective strategy will be
	enabled and the variable is automatically set to 2, otherwise
	the strategy is disabled and the variable is  set  to 1.

backup_only - BOOLEAN
	- 0 - disabled (default)
	- not 0 - enabled

	If set, disable the director function while the server is
	in backup mode to avoid packet loops for DR/TUN methods.

conn_reuse_mode - INTEGER
	1 - default

	Controls how ipvs will deal with connections that are detected
	port reuse. It is a bitmap, with the values being:

	0: disable any special handling on port reuse. The new
	connection will be delivered to the same real server that was
	servicing the previous connection.

	bit 1: enable rescheduling of new connections when it is safe.
	That is, whenever expire_nodest_conn and for TCP sockets, when
	the connection is in TIME_WAIT state (which is only possible if
	you use NAT mode).

	bit 2: it is bit 1 plus, for TCP connections, when connections
	are in FIN_WAIT state, as this is the last state seen by load
	balancer in Direct Routing mode. This bit helps on adding new
	real servers to a very busy cluster.

conntrack - BOOLEAN
	- 0 - disabled (default)
	- not 0 - enabled

	If set, maintain connection tracking entries for
	connections handled by IPVS.

	This should be enabled if connections handled by IPVS are to be
	also handled by stateful firewall rules. That is, iptables rules
	that make use of connection tracking.  It is a performance
	optimisation to disable this setting otherwise.

	Connections handled by the IPVS FTP application module
	will have connection tracking entries regardless of this setting.

	Only available when IPVS is compiled with CONFIG_IP_VS_NFCT enabled.

cache_bypass - BOOLEAN
	- 0 - disabled (default)
	- not 0 - enabled

	If it is enabled, forward packets to the original destination
	directly when no cache server is available and destination
	address is not local (iph->daddr is RTN_UNICAST). It is mostly
	used in transparent web cache cluster.

debug_level - INTEGER
	- 0          - transmission error messages (default)
	- 1          - non-fatal error messages
	- 2          - configuration
	- 3          - destination trash
	- 4          - drop entry
	- 5          - service lookup
	- 6          - scheduling
	- 7          - connection new/expire, lookup and synchronization
	- 8          - state transition
	- 9          - binding destination, template checks and applications
	- 10         - IPVS packet transmission
	- 11         - IPVS packet handling (ip_vs_in/ip_vs_out)
	- 12 or more - packet traversal

	Only available when IPVS is compiled with CONFIG_IP_VS_DEBUG enabled.

	Higher debugging levels include the messages for lower debugging
	levels, so setting debug level 2, includes level 0, 1 and 2
	messages. Thus, logging becomes more and more verbose the higher
	the level.

drop_entry - INTEGER
	- 0  - disabled (default)

	The drop_entry defense is to randomly drop entries in the
	connection hash table, just in order to collect back some
	memory for new connections. In the current code, the
	drop_entry procedure can be activated every second, then it
	randomly scans 1/32 of the whole and drops entries that are in
	the SYN-RECV/SYNACK state, which should be effective against
	syn-flooding attack.

	The valid values of drop_entry are from 0 to 3, where 0 means
	that this strategy is always disabled, 1 and 2 mean automatic
	modes (when there is no enough available memory, the strategy
	is enabled and the variable is automatically set to 2,
	otherwise the strategy is disabled and the variable is set to
	1), and 3 means that the strategy is always enabled.

drop_packet - INTEGER
	- 0  - disabled (default)

	The drop_packet defense is designed to drop 1/rate packets
	before forwarding them to real servers. If the rate is 1, then
	drop all the incoming packets.

	The value definition is the same as that of the drop_entry. In
	the automatic mode, the rate is determined by the follow
	formula: rate = amemthresh / (amemthresh - available_memory)
	when available memory is less than the available memory
	threshold. When the mode 3 is set, the always mode drop rate
	is controlled by the /proc/sys/net/ipv4/vs/am_droprate.

est_cpulist - CPULIST
	Allowed	CPUs for estimation kthreads

	Syntax: standard cpulist format
	empty list - stop kthread tasks and estimation
	default - the system's housekeeping CPUs for kthreads

	Example:
	"all": all possible CPUs
	"0-N": all possible CPUs, N denotes last CPU number
	"0,1-N:1/2": first and all CPUs with odd number
	"": empty list

est_nice - INTEGER
	default 0
	Valid range: -20 (more favorable) .. 19 (less favorable)

	Niceness value to use for the estimation kthreads (scheduling
	priority)

expire_nodest_conn - BOOLEAN
	- 0 - disabled (default)
	- not 0 - enabled

	The default value is 0, the load balancer will silently drop
	packets when its destination server is not available. It may
	be useful, when user-space monitoring program deletes the
	destination server (because of server overload or wrong
	detection) and add back the server later, and the connections
	to the server can continue.

	If this feature is enabled, the load balancer will expire the
	connection immediately when a packet arrives and its
	destination server is not available, then the client program
	will be notified that the connection is closed. This is
	equivalent to the feature some people requires to flush
	connections when its destination is not available.

expire_quiescent_template - BOOLEAN
	- 0 - disabled (default)
	- not 0 - enabled

	When set to a non-zero value, the load balancer will expire
	persistent templates when the destination server is quiescent.
	This may be useful, when a user makes a destination server
	quiescent by setting its weight to 0 and it is desired that
	subsequent otherwise persistent connections are sent to a
	different destination server.  By default new persistent
	connections are allowed to quiescent destination servers.

	If this feature is enabled, the load balancer will expire the
	persistence template if it is to be used to schedule a new
	connection and the destination server is quiescent.

ignore_tunneled - BOOLEAN
	- 0 - disabled (default)
	- not 0 - enabled

	If set, ipvs will set the ipvs_property on all packets which are of
	unrecognized protocols.  This prevents us from routing tunneled
	protocols like ipip, which is useful to prevent rescheduling
	packets that have been tunneled to the ipvs host (i.e. to prevent
	ipvs routing loops when ipvs is also acting as a real server).

nat_icmp_send - BOOLEAN
	- 0 - disabled (default)
	- not 0 - enabled

	It controls sending icmp error messages (ICMP_DEST_UNREACH)
	for VS/NAT when the load balancer receives packets from real
	servers but the connection entries don't exist.

pmtu_disc - BOOLEAN
	- 0 - disabled
	- not 0 - enabled (default)

	By default, reject with FRAG_NEEDED all DF packets that exceed
	the PMTU, irrespective of the forwarding method. For TUN method
	the flag can be disabled to fragment such packets.

secure_tcp - INTEGER
	- 0  - disabled (default)

	The secure_tcp defense is to use a more complicated TCP state
	transition table. For VS/NAT, it also delays entering the
	TCP ESTABLISHED state until the three way handshake is completed.

	The value definition is the same as that of drop_entry and
	drop_packet.

sync_threshold - vector of 2 INTEGERs: sync_threshold, sync_period
	default 3 50

	It sets synchronization threshold, which is the minimum number
	of incoming packets that a connection needs to receive before
	the connection will be synchronized. A connection will be
	synchronized, every time the number of its incoming packets
	modulus sync_period equals the threshold. The range of the
	threshold is from 0 to sync_period.

	When sync_period and sync_refresh_period are 0, send sync only
	for state changes or only once when pkts matches sync_threshold

sync_refresh_period - UNSIGNED INTEGER
	default 0

	In seconds, difference in reported connection timer that triggers
	new sync message. It can be used to avoid sync messages for the
	specified period (or half of the connection timeout if it is lower)
	if connection state is not changed since last sync.

	This is useful for normal connections with high traffic to reduce
	sync rate. Additionally, retry sync_retries times with period of
	sync_refresh_period/8.

sync_retries - INTEGER
	default 0

	Defines sync retries with period of sync_refresh_period/8. Useful
	to protect against loss of sync messages. The range of the
	sync_retries is from 0 to 3.

sync_qlen_max - UNSIGNED LONG

	Hard limit for queued sync messages that are not sent yet. It
	defaults to 1/32 of the memory pages but actually represents
	number of messages. It will protect us from allocating large
	parts of memory when the sending rate is lower than the queuing
	rate.

sync_sock_size - INTEGER
	default 0

	Configuration of SNDBUF (master) or RCVBUF (slave) socket limit.
	Default value is 0 (preserve system defaults).

sync_ports - INTEGER
	default 1

	The number of threads that master and backup servers can use for
	sync traffic. Every thread will use single UDP port, thread 0 will
	use the default port 8848 while last thread will use port
	8848+sync_ports-1.

snat_reroute - BOOLEAN
	- 0 - disabled
	- not 0 - enabled (default)

	If enabled, recalculate the route of SNATed packets from
	realservers so that they are routed as if they originate from the
	director. Otherwise they are routed as if they are forwarded by the
	director.

	If policy routing is in effect then it is possible that the route
	of a packet originating from a director is routed differently to a
	packet being forwarded by the director.

	If policy routing is not in effect then the recalculated route will
	always be the same as the original route so it is an optimisation
	to disable snat_reroute and avoid the recalculation.

sync_persist_mode - INTEGER
	default 0

	Controls the synchronisation of connections when using persistence

	0: All types of connections are synchronised

	1: Attempt to reduce the synchronisation traffic depending on
	the connection type. For persistent services avoid synchronisation
	for normal connections, do it only for persistence templates.
	In such case, for TCP and SCTP it may need enabling sloppy_tcp and
	sloppy_sctp flags on backup servers. For non-persistent services
	such optimization is not applied, mode 0 is assumed.

sync_version - INTEGER
	default 1

	The version of the synchronisation protocol used when sending
	synchronisation messages.

	0 selects the original synchronisation protocol (version 0). This
	should be used when sending synchronisation messages to a legacy
	system that only understands the original synchronisation protocol.

	1 selects the current synchronisation protocol (version 1). This
	should be used where possible.

	Kernels with this sync_version entry are able to receive messages
	of both version 1 and version 2 of the synchronisation protocol.

run_estimation - BOOLEAN
	0 - disabled
	not 0 - enabled (default)

	If disabled, the estimation will be suspended and kthread tasks
	stopped.

	You can always re-enable estimation by setting this value to 1.
	But be careful, the first estimation after re-enable is not
	accurate.


code:
.. SPDX-License-Identifier: (GPL-2.0 OR MIT)

===================
J1939 Documentation
===================

Overview / What Is J1939
========================

SAE J1939 defines a higher layer protocol on CAN. It implements a more
sophisticated addressing scheme and extends the maximum packet size above 8
bytes. Several derived specifications exist, which differ from the original
J1939 on the application level, like MilCAN A, NMEA2000, and especially
ISO-11783 (ISOBUS). This last one specifies the so-called ETP (Extended
Transport Protocol), which has been included in this implementation. This
results in a maximum packet size of ((2 ^ 24) - 1) * 7 bytes == 111 MiB.

Specifications used
-------------------

* SAE J1939-21 : data link layer
* SAE J1939-81 : network management
* ISO 11783-6  : Virtual Terminal (Extended Transport Protocol)

.. _j1939-motivation:

Motivation
==========

Given the fact there's something like SocketCAN with an API similar to BSD
sockets, we found some reasons to justify a kernel implementation for the
addressing and transport methods used by J1939.

* **Addressing:** when a process on an ECU communicates via J1939, it should
  not necessarily know its source address. Although, at least one process per
  ECU should know the source address. Other processes should be able to reuse
  that address. This way, address parameters for different processes
  cooperating for the same ECU, are not duplicated. This way of working is
  closely related to the UNIX concept, where programs do just one thing and do
  it well.

* **Dynamic addressing:** Address Claiming in J1939 is time critical.
  Furthermore, data transport should be handled properly during the address
  negotiation. Putting this functionality in the kernel eliminates it as a
  requirement for _every_ user space process that communicates via J1939. This
  results in a consistent J1939 bus with proper addressing.

* **Transport:** both TP & ETP reuse some PGNs to relay big packets over them.
  Different processes may thus use the same TP & ETP PGNs without actually
  knowing it. The individual TP & ETP sessions _must_ be serialized
  (synchronized) between different processes. The kernel solves this problem
  properly and eliminates the serialization (synchronization) as a requirement
  for _every_ user space process that communicates via J1939.

J1939 defines some other features (relaying, gateway, fast packet transport,
...). In-kernel code for these would not contribute to protocol stability.
Therefore, these parts are left to user space.

The J1939 sockets operate on CAN network devices (see SocketCAN). Any J1939
user space library operating on CAN raw sockets will still operate properly.
Since such a library does not communicate with the in-kernel implementation, care
must be taken that these two do not interfere. In practice, this means they
cannot share ECU addresses. A single ECU (or virtual ECU) address is used by
the library exclusively, or by the in-kernel system exclusively.

J1939 concepts
==============

PGN
---

The J1939 protocol uses the 29-bit CAN identifier with the following structure:

  ============  ==============  ====================
  29 bit CAN-ID
  --------------------------------------------------
  Bit positions within the CAN-ID
  --------------------------------------------------
  28 ... 26     25 ... 8        7 ... 0
  ============  ==============  ====================
  Priority      PGN             SA (Source Address)
  ============  ==============  ====================

The PGN (Parameter Group Number) is a number to identify a packet. The PGN
is composed as follows:

  ============  ==============  =================  =================
  PGN
  ------------------------------------------------------------------
  Bit positions within the CAN-ID
  ------------------------------------------------------------------
  25            24              23 ... 16          15 ... 8
  ============  ==============  =================  =================
  R (Reserved)  DP (Data Page)  PF (PDU Format)    PS (PDU Specific)
  ============  ==============  =================  =================

In J1939-21 distinction is made between PDU1 format (where PF < 240) and PDU2
format (where PF >= 240). Furthermore, when using the PDU2 format, the PS-field
contains a so-called Group Extension, which is part of the PGN. When using PDU2
format, the Group Extension is set in the PS-field.

  ==============  ========================
  PDU1 Format (specific) (peer to peer)
  ----------------------------------------
  Bit positions within the CAN-ID
  ----------------------------------------
  23 ... 16       15 ... 8
  ==============  ========================
  00h ... EFh     DA (Destination address)
  ==============  ========================

  ==============  ========================
  PDU2 Format (global) (broadcast)
  ----------------------------------------
  Bit positions within the CAN-ID
  ----------------------------------------
  23 ... 16       15 ... 8
  ==============  ========================
  F0h ... FFh     GE (Group Extenstion)
  ==============  ========================

On the other hand, when using PDU1 format, the PS-field contains a so-called
Destination Address, which is _not_ part of the PGN. When communicating a PGN
from user space to kernel (or vice versa) and PDU2 format is used, the PS-field
of the PGN shall be set to zero. The Destination Address shall be set
elsewhere.

Regarding PGN mapping to 29-bit CAN identifier, the Destination Address shall
be get/set from/to the appropriate bits of the identifier by the kernel.

Addressing
----------

Both static and dynamic addressing methods can be used.

For static addresses, no extra checks are made by the kernel and provided
addresses are considered right. This responsibility is for the OEM or system
integrator.

For dynamic addressing, so-called Address Claiming, extra support is foreseen
in the kernel. In J1939 any ECU is known by its 64-bit NAME. At the moment of
a successful address claim, the kernel keeps track of both NAME and source
address being claimed. This serves as a base for filter schemes. By default,
packets with a destination that is not locally will be rejected.

Mixed mode packets (from a static to a dynamic address or vice versa) are
allowed. The BSD sockets define separate API calls for getting/setting the
local & remote address and are applicable for J1939 sockets.

Filtering
---------

J1939 defines white list filters per socket that a user can set in order to
receive a subset of the J1939 traffic. Filtering can be based on:

* SA
* SOURCE_NAME
* PGN

When multiple filters are in place for a single socket, and a packet comes in
that matches several of those filters, the packet is only received once for
that socket.

How to Use J1939
================

API Calls
---------

On CAN, you first need to open a socket for communicating over a CAN network.
To use J1939, ``#include <linux/can/j1939.h>``. From there, ``<linux/can.h>`` will be
included too. To open a socket, use:

.. code-block:: C

    s = socket(PF_CAN, SOCK_DGRAM, CAN_J1939);

J1939 does use ``SOCK_DGRAM`` sockets. In the J1939 specification, connections are
mentioned in the context of transport protocol sessions. These still deliver
packets to the other end (using several CAN packets). ``SOCK_STREAM`` is not
supported.

After the successful creation of the socket, you would normally use the ``bind(2)``
and/or ``connect(2)`` system call to bind the socket to a CAN interface. After
binding and/or connecting the socket, you can ``read(2)`` and ``write(2)`` from/to the
socket or use ``send(2)``, ``sendto(2)``, ``sendmsg(2)`` and the ``recv*()`` counterpart
operations on the socket as usual. There are also J1939 specific socket options
described below.

In order to send data, a ``bind(2)`` must have been successful. ``bind(2)`` assigns a
local address to a socket.

Different from CAN is that the payload data is just the data that get sends,
without its header info. The header info is derived from the sockaddr supplied
to ``bind(2)``, ``connect(2)``, ``sendto(2)`` and ``recvfrom(2)``. A ``write(2)`` with size 4 will
result in a packet with 4 bytes.

The sockaddr structure has extensions for use with J1939 as specified below:

.. code-block:: C

      struct sockaddr_can {
         sa_family_t can_family;
         int         can_ifindex;
         union {
            struct {
               __u64 name;
                        /* pgn:
                         * 8 bit: PS in PDU2 case, else 0
                         * 8 bit: PF
                         * 1 bit: DP
                         * 1 bit: reserved
                         */
               __u32 pgn;
               __u8  addr;
            } j1939;
         } can_addr;
      }

``can_family`` & ``can_ifindex`` serve the same purpose as for other SocketCAN sockets.

``can_addr.j1939.pgn`` specifies the PGN (max 0x3ffff). Individual bits are
specified above.

``can_addr.j1939.name`` contains the 64-bit J1939 NAME.

``can_addr.j1939.addr`` contains the address.

The ``bind(2)`` system call assigns the local address, i.e. the source address when
sending packages. If a PGN during ``bind(2)`` is set, it's used as a RX filter.
I.e. only packets with a matching PGN are received. If an ADDR or NAME is set
it is used as a receive filter, too. It will match the destination NAME or ADDR
of the incoming packet. The NAME filter will work only if appropriate Address
Claiming for this name was done on the CAN bus and registered/cached by the
kernel.

On the other hand ``connect(2)`` assigns the remote address, i.e. the destination
address. The PGN from ``connect(2)`` is used as the default PGN when sending
packets. If ADDR or NAME is set it will be used as the default destination ADDR
or NAME. Further a set ADDR or NAME during ``connect(2)`` is used as a receive
filter. It will match the source NAME or ADDR of the incoming packet.

Both ``write(2)`` and ``send(2)`` will send a packet with local address from ``bind(2)`` and the
remote address from ``connect(2)``. Use ``sendto(2)`` to overwrite the destination
address.

If ``can_addr.j1939.name`` is set (!= 0) the NAME is looked up by the kernel and
the corresponding ADDR is used. If ``can_addr.j1939.name`` is not set (== 0),
``can_addr.j1939.addr`` is used.

When creating a socket, reasonable defaults are set. Some options can be
modified with ``setsockopt(2)`` & ``getsockopt(2)``.

RX path related options:

- ``SO_J1939_FILTER`` - configure array of filters
- ``SO_J1939_PROMISC`` - disable filters set by ``bind(2)`` and ``connect(2)``

By default no broadcast packets can be send or received. To enable sending or
receiving broadcast packets use the socket option ``SO_BROADCAST``:

.. code-block:: C

     int value = 1;
     setsockopt(sock, SOL_SOCKET, SO_BROADCAST, &value, sizeof(value));

The following diagram illustrates the RX path:

.. code::

                    +--------------------+
                    |  incoming packet   |
                    +--------------------+
                              |
                              V
                    +--------------------+
                    | SO_J1939_PROMISC?  |
                    +--------------------+
                             |  |
                         no  |  | yes
                             |  |
                   .---------'  `---------.
                   |                      |
     +---------------------------+        |
     | bind() + connect() +      |        |
     | SOCK_BROADCAST filter     |        |
     +---------------------------+        |
                   |                      |
                   |<---------------------'
                   V
     +---------------------------+
     |      SO_J1939_FILTER      |
     +---------------------------+
                   |
                   V
     +---------------------------+
     |        socket recv()      |
     +---------------------------+

TX path related options:
``SO_J1939_SEND_PRIO`` - change default send priority for the socket

Message Flags during send() and Related System Calls
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

``send(2)``, ``sendto(2)`` and ``sendmsg(2)`` take a 'flags' argument. Currently
supported flags are:

* ``MSG_DONTWAIT``, i.e. non-blocking operation.

recvmsg(2)
^^^^^^^^^^

In most cases ``recvmsg(2)`` is needed if you want to extract more information than
``recvfrom(2)`` can provide. For example package priority and timestamp. The
Destination Address, name and packet priority (if applicable) are attached to
the msghdr in the ``recvmsg(2)`` call. They can be extracted using ``cmsg(3)`` macros,
with ``cmsg_level == SOL_J1939 && cmsg_type == SCM_J1939_DEST_ADDR``,
``SCM_J1939_DEST_NAME`` or ``SCM_J1939_PRIO``. The returned data is a ``uint8_t`` for
``priority`` and ``dst_addr``, and ``uint64_t`` for ``dst_name``.

.. code-block:: C

	uint8_t priority, dst_addr;
	uint64_t dst_name;

	for (cmsg = CMSG_FIRSTHDR(&msg); cmsg; cmsg = CMSG_NXTHDR(&msg, cmsg)) {
		switch (cmsg->cmsg_level) {
		case SOL_CAN_J1939:
			if (cmsg->cmsg_type == SCM_J1939_DEST_ADDR)
				dst_addr = *CMSG_DATA(cmsg);
			else if (cmsg->cmsg_type == SCM_J1939_DEST_NAME)
				memcpy(&dst_name, CMSG_DATA(cmsg), cmsg->cmsg_len - CMSG_LEN(0));
			else if (cmsg->cmsg_type == SCM_J1939_PRIO)
				priority = *CMSG_DATA(cmsg);
			break;
		}
	}

Dynamic Addressing
------------------

Distinction has to be made between using the claimed address and doing an
address claim. To use an already claimed address, one has to fill in the
``j1939.name`` member and provide it to ``bind(2)``. If the name had claimed an address
earlier, all further messages being sent will use that address. And the
``j1939.addr`` member will be ignored.

An exception on this is PGN 0x0ee00. This is the "Address Claim/Cannot Claim
Address" message and the kernel will use the ``j1939.addr`` member for that PGN if
necessary.

To claim an address following code example can be used:

.. code-block:: C

	struct sockaddr_can baddr = {
		.can_family = AF_CAN,
		.can_addr.j1939 = {
			.name = name,
			.addr = J1939_IDLE_ADDR,
			.pgn = J1939_NO_PGN,	/* to disable bind() rx filter for PGN */
		},
		.can_ifindex = if_nametoindex("can0"),
	};

	bind(sock, (struct sockaddr *)&baddr, sizeof(baddr));

	/* for Address Claiming broadcast must be allowed */
	int value = 1;
	setsockopt(sock, SOL_SOCKET, SO_BROADCAST, &value, sizeof(value));

	/* configured advanced RX filter with PGN needed for Address Claiming */
	const struct j1939_filter filt[] = {
		{
			.pgn = J1939_PGN_ADDRESS_CLAIMED,
			.pgn_mask = J1939_PGN_PDU1_MAX,
		}, {
			.pgn = J1939_PGN_REQUEST,
			.pgn_mask = J1939_PGN_PDU1_MAX,
		}, {
			.pgn = J1939_PGN_ADDRESS_COMMANDED,
			.pgn_mask = J1939_PGN_MAX,
		},
	};

	setsockopt(sock, SOL_CAN_J1939, SO_J1939_FILTER, &filt, sizeof(filt));

	uint64_t dat = htole64(name);
	const struct sockaddr_can saddr = {
		.can_family = AF_CAN,
		.can_addr.j1939 = {
			.pgn = J1939_PGN_ADDRESS_CLAIMED,
			.addr = J1939_NO_ADDR,
		},
	};

	/* Afterwards do a sendto(2) with data set to the NAME (Little Endian). If the
	 * NAME provided, does not match the j1939.name provided to bind(2), EPROTO
	 * will be returned.
	 */
	sendto(sock, dat, sizeof(dat), 0, (const struct sockaddr *)&saddr, sizeof(saddr));

If no-one else contests the address claim within 250ms after transmission, the
kernel marks the NAME-SA assignment as valid. The valid assignment will be kept
among other valid NAME-SA assignments. From that point, any socket bound to the
NAME can send packets.

If another ECU claims the address, the kernel will mark the NAME-SA expired.
No socket bound to the NAME can send packets (other than address claims). To
claim another address, some socket bound to NAME, must ``bind(2)`` again, but with
only ``j1939.addr`` changed to the new SA, and must then send a valid address claim
packet. This restarts the state machine in the kernel (and any other
participant on the bus) for this NAME.

``can-utils`` also include the ``j1939acd`` tool, so it can be used as code example or as
default Address Claiming daemon.

Send Examples
-------------

Static Addressing
^^^^^^^^^^^^^^^^^

This example will send a PGN (0x12300) from SA 0x20 to DA 0x30.

Bind:

.. code-block:: C

	struct sockaddr_can baddr = {
		.can_family = AF_CAN,
		.can_addr.j1939 = {
			.name = J1939_NO_NAME,
			.addr = 0x20,
			.pgn = J1939_NO_PGN,
		},
		.can_ifindex = if_nametoindex("can0"),
	};

	bind(sock, (struct sockaddr *)&baddr, sizeof(baddr));

Now, the socket 'sock' is bound to the SA 0x20. Since no ``connect(2)`` was called,
at this point we can use only ``sendto(2)`` or ``sendmsg(2)``.

Send:

.. code-block:: C

	const struct sockaddr_can saddr = {
		.can_family = AF_CAN,
		.can_addr.j1939 = {
			.name = J1939_NO_NAME;
			.addr = 0x30,
			.pgn = 0x12300,
		},
	};

	sendto(sock, dat, sizeof(dat), 0, (const struct sockaddr *)&saddr, sizeof(saddr));


code:
=========================================
Linux Networking and Network Devices APIs
=========================================

Linux Networking
================

Networking Base Types
---------------------

.. kernel-doc:: include/linux/net.h
   :internal:

Socket Buffer Functions
-----------------------

.. kernel-doc:: include/linux/skbuff.h
   :internal:

.. kernel-doc:: include/net/sock.h
   :internal:

.. kernel-doc:: net/socket.c
   :export:

.. kernel-doc:: net/core/skbuff.c
   :export:

.. kernel-doc:: net/core/sock.c
   :export:

.. kernel-doc:: net/core/datagram.c
   :export:

.. kernel-doc:: net/core/stream.c
   :export:

Socket Filter
-------------

.. kernel-doc:: net/core/filter.c
   :export:

Generic Network Statistics
--------------------------

.. kernel-doc:: include/uapi/linux/gen_stats.h
   :internal:

.. kernel-doc:: net/core/gen_stats.c
   :export:

.. kernel-doc:: net/core/gen_estimator.c
   :export:

SUN RPC subsystem
-----------------

.. kernel-doc:: net/sunrpc/xdr.c
   :export:

.. kernel-doc:: net/sunrpc/svc_xprt.c
   :export:

.. kernel-doc:: net/sunrpc/xprt.c
   :export:

.. kernel-doc:: net/sunrpc/sched.c
   :export:

.. kernel-doc:: net/sunrpc/socklib.c
   :export:

.. kernel-doc:: net/sunrpc/stats.c
   :export:

.. kernel-doc:: net/sunrpc/rpc_pipe.c
   :export:

.. kernel-doc:: net/sunrpc/rpcb_clnt.c
   :export:

.. kernel-doc:: net/sunrpc/clnt.c
   :export:

Network device support
======================

Driver Support
--------------

.. kernel-doc:: net/core/dev.c
   :export:

.. kernel-doc:: net/ethernet/eth.c
   :export:

.. kernel-doc:: net/sched/sch_generic.c
   :export:

.. kernel-doc:: include/linux/etherdevice.h
   :internal:

.. kernel-doc:: include/linux/netdevice.h
   :internal:

PHY Support
-----------

.. kernel-doc:: drivers/net/phy/phy.c
   :export:

.. kernel-doc:: drivers/net/phy/phy.c
   :internal:

.. kernel-doc:: drivers/net/phy/phy-core.c
   :export:

.. kernel-doc:: drivers/net/phy/phy-c45.c
   :export:

.. kernel-doc:: include/linux/phy.h
   :internal:

.. kernel-doc:: drivers/net/phy/phy_device.c
   :export:

.. kernel-doc:: drivers/net/phy/phy_device.c
   :internal:

.. kernel-doc:: drivers/net/phy/mdio_bus.c
   :export:

.. kernel-doc:: drivers/net/phy/mdio_bus.c
   :internal:

PHYLINK
-------

  PHYLINK interfaces traditional network drivers with PHYLIB, fixed-links,
  and SFF modules (eg, hot-pluggable SFP) that may contain PHYs.  PHYLINK
  provides management of the link state and link modes.

.. kernel-doc:: include/linux/phylink.h
   :internal:

.. kernel-doc:: drivers/net/phy/phylink.c

SFP support
-----------

.. kernel-doc:: drivers/net/phy/sfp-bus.c
   :internal:

.. kernel-doc:: include/linux/sfp.h
   :internal:

.. kernel-doc:: drivers/net/phy/sfp-bus.c
   :export:


code:
.. SPDX-License-Identifier: GPL-2.0

=============================
Kernel Connection Multiplexor
=============================

Kernel Connection Multiplexor (KCM) is a mechanism that provides a message based
interface over TCP for generic application protocols. With KCM an application
can efficiently send and receive application protocol messages over TCP using
datagram sockets.

KCM implements an NxM multiplexor in the kernel as diagrammed below::

    +------------+   +------------+   +------------+   +------------+
    | KCM socket |   | KCM socket |   | KCM socket |   | KCM socket |
    +------------+   +------------+   +------------+   +------------+
	|                 |               |                |
	+-----------+     |               |     +----------+
		    |     |               |     |
		+----------------------------------+
		|           Multiplexor            |
		+----------------------------------+
		    |   |           |           |  |
	+---------+   |           |           |  ------------+
	|             |           |           |              |
    +----------+  +----------+  +----------+  +----------+ +----------+
    |  Psock   |  |  Psock   |  |  Psock   |  |  Psock   | |  Psock   |
    +----------+  +----------+  +----------+  +----------+ +----------+
	|              |           |            |             |
    +----------+  +----------+  +----------+  +----------+ +----------+
    | TCP sock |  | TCP sock |  | TCP sock |  | TCP sock | | TCP sock |
    +----------+  +----------+  +----------+  +----------+ +----------+

KCM sockets
===========

The KCM sockets provide the user interface to the multiplexor. All the KCM sockets
bound to a multiplexor are considered to have equivalent function, and I/O
operations in different sockets may be done in parallel without the need for
synchronization between threads in userspace.

Multiplexor
===========

The multiplexor provides the message steering. In the transmit path, messages
written on a KCM socket are sent atomically on an appropriate TCP socket.
Similarly, in the receive path, messages are constructed on each TCP socket
(Psock) and complete messages are steered to a KCM socket.

TCP sockets & Psocks
====================

TCP sockets may be bound to a KCM multiplexor. A Psock structure is allocated
for each bound TCP socket, this structure holds the state for constructing
messages on receive as well as other connection specific information for KCM.

Connected mode semantics
========================

Each multiplexor assumes that all attached TCP connections are to the same
destination and can use the different connections for load balancing when
transmitting. The normal send and recv calls (include sendmmsg and recvmmsg)
can be used to send and receive messages from the KCM socket.

Socket types
============

KCM supports SOCK_DGRAM and SOCK_SEQPACKET socket types.

Message delineation
-------------------

Messages are sent over a TCP stream with some application protocol message
format that typically includes a header which frames the messages. The length
of a received message can be deduced from the application protocol header
(often just a simple length field).

A TCP stream must be parsed to determine message boundaries. Berkeley Packet
Filter (BPF) is used for this. When attaching a TCP socket to a multiplexor a
BPF program must be specified. The program is called at the start of receiving
a new message and is given an skbuff that contains the bytes received so far.
It parses the message header and returns the length of the message. Given this
information, KCM will construct the message of the stated length and deliver it
to a KCM socket.

TCP socket management
---------------------

When a TCP socket is attached to a KCM multiplexor data ready (POLLIN) and
write space available (POLLOUT) events are handled by the multiplexor. If there
is a state change (disconnection) or other error on a TCP socket, an error is
posted on the TCP socket so that a POLLERR event happens and KCM discontinues
using the socket. When the application gets the error notification for a
TCP socket, it should unattach the socket from KCM and then handle the error
condition (the typical response is to close the socket and create a new
connection if necessary).

KCM limits the maximum receive message size to be the size of the receive
socket buffer on the attached TCP socket (the socket buffer size can be set by
SO_RCVBUF). If the length of a new message reported by the BPF program is
greater than this limit a corresponding error (EMSGSIZE) is posted on the TCP
socket. The BPF program may also enforce a maximum messages size and report an
error when it is exceeded.

A timeout may be set for assembling messages on a receive socket. The timeout
value is taken from the receive timeout of the attached TCP socket (this is set
by SO_RCVTIMEO). If the timer expires before assembly is complete an error
(ETIMEDOUT) is posted on the socket.

User interface
==============

Creating a multiplexor
----------------------

A new multiplexor and initial KCM socket is created by a socket call::

  socket(AF_KCM, type, protocol)

- type is either SOCK_DGRAM or SOCK_SEQPACKET
- protocol is KCMPROTO_CONNECTED

Cloning KCM sockets
-------------------

After the first KCM socket is created using the socket call as described
above, additional sockets for the multiplexor can be created by cloning
a KCM socket. This is accomplished by an ioctl on a KCM socket::

  /* From linux/kcm.h */
  struct kcm_clone {
	int fd;
  };

  struct kcm_clone info;

  memset(&info, 0, sizeof(info));

  err = ioctl(kcmfd, SIOCKCMCLONE, &info);

  if (!err)
    newkcmfd = info.fd;

Attach transport sockets
------------------------

Attaching of transport sockets to a multiplexor is performed by calling an
ioctl on a KCM socket for the multiplexor. e.g.::

  /* From linux/kcm.h */
  struct kcm_attach {
	int fd;
	int bpf_fd;
  };

  struct kcm_attach info;

  memset(&info, 0, sizeof(info));

  info.fd = tcpfd;
  info.bpf_fd = bpf_prog_fd;

  ioctl(kcmfd, SIOCKCMATTACH, &info);

The kcm_attach structure contains:

  - fd: file descriptor for TCP socket being attached
  - bpf_prog_fd: file descriptor for compiled BPF program downloaded

Unattach transport sockets
--------------------------

Unattaching a transport socket from a multiplexor is straightforward. An
"unattach" ioctl is done with the kcm_unattach structure as the argument::

  /* From linux/kcm.h */
  struct kcm_unattach {
	int fd;
  };

  struct kcm_unattach info;

  memset(&info, 0, sizeof(info));

  info.fd = cfd;

  ioctl(fd, SIOCKCMUNATTACH, &info);

Disabling receive on KCM socket
-------------------------------

A setsockopt is used to disable or enable receiving on a KCM socket.
When receive is disabled, any pending messages in the socket's
receive buffer are moved to other sockets. This feature is useful
if an application thread knows that it will be doing a lot of
work on a request and won't be able to service new messages for a
while. Example use::

  int val = 1;

  setsockopt(kcmfd, SOL_KCM, KCM_RECV_DISABLE, &val, sizeof(val))

BFP programs for message delineation
------------------------------------

BPF programs can be compiled using the BPF LLVM backend. For example,
the BPF program for parsing Thrift is::

  #include "bpf.h" /* for __sk_buff */
  #include "bpf_helpers.h" /* for load_word intrinsic */

  SEC("socket_kcm")
  int bpf_prog1(struct __sk_buff *skb)
  {
       return load_word(skb, 0) + 4;
  }

  char _license[] SEC("license") = "GPL";

Use in applications
===================

KCM accelerates application layer protocols. Specifically, it allows
applications to use a message based interface for sending and receiving
messages. The kernel provides necessary assurances that messages are sent
and received atomically. This relieves much of the burden applications have
in mapping a message based protocol onto the TCP stream. KCM also make
application layer messages a unit of work in the kernel for the purposes of
steering and scheduling, which in turn allows a simpler networking model in
multithreaded applications.

Configurations
--------------

In an Nx1 configuration, KCM logically provides multiple socket handles
to the same TCP connection. This allows parallelism between in I/O
operations on the TCP socket (for instance copyin and copyout of data is
parallelized). In an application, a KCM socket can be opened for each
processing thread and inserted into the epoll (similar to how SO_REUSEPORT
is used to allow multiple listener sockets on the same port).

In a MxN configuration, multiple connections are established to the
same destination. These are used for simple load balancing.

Message batching
----------------

The primary purpose of KCM is load balancing between KCM sockets and hence
threads in a nominal use case. Perfect load balancing, that is steering
each received message to a different KCM socket or steering each sent
message to a different TCP socket, can negatively impact performance
since this doesn't allow for affinities to be established. Balancing
based on groups, or batches of messages, can be beneficial for performance.

On transmit, there are three ways an application can batch (pipeline)
messages on a KCM socket.

  1) Send multiple messages in a single sendmmsg.
  2) Send a group of messages each with a sendmsg call, where all messages
     except the last have MSG_BATCH in the flags of sendmsg call.
  3) Create "super message" composed of multiple messages and send this
     with a single sendmsg.

On receive, the KCM module attempts to queue messages received on the
same KCM socket during each TCP ready callback. The targeted KCM socket
changes at each receive ready callback on the KCM socket. The application
does not need to configure this.

Error handling
--------------

An application should include a thread to monitor errors raised on
the TCP connection. Normally, this will be done by placing each
TCP socket attached to a KCM multiplexor in epoll set for POLLERR
event. If an error occurs on an attached TCP socket, KCM sets an EPIPE
on the socket thus waking up the application thread. When the application
sees the error (which may just be a disconnect) it should unattach the
socket from KCM and then close it. It is assumed that once an error is
posted on the TCP socket the data stream is unrecoverable (i.e. an error
may have occurred in the middle of receiving a message).

TCP connection monitoring
-------------------------

In KCM there is no means to correlate a message to the TCP socket that
was used to send or receive the message (except in the case there is
only one attached TCP socket). However, the application does retain
an open file descriptor to the socket so it will be able to get statistics
from the socket which can be used in detecting issues (such as high
retransmissions on the socket).


code:
.. SPDX-License-Identifier: GPL-2.0

====
L2TP
====

Layer 2 Tunneling Protocol (L2TP) allows L2 frames to be tunneled over
an IP network.

This document covers the kernel's L2TP subsystem. It documents kernel
APIs for application developers who want to use the L2TP subsystem and
it provides some technical details about the internal implementation
which may be useful to kernel developers and maintainers.

Overview
========

The kernel's L2TP subsystem implements the datapath for L2TPv2 and
L2TPv3. L2TPv2 is carried over UDP. L2TPv3 is carried over UDP or
directly over IP (protocol 115).

The L2TP RFCs define two basic kinds of L2TP packets: control packets
(the "control plane"), and data packets (the "data plane"). The kernel
deals only with data packets. The more complex control packets are
handled by user space.

An L2TP tunnel carries one or more L2TP sessions. Each tunnel is
associated with a socket. Each session is associated with a virtual
netdevice, e.g. ``pppN``, ``l2tpethN``, through which data frames pass
to/from L2TP. Fields in the L2TP header identify the tunnel or session
and whether it is a control or data packet. When tunnels and sessions
are set up using the Linux kernel API, we're just setting up the L2TP
data path. All aspects of the control protocol are to be handled by
user space.

This split in responsibilities leads to a natural sequence of
operations when establishing tunnels and sessions. The procedure looks
like this:

    1) Create a tunnel socket. Exchange L2TP control protocol messages
       with the peer over that socket in order to establish a tunnel.

    2) Create a tunnel context in the kernel, using information
       obtained from the peer using the control protocol messages.

    3) Exchange L2TP control protocol messages with the peer over the
       tunnel socket in order to establish a session.

    4) Create a session context in the kernel using information
       obtained from the peer using the control protocol messages.

L2TP APIs
=========

This section documents each userspace API of the L2TP subsystem.

Tunnel Sockets
--------------

L2TPv2 always uses UDP. L2TPv3 may use UDP or IP encapsulation.

To create a tunnel socket for use by L2TP, the standard POSIX
socket API is used.

For example, for a tunnel using IPv4 addresses and UDP encapsulation::

    int sockfd = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);

Or for a tunnel using IPv6 addresses and IP encapsulation::

    int sockfd = socket(AF_INET6, SOCK_DGRAM, IPPROTO_L2TP);

UDP socket programming doesn't need to be covered here.

IPPROTO_L2TP is an IP protocol type implemented by the kernel's L2TP
subsystem. The L2TPIP socket address is defined in struct
sockaddr_l2tpip and struct sockaddr_l2tpip6 at
`include/uapi/linux/l2tp.h`_. The address includes the L2TP tunnel
(connection) id. To use L2TP IP encapsulation, an L2TPv3 application
should bind the L2TPIP socket using the locally assigned
tunnel id. When the peer's tunnel id and IP address is known, a
connect must be done.

If the L2TP application needs to handle L2TPv3 tunnel setup requests
from peers using L2TPIP, it must open a dedicated L2TPIP
socket to listen for those requests and bind the socket using tunnel
id 0 since tunnel setup requests are addressed to tunnel id 0.

An L2TP tunnel and all of its sessions are automatically closed when
its tunnel socket is closed.

Netlink API
-----------

L2TP applications use netlink to manage L2TP tunnel and session
instances in the kernel. The L2TP netlink API is defined in
`include/uapi/linux/l2tp.h`_.

L2TP uses `Generic Netlink`_ (GENL). Several commands are defined:
Create, Delete, Modify and Get for tunnel and session
instances, e.g. ``L2TP_CMD_TUNNEL_CREATE``. The API header lists the
netlink attribute types that can be used with each command.

Tunnel and session instances are identified by a locally unique
32-bit id.  L2TP tunnel ids are given by ``L2TP_ATTR_CONN_ID`` and
``L2TP_ATTR_PEER_CONN_ID`` attributes and L2TP session ids are given
by ``L2TP_ATTR_SESSION_ID`` and ``L2TP_ATTR_PEER_SESSION_ID``
attributes. If netlink is used to manage L2TPv2 tunnel and session
instances, the L2TPv2 16-bit tunnel/session id is cast to a 32-bit
value in these attributes.

In the ``L2TP_CMD_TUNNEL_CREATE`` command, ``L2TP_ATTR_FD`` tells the
kernel the tunnel socket fd being used. If not specified, the kernel
creates a kernel socket for the tunnel, using IP parameters set in
``L2TP_ATTR_IP[6]_SADDR``, ``L2TP_ATTR_IP[6]_DADDR``,
``L2TP_ATTR_UDP_SPORT``, ``L2TP_ATTR_UDP_DPORT`` attributes. Kernel
sockets are used to implement unmanaged L2TPv3 tunnels (iproute2's "ip
l2tp" commands). If ``L2TP_ATTR_FD`` is given, it must be a socket fd
that is already bound and connected. There is more information about
unmanaged tunnels later in this document.

``L2TP_CMD_TUNNEL_CREATE`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        Sets the tunnel (connection) id.
PEER_CONN_ID       Y        Sets the peer tunnel (connection) id.
PROTO_VERSION      Y        Protocol version. 2 or 3.
ENCAP_TYPE         Y        Encapsulation type: UDP or IP.
FD                 N        Tunnel socket file descriptor.
UDP_CSUM           N        Enable IPv4 UDP checksums. Used only if FD is
                            not set.
UDP_ZERO_CSUM6_TX  N        Zero IPv6 UDP checksum on transmit. Used only
                            if FD is not set.
UDP_ZERO_CSUM6_RX  N        Zero IPv6 UDP checksum on receive. Used only if
                            FD is not set.
IP_SADDR           N        IPv4 source address. Used only if FD is not
                            set.
IP_DADDR           N        IPv4 destination address. Used only if FD is
                            not set.
UDP_SPORT          N        UDP source port. Used only if FD is not set.
UDP_DPORT          N        UDP destination port. Used only if FD is not
                            set.
IP6_SADDR          N        IPv6 source address. Used only if FD is not
                            set.
IP6_DADDR          N        IPv6 destination address. Used only if FD is
                            not set.
DEBUG              N        Debug flags.
================== ======== ===

``L2TP_CMD_TUNNEL_DESTROY`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        Identifies the tunnel id to be destroyed.
================== ======== ===

``L2TP_CMD_TUNNEL_MODIFY`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        Identifies the tunnel id to be modified.
DEBUG              N        Debug flags.
================== ======== ===

``L2TP_CMD_TUNNEL_GET`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            N        Identifies the tunnel id to be queried.
                            Ignored in DUMP requests.
================== ======== ===

``L2TP_CMD_SESSION_CREATE`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        The parent tunnel id.
SESSION_ID         Y        Sets the session id.
PEER_SESSION_ID    Y        Sets the parent session id.
PW_TYPE            Y        Sets the pseudowire type.
DEBUG              N        Debug flags.
RECV_SEQ           N        Enable rx data sequence numbers.
SEND_SEQ           N        Enable tx data sequence numbers.
LNS_MODE           N        Enable LNS mode (auto-enable data sequence
                            numbers).
RECV_TIMEOUT       N        Timeout to wait when reordering received
                            packets.
L2SPEC_TYPE        N        Sets layer2-specific-sublayer type (L2TPv3
                            only).
COOKIE             N        Sets optional cookie (L2TPv3 only).
PEER_COOKIE        N        Sets optional peer cookie (L2TPv3 only).
IFNAME             N        Sets interface name (L2TPv3 only).
================== ======== ===

For Ethernet session types, this will create an l2tpeth virtual
interface which can then be configured as required. For PPP session
types, a PPPoL2TP socket must also be opened and connected, mapping it
onto the new session. This is covered in "PPPoL2TP Sockets" later.

``L2TP_CMD_SESSION_DESTROY`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        Identifies the parent tunnel id of the session
                            to be destroyed.
SESSION_ID         Y        Identifies the session id to be destroyed.
IFNAME             N        Identifies the session by interface name. If
                            set, this overrides any CONN_ID and SESSION_ID
                            attributes. Currently supported for L2TPv3
                            Ethernet sessions only.
================== ======== ===

``L2TP_CMD_SESSION_MODIFY`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        Identifies the parent tunnel id of the session
                            to be modified.
SESSION_ID         Y        Identifies the session id to be modified.
IFNAME             N        Identifies the session by interface name. If
                            set, this overrides any CONN_ID and SESSION_ID
                            attributes. Currently supported for L2TPv3
                            Ethernet sessions only.
DEBUG              N        Debug flags.
RECV_SEQ           N        Enable rx data sequence numbers.
SEND_SEQ           N        Enable tx data sequence numbers.
LNS_MODE           N        Enable LNS mode (auto-enable data sequence
                            numbers).
RECV_TIMEOUT       N        Timeout to wait when reordering received
                            packets.
================== ======== ===

``L2TP_CMD_SESSION_GET`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            N        Identifies the tunnel id to be queried.
                            Ignored for DUMP requests.
SESSION_ID         N        Identifies the session id to be queried.
                            Ignored for DUMP requests.
IFNAME             N        Identifies the session by interface name.
                            If set, this overrides any CONN_ID and
                            SESSION_ID attributes. Ignored for DUMP
                            requests. Currently supported for L2TPv3
                            Ethernet sessions only.
================== ======== ===

Application developers should refer to `include/uapi/linux/l2tp.h`_ for
netlink command and attribute definitions.

Sample userspace code using libmnl_:

  - Open L2TP netlink socket::

        struct nl_sock *nl_sock;
        int l2tp_nl_family_id;

        nl_sock = nl_socket_alloc();
        genl_connect(nl_sock);
        genl_id = genl_ctrl_resolve(nl_sock, L2TP_GENL_NAME);

  - Create a tunnel::

        struct nlmsghdr *nlh;
        struct genlmsghdr *gnlh;

        nlh = mnl_nlmsg_put_header(buf);
        nlh->nlmsg_type = genl_id; /* assigned to genl socket */
        nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK;
        nlh->nlmsg_seq = seq;

        gnlh = mnl_nlmsg_put_extra_header(nlh, sizeof(*gnlh));
        gnlh->cmd = L2TP_CMD_TUNNEL_CREATE;
        gnlh->version = L2TP_GENL_VERSION;
        gnlh->reserved = 0;

        mnl_attr_put_u32(nlh, L2TP_ATTR_FD, tunl_sock_fd);
        mnl_attr_put_u32(nlh, L2TP_ATTR_CONN_ID, tid);
        mnl_attr_put_u32(nlh, L2TP_ATTR_PEER_CONN_ID, peer_tid);
        mnl_attr_put_u8(nlh, L2TP_ATTR_PROTO_VERSION, protocol_version);
        mnl_attr_put_u16(nlh, L2TP_ATTR_ENCAP_TYPE, encap);

  - Create a session::

        struct nlmsghdr *nlh;
        struct genlmsghdr *gnlh;

        nlh = mnl_nlmsg_put_header(buf);
        nlh->nlmsg_type = genl_id; /* assigned to genl socket */
        nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK;
        nlh->nlmsg_seq = seq;

        gnlh = mnl_nlmsg_put_extra_header(nlh, sizeof(*gnlh));
        gnlh->cmd = L2TP_CMD_SESSION_CREATE;
        gnlh->version = L2TP_GENL_VERSION;
        gnlh->reserved = 0;

        mnl_attr_put_u32(nlh, L2TP_ATTR_CONN_ID, tid);
        mnl_attr_put_u32(nlh, L2TP_ATTR_PEER_CONN_ID, peer_tid);
        mnl_attr_put_u32(nlh, L2TP_ATTR_SESSION_ID, sid);
        mnl_attr_put_u32(nlh, L2TP_ATTR_PEER_SESSION_ID, peer_sid);
        mnl_attr_put_u16(nlh, L2TP_ATTR_PW_TYPE, pwtype);
        /* there are other session options which can be set using netlink
         * attributes during session creation -- see l2tp.h
         */

  - Delete a session::

        struct nlmsghdr *nlh;
        struct genlmsghdr *gnlh;

        nlh = mnl_nlmsg_put_header(buf);
        nlh->nlmsg_type = genl_id; /* assigned to genl socket */
        nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK;
        nlh->nlmsg_seq = seq;

        gnlh = mnl_nlmsg_put_extra_header(nlh, sizeof(*gnlh));
        gnlh->cmd = L2TP_CMD_SESSION_DELETE;
        gnlh->version = L2TP_GENL_VERSION;
        gnlh->reserved = 0;

        mnl_attr_put_u32(nlh, L2TP_ATTR_CONN_ID, tid);
        mnl_attr_put_u32(nlh, L2TP_ATTR_SESSION_ID, sid);

  - Delete a tunnel and all of its sessions (if any)::

        struct nlmsghdr *nlh;
        struct genlmsghdr *gnlh;

        nlh = mnl_nlmsg_put_header(buf);
        nlh->nlmsg_type = genl_id; /* assigned to genl socket */
        nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK;
        nlh->nlmsg_seq = seq;

        gnlh = mnl_nlmsg_put_extra_header(nlh, sizeof(*gnlh));
        gnlh->cmd = L2TP_CMD_TUNNEL_DELETE;
        gnlh->version = L2TP_GENL_VERSION;
        gnlh->reserved = 0;

        mnl_attr_put_u32(nlh, L2TP_ATTR_CONN_ID, tid);

PPPoL2TP Session Socket API
---------------------------

For PPP session types, a PPPoL2TP socket must be opened and connected
to the L2TP session.

When creating PPPoL2TP sockets, the application provides information
to the kernel about the tunnel and session in a socket connect()
call. Source and destination tunnel and session ids are provided, as
well as the file descriptor of a UDP or L2TPIP socket. See struct
pppol2tp_addr in `include/linux/if_pppol2tp.h`_. For historical reasons,
there are unfortunately slightly different address structures for
L2TPv2/L2TPv3 IPv4/IPv6 tunnels and userspace must use the appropriate
structure that matches the tunnel socket type.

Userspace may control behavior of the tunnel or session using
setsockopt and ioctl on the PPPoX socket. The following socket
options are supported:-

=========   ===========================================================
DEBUG       bitmask of debug message categories. See below.
SENDSEQ     - 0 => don't send packets with sequence numbers
            - 1 => send packets with sequence numbers
RECVSEQ     - 0 => receive packet sequence numbers are optional
            - 1 => drop receive packets without sequence numbers
LNSMODE     - 0 => act as LAC.
            - 1 => act as LNS.
REORDERTO   reorder timeout (in millisecs). If 0, don't try to reorder.
=========   ===========================================================

In addition to the standard PPP ioctls, a PPPIOCGL2TPSTATS is provided
to retrieve tunnel and session statistics from the kernel using the
PPPoX socket of the appropriate tunnel or session.

Sample userspace code:

  - Create session PPPoX data socket::

        struct sockaddr_pppol2tp sax;
        int fd;

        /* Note, the tunnel socket must be bound already, else it
         * will not be ready
         */
        sax.sa_family = AF_PPPOX;
        sax.sa_protocol = PX_PROTO_OL2TP;
        sax.pppol2tp.fd = tunnel_fd;
        sax.pppol2tp.addr.sin_addr.s_addr = addr->sin_addr.s_addr;
        sax.pppol2tp.addr.sin_port = addr->sin_port;
        sax.pppol2tp.addr.sin_family = AF_INET;
        sax.pppol2tp.s_tunnel  = tunnel_id;
        sax.pppol2tp.s_session = session_id;
        sax.pppol2tp.d_tunnel  = peer_tunnel_id;
        sax.pppol2tp.d_session = peer_session_id;

        /* session_fd is the fd of the session's PPPoL2TP socket.
         * tunnel_fd is the fd of the tunnel UDP / L2TPIP socket.
         */
        fd = connect(session_fd, (struct sockaddr *)&sax, sizeof(sax));
        if (fd < 0 ) {
                return -errno;
        }
        return 0;

Old L2TPv2-only API
-------------------

When L2TP was first added to the Linux kernel in 2.6.23, it
implemented only L2TPv2 and did not include a netlink API. Instead,
tunnel and session instances in the kernel were managed directly using
only PPPoL2TP sockets. The PPPoL2TP socket is used as described in
section "PPPoL2TP Session Socket API" but tunnel and session instances
are automatically created on a connect() of the socket instead of
being created by a separate netlink request:

    - Tunnels are managed using a tunnel management socket which is a
      dedicated PPPoL2TP socket, connected to (invalid) session
      id 0. The L2TP tunnel instance is created when the PPPoL2TP
      tunnel management socket is connected and is destroyed when the
      socket is closed.

    - Session instances are created in the kernel when a PPPoL2TP
      socket is connected to a non-zero session id. Session parameters
      are set using setsockopt. The L2TP session instance is destroyed
      when the socket is closed.

This API is still supported but its use is discouraged. Instead, new
L2TPv2 applications should use netlink to first create the tunnel and
session, then create a PPPoL2TP socket for the session.

Unmanaged L2TPv3 tunnels
------------------------

The kernel L2TP subsystem also supports static (unmanaged) L2TPv3
tunnels. Unmanaged tunnels have no userspace tunnel socket, and
exchange no control messages with the peer to set up the tunnel; the
tunnel is configured manually at each end of the tunnel. All
configuration is done using netlink. There is no need for an L2TP
userspace application in this case -- the tunnel socket is created by
the kernel and configured using parameters sent in the
``L2TP_CMD_TUNNEL_CREATE`` netlink request. The ``ip`` utility of
``iproute2`` has commands for managing static L2TPv3 tunnels; do ``ip
l2tp help`` for more information.

Debugging
---------

The L2TP subsystem offers a range of debugging interfaces through the
debugfs filesystem.

To access these interfaces, the debugfs filesystem must first be mounted::

    # mount -t debugfs debugfs /debug

Files under the l2tp directory can then be accessed, providing a summary
of the current population of tunnel and session contexts existing in the
kernel::

    # cat /debug/l2tp/tunnels

The debugfs files should not be used by applications to obtain L2TP
state information because the file format is subject to change. It is
implemented to provide extra debug information to help diagnose
problems. Applications should instead use the netlink API.

In addition the L2TP subsystem implements tracepoints using the standard
kernel event tracing API.  The available L2TP events can be reviewed as
follows::

    # find /debug/tracing/events/l2tp

Finally, /proc/net/pppol2tp is also provided for backwards compatibility
with the original pppol2tp code. It lists information about L2TPv2
tunnels and sessions only. Its use is discouraged.

Internal Implementation
=======================

This section is for kernel developers and maintainers.

Sockets
-------

UDP sockets are implemented by the networking core. When an L2TP
tunnel is created using a UDP socket, the socket is set up as an
encapsulated UDP socket by setting encap_rcv and encap_destroy
callbacks on the UDP socket. l2tp_udp_encap_recv is called when
packets are received on the socket. l2tp_udp_encap_destroy is called
when userspace closes the socket.

L2TPIP sockets are implemented in `net/l2tp/l2tp_ip.c`_ and
`net/l2tp/l2tp_ip6.c`_.

Tunnels
-------

The kernel keeps a struct l2tp_tunnel context per L2TP tunnel. The
l2tp_tunnel is always associated with a UDP or L2TP/IP socket and
keeps a list of sessions in the tunnel. When a tunnel is first
registered with L2TP core, the reference count on the socket is
increased. This ensures that the socket cannot be removed while L2TP's
data structures reference it.

Tunnels are identified by a unique tunnel id. The id is 16-bit for
L2TPv2 and 32-bit for L2TPv3. Internally, the id is stored as a 32-bit
value.

Tunnels are kept in a per-net list, indexed by tunnel id. The tunnel
id namespace is shared by L2TPv2 and L2TPv3. The tunnel context can be
derived from the socket's sk_user_data.

Handling tunnel socket close is perhaps the most tricky part of the
L2TP implementation. If userspace closes a tunnel socket, the L2TP
tunnel and all of its sessions must be closed and destroyed. Since the
tunnel context holds a ref on the tunnel socket, the socket's
sk_destruct won't be called until the tunnel sock_put's its
socket. For UDP sockets, when userspace closes the tunnel socket, the
socket's encap_destroy handler is invoked, which L2TP uses to initiate
its tunnel close actions. For L2TPIP sockets, the socket's close
handler initiates the same tunnel close actions. All sessions are
first closed. Each session drops its tunnel ref. When the tunnel ref
reaches zero, the tunnel puts its socket ref. When the socket is
eventually destroyed, its sk_destruct finally frees the L2TP tunnel
context.

Sessions
--------

The kernel keeps a struct l2tp_session context for each session.  Each
session has private data which is used for data specific to the
session type. With L2TPv2, the session always carries PPP
traffic. With L2TPv3, the session can carry Ethernet frames (Ethernet
pseudowire) or other data types such as PPP, ATM, HDLC or Frame
Relay. Linux currently implements only Ethernet and PPP session types.

Some L2TP session types also have a socket (PPP pseudowires) while
others do not (Ethernet pseudowires). We can't therefore use the
socket reference count as the reference count for session
contexts. The L2TP implementation therefore has its own internal
reference counts on the session contexts.

Like tunnels, L2TP sessions are identified by a unique
session id. Just as with tunnel ids, the session id is 16-bit for
L2TPv2 and 32-bit for L2TPv3. Internally, the id is stored as a 32-bit
value.

Sessions hold a ref on their parent tunnel to ensure that the tunnel
stays extant while one or more sessions references it.

Sessions are kept in a per-tunnel list, indexed by session id. L2TPv3
sessions are also kept in a per-net list indexed by session id,
because L2TPv3 session ids are unique across all tunnels and L2TPv3
data packets do not contain a tunnel id in the header. This list is
therefore needed to find the session context associated with a
received data packet when the tunnel context cannot be derived from
the tunnel socket.

Although the L2TPv3 RFC specifies that L2TPv3 session ids are not
scoped by the tunnel, the kernel does not police this for L2TPv3 UDP
tunnels and does not add sessions of L2TPv3 UDP tunnels into the
per-net session list. In the UDP receive code, we must trust that the
tunnel can be identified using the tunnel socket's sk_user_data and
lookup the session in the tunnel's session list instead of the per-net
session list.

PPP
---

`net/l2tp/l2tp_ppp.c`_ implements the PPPoL2TP socket family. Each PPP
session has a PPPoL2TP socket.

The PPPoL2TP socket's sk_user_data references the l2tp_session.

Userspace sends and receives PPP packets over L2TP using a PPPoL2TP
socket. Only PPP control frames pass over this socket: PPP data
packets are handled entirely by the kernel, passing between the L2TP
session and its associated ``pppN`` netdev through the PPP channel
interface of the kernel PPP subsystem.

The L2TP PPP implementation handles the closing of a PPPoL2TP socket
by closing its corresponding L2TP session. This is complicated because
it must consider racing with netlink session create/destroy requests
and pppol2tp_connect trying to reconnect with a session that is in the
process of being closed. Unlike tunnels, PPP sessions do not hold a
ref on their associated socket, so code must be careful to sock_hold
the socket where necessary. For all the details, see commit
3d609342cc04129ff7568e19316ce3d7451a27e8.

Ethernet
--------

`net/l2tp/l2tp_eth.c`_ implements L2TPv3 Ethernet pseudowires. It
manages a netdev for each session.

L2TP Ethernet sessions are created and destroyed by netlink request,
or are destroyed when the tunnel is destroyed. Unlike PPP sessions,
Ethernet sessions do not have an associated socket.

Miscellaneous
=============

RFCs
----

The kernel code implements the datapath features specified in the
following RFCs:

======= =============== ===================================
RFC2661 L2TPv2          https://tools.ietf.org/html/rfc2661
RFC3931 L2TPv3          https://tools.ietf.org/html/rfc3931
RFC4719 L2TPv3 Ethernet https://tools.ietf.org/html/rfc4719
======= =============== ===================================

Implementations
---------------

A number of open source applications use the L2TP kernel subsystem:

============ ==============================================
iproute2     https://github.com/shemminger/iproute2
go-l2tp      https://github.com/katalix/go-l2tp
tunneldigger https://github.com/wlanslovenija/tunneldigger
xl2tpd       https://github.com/xelerance/xl2tpd
============ ==============================================

Limitations
-----------

The current implementation has a number of limitations:

  1) Multiple UDP sockets with the same 5-tuple address cannot be
     used. The kernel's tunnel context is identified using private
     data associated with the socket so it is important that each
     socket is uniquely identified by its address.

  2) Interfacing with openvswitch is not yet implemented. It may be
     useful to map OVS Ethernet and VLAN ports into L2TPv3 tunnels.

  3) VLAN pseudowires are implemented using an ``l2tpethN`` interface
     configured with a VLAN sub-interface. Since L2TPv3 VLAN
     pseudowires carry one and only one VLAN, it may be better to use
     a single netdevice rather than an ``l2tpethN`` and ``l2tpethN``:M
     pair per VLAN session. The netlink attribute
     ``L2TP_ATTR_VLAN_ID`` was added for this, but it was never
     implemented.

Testing
-------

Unmanaged L2TPv3 Ethernet features are tested by the kernel's built-in
selftests. See `tools/testing/selftests/net/l2tp.sh`_.

Another test suite, l2tp-ktest_, covers all
of the L2TP APIs and tunnel/session types. This may be integrated into
the kernel's built-in L2TP selftests in the future.

.. Links
.. _Generic Netlink: generic_netlink.html
.. _libmnl: https://www.netfilter.org/projects/libmnl
.. _include/uapi/linux/l2tp.h: ../../../include/uapi/linux/l2tp.h
.. _include/linux/if_pppol2tp.h: ../../../include/linux/if_pppol2tp.h
.. _net/l2tp/l2tp_ip.c: ../../../net/l2tp/l2tp_ip.c
.. _net/l2tp/l2tp_ip6.c: ../../../net/l2tp/l2tp_ip6.c
.. _net/l2tp/l2tp_ppp.c: ../../../net/l2tp/l2tp_ppp.c
.. _net/l2tp/l2tp_eth.c: ../../../net/l2tp/l2tp_eth.c
.. _tools/testing/selftests/net/l2tp.sh: ../../../tools/testing/selftests/net/l2tp.sh
.. _l2tp-ktest: https://github.com/katalix/l2tp-ktest


code:
.. SPDX-License-Identifier: GPL-2.0

===============================
The Linux LAPB Module Interface
===============================

Version 1.3

Jonathan Naylor 29.12.96

Changed (Henner Eisen, 2000-10-29): int return value for data_indication()

The LAPB module will be a separately compiled module for use by any parts of
the Linux operating system that require a LAPB service. This document
defines the interfaces to, and the services provided by this module. The
term module in this context does not imply that the LAPB module is a
separately loadable module, although it may be. The term module is used in
its more standard meaning.

The interface to the LAPB module consists of functions to the module,
callbacks from the module to indicate important state changes, and
structures for getting and setting information about the module.

Structures
----------

Probably the most important structure is the skbuff structure for holding
received and transmitted data, however it is beyond the scope of this
document.

The two LAPB specific structures are the LAPB initialisation structure and
the LAPB parameter structure. These will be defined in a standard header
file, <linux/lapb.h>. The header file <net/lapb.h> is internal to the LAPB
module and is not for use.

LAPB Initialisation Structure
-----------------------------

This structure is used only once, in the call to lapb_register (see below).
It contains information about the device driver that requires the services
of the LAPB module::

	struct lapb_register_struct {
		void (*connect_confirmation)(int token, int reason);
		void (*connect_indication)(int token, int reason);
		void (*disconnect_confirmation)(int token, int reason);
		void (*disconnect_indication)(int token, int reason);
		int  (*data_indication)(int token, struct sk_buff *skb);
		void (*data_transmit)(int token, struct sk_buff *skb);
	};

Each member of this structure corresponds to a function in the device driver
that is called when a particular event in the LAPB module occurs. These will
be described in detail below. If a callback is not required (!!) then a NULL
may be substituted.

LAPB Parameter Structure
------------------------

This structure is used with the lapb_getparms and lapb_setparms functions
(see below). They are used to allow the device driver to get and set the
operational parameters of the LAPB implementation for a given connection::

	struct lapb_parms_struct {
		unsigned int t1;
		unsigned int t1timer;
		unsigned int t2;
		unsigned int t2timer;
		unsigned int n2;
		unsigned int n2count;
		unsigned int window;
		unsigned int state;
		unsigned int mode;
	};

T1 and T2 are protocol timing parameters and are given in units of 100ms. N2
is the maximum number of tries on the link before it is declared a failure.
The window size is the maximum number of outstanding data packets allowed to
be unacknowledged by the remote end, the value of the window is between 1
and 7 for a standard LAPB link, and between 1 and 127 for an extended LAPB
link.

The mode variable is a bit field used for setting (at present) three values.
The bit fields have the following meanings:

======  =================================================
Bit	Meaning
======  =================================================
0	LAPB operation (0=LAPB_STANDARD 1=LAPB_EXTENDED).
1	[SM]LP operation (0=LAPB_SLP 1=LAPB=MLP).
2	DTE/DCE operation (0=LAPB_DTE 1=LAPB_DCE)
3-31	Reserved, must be 0.
======  =================================================

Extended LAPB operation indicates the use of extended sequence numbers and
consequently larger window sizes, the default is standard LAPB operation.
MLP operation is the same as SLP operation except that the addresses used by
LAPB are different to indicate the mode of operation, the default is Single
Link Procedure. The difference between DCE and DTE operation is (i) the
addresses used for commands and responses, and (ii) when the DCE is not
connected, it sends DM without polls set, every T1. The upper case constant
names will be defined in the public LAPB header file.

Functions
---------

The LAPB module provides a number of function entry points.

::

    int lapb_register(void *token, struct lapb_register_struct);

This must be called before the LAPB module may be used. If the call is
successful then LAPB_OK is returned. The token must be a unique identifier
generated by the device driver to allow for the unique identification of the
instance of the LAPB link. It is returned by the LAPB module in all of the
callbacks, and is used by the device driver in all calls to the LAPB module.
For multiple LAPB links in a single device driver, multiple calls to
lapb_register must be made. The format of the lapb_register_struct is given
above. The return values are:

=============		=============================
LAPB_OK			LAPB registered successfully.
LAPB_BADTOKEN		Token is already registered.
LAPB_NOMEM		Out of memory
=============		=============================

::

    int lapb_unregister(void *token);

This releases all the resources associated with a LAPB link. Any current
LAPB link will be abandoned without further messages being passed. After
this call, the value of token is no longer valid for any calls to the LAPB
function. The valid return values are:

=============		===============================
LAPB_OK			LAPB unregistered successfully.
LAPB_BADTOKEN		Invalid/unknown LAPB token.
=============		===============================

::

    int lapb_getparms(void *token, struct lapb_parms_struct *parms);

This allows the device driver to get the values of the current LAPB
variables, the lapb_parms_struct is described above. The valid return values
are:

=============		=============================
LAPB_OK			LAPB getparms was successful.
LAPB_BADTOKEN		Invalid/unknown LAPB token.
=============		=============================

::

    int lapb_setparms(void *token, struct lapb_parms_struct *parms);

This allows the device driver to set the values of the current LAPB
variables, the lapb_parms_struct is described above. The values of t1timer,
t2timer and n2count are ignored, likewise changing the mode bits when
connected will be ignored. An error implies that none of the values have
been changed. The valid return values are:

=============		=================================================
LAPB_OK			LAPB getparms was successful.
LAPB_BADTOKEN		Invalid/unknown LAPB token.
LAPB_INVALUE		One of the values was out of its allowable range.
=============		=================================================

::

    int lapb_connect_request(void *token);

Initiate a connect using the current parameter settings. The valid return
values are:

==============		=================================
LAPB_OK			LAPB is starting to connect.
LAPB_BADTOKEN		Invalid/unknown LAPB token.
LAPB_CONNECTED		LAPB module is already connected.
==============		=================================

::

    int lapb_disconnect_request(void *token);

Initiate a disconnect. The valid return values are:

=================	===============================
LAPB_OK			LAPB is starting to disconnect.
LAPB_BADTOKEN		Invalid/unknown LAPB token.
LAPB_NOTCONNECTED	LAPB module is not connected.
=================	===============================

::

    int lapb_data_request(void *token, struct sk_buff *skb);

Queue data with the LAPB module for transmitting over the link. If the call
is successful then the skbuff is owned by the LAPB module and may not be
used by the device driver again. The valid return values are:

=================	=============================
LAPB_OK			LAPB has accepted the data.
LAPB_BADTOKEN		Invalid/unknown LAPB token.
LAPB_NOTCONNECTED	LAPB module is not connected.
=================	=============================

::

    int lapb_data_received(void *token, struct sk_buff *skb);

Queue data with the LAPB module which has been received from the device. It
is expected that the data passed to the LAPB module has skb->data pointing
to the beginning of the LAPB data. If the call is successful then the skbuff
is owned by the LAPB module and may not be used by the device driver again.
The valid return values are:

=============		===========================
LAPB_OK			LAPB has accepted the data.
LAPB_BADTOKEN		Invalid/unknown LAPB token.
=============		===========================

Callbacks
---------

These callbacks are functions provided by the device driver for the LAPB
module to call when an event occurs. They are registered with the LAPB
module with lapb_register (see above) in the structure lapb_register_struct
(see above).

::

    void (*connect_confirmation)(void *token, int reason);

This is called by the LAPB module when a connection is established after
being requested by a call to lapb_connect_request (see above). The reason is
always LAPB_OK.

::

    void (*connect_indication)(void *token, int reason);

This is called by the LAPB module when the link is established by the remote
system. The value of reason is always LAPB_OK.

::

    void (*disconnect_confirmation)(void *token, int reason);

This is called by the LAPB module when an event occurs after the device
driver has called lapb_disconnect_request (see above). The reason indicates
what has happened. In all cases the LAPB link can be regarded as being
terminated. The values for reason are:

=================	====================================================
LAPB_OK			The LAPB link was terminated normally.
LAPB_NOTCONNECTED	The remote system was not connected.
LAPB_TIMEDOUT		No response was received in N2 tries from the remote
			system.
=================	====================================================

::

    void (*disconnect_indication)(void *token, int reason);

This is called by the LAPB module when the link is terminated by the remote
system or another event has occurred to terminate the link. This may be
returned in response to a lapb_connect_request (see above) if the remote
system refused the request. The values for reason are:

=================	====================================================
LAPB_OK			The LAPB link was terminated normally by the remote
			system.
LAPB_REFUSED		The remote system refused the connect request.
LAPB_NOTCONNECTED	The remote system was not connected.
LAPB_TIMEDOUT		No response was received in N2 tries from the remote
			system.
=================	====================================================

::

    int (*data_indication)(void *token, struct sk_buff *skb);

This is called by the LAPB module when data has been received from the
remote system that should be passed onto the next layer in the protocol
stack. The skbuff becomes the property of the device driver and the LAPB
module will not perform any more actions on it. The skb->data pointer will
be pointing to the first byte of data after the LAPB header.

This method should return NET_RX_DROP (as defined in the header
file include/linux/netdevice.h) if and only if the frame was dropped
before it could be delivered to the upper layer.

::

    void (*data_transmit)(void *token, struct sk_buff *skb);

This is called by the LAPB module when data is to be transmitted to the
remote system by the device driver. The skbuff becomes the property of the
device driver and the LAPB module will not perform any more actions on it.
The skb->data pointer will be pointing to the first byte of the LAPB header.


code:
#
# This outlines the Linux authentication/association and
# deauthentication/disassociation flows.
#
# This can be converted into a diagram using the service
# at http://www.websequencediagrams.com/
#

participant userspace
participant mac80211
participant driver

alt authentication needed (not FT)
userspace->mac80211: authenticate

alt authenticated/authenticating already
mac80211->driver: sta_state(AP, not-exists)
mac80211->driver: bss_info_changed(clear BSSID)
else associated
note over mac80211,driver
like deauth/disassoc, without sending the
BA session stop & deauth/disassoc frames
end note
end

mac80211->driver: config(channel, channel type)
mac80211->driver: bss_info_changed(set BSSID, basic rate bitmap)
mac80211->driver: sta_state(AP, exists)

alt no probe request data known
mac80211->driver: TX directed probe request
driver->mac80211: RX probe response
end

mac80211->driver: TX auth frame
driver->mac80211: RX auth frame

alt WEP shared key auth
mac80211->driver: TX auth frame
driver->mac80211: RX auth frame
end

mac80211->driver: sta_state(AP, authenticated)
mac80211->userspace: RX auth frame

end

userspace->mac80211: associate
alt authenticated or associated
note over mac80211,driver: cleanup like for authenticate
end

alt not previously authenticated (FT)
mac80211->driver: config(channel, channel type)
mac80211->driver: bss_info_changed(set BSSID, basic rate bitmap)
mac80211->driver: sta_state(AP, exists)
mac80211->driver: sta_state(AP, authenticated)
end
mac80211->driver: TX assoc
driver->mac80211: RX assoc response
note over mac80211: init rate control
mac80211->driver: sta_state(AP, associated)

alt not using WPA
mac80211->driver: sta_state(AP, authorized)
end

mac80211->driver: set up QoS parameters

mac80211->driver: bss_info_changed(QoS, HT, associated with AID)
mac80211->userspace: associated

note left of userspace: associated now

alt using WPA
note over userspace
do 4-way-handshake
(data frames)
end note
userspace->mac80211: authorized
mac80211->driver: sta_state(AP, authorized)
end

userspace->mac80211: deauthenticate/disassociate
mac80211->driver: stop BA sessions
mac80211->driver: TX deauth/disassoc
mac80211->driver: flush frames
mac80211->driver: sta_state(AP,associated)
mac80211->driver: sta_state(AP,authenticated)
mac80211->driver: sta_state(AP,exists)
mac80211->driver: sta_state(AP,not-exists)
mac80211->driver: turn off powersave
mac80211->driver: bss_info_changed(clear BSSID, not associated, no QoS, ...)
mac80211->driver: config(channel type to non-HT)
mac80211->userspace: disconnected