code:
.. _development_coding:

Getting the code right
======================

While there is much to be said for a solid and community-oriented design
process, the proof of any kernel development project is in the resulting
code.  It is the code which will be examined by other developers and merged
(or not) into the mainline tree.  So it is the quality of this code which
will determine the ultimate success of the project.

This section will examine the coding process.  We'll start with a look at a
number of ways in which kernel developers can go wrong.  Then the focus
will shift toward doing things right and the tools which can help in that
quest.

Pitfalls
---------

Coding style
************

The kernel has long had a standard coding style, described in
:ref:`Documentation/process/coding-style.rst <codingstyle>`.  For much of
that time, the policies described in that file were taken as being, at most,
advisory.  As a result, there is a substantial amount of code in the kernel
which does not meet the coding style guidelines.  The presence of that code
leads to two independent hazards for kernel developers.

The first of these is to believe that the kernel coding standards do not
matter and are not enforced.  The truth of the matter is that adding new
code to the kernel is very difficult if that code is not coded according to
the standard; many developers will request that the code be reformatted
before they will even review it.  A code base as large as the kernel
requires some uniformity of code to make it possible for developers to
quickly understand any part of it.  So there is no longer room for
strangely-formatted code.

Occasionally, the kernel's coding style will run into conflict with an
employer's mandated style.  In such cases, the kernel's style will have to
win before the code can be merged.  Putting code into the kernel means
giving up a degree of control in a number of ways - including control over
how the code is formatted.

The other trap is to assume that code which is already in the kernel is
urgently in need of coding style fixes.  Developers may start to generate
reformatting patches as a way of gaining familiarity with the process, or
as a way of getting their name into the kernel changelogs - or both.  But
pure coding style fixes are seen as noise by the development community;
they tend to get a chilly reception.  So this type of patch is best
avoided.  It is natural to fix the style of a piece of code while working
on it for other reasons, but coding style changes should not be made for
their own sake.

The coding style document also should not be read as an absolute law which
can never be transgressed.  If there is a good reason to go against the
style (a line which becomes far less readable if split to fit within the
80-column limit, for example), just do it.

Note that you can also use the ``clang-format`` tool to help you with
these rules, to quickly re-format parts of your code automatically,
and to review full files in order to spot coding style mistakes,
typos and possible improvements. It is also handy for sorting ``#includes``,
for aligning variables/macros, for reflowing text and other similar tasks.
See the file :ref:`Documentation/process/clang-format.rst <clangformat>`
for more details.

Abstraction layers
******************

Computer Science professors teach students to make extensive use of
abstraction layers in the name of flexibility and information hiding.
Certainly the kernel makes extensive use of abstraction; no project
involving several million lines of code could do otherwise and survive.
But experience has shown that excessive or premature abstraction can be
just as harmful as premature optimization.  Abstraction should be used to
the level required and no further.

At a simple level, consider a function which has an argument which is
always passed as zero by all callers.  One could retain that argument just
in case somebody eventually needs to use the extra flexibility that it
provides.  By that time, though, chances are good that the code which
implements this extra argument has been broken in some subtle way which was
never noticed - because it has never been used.  Or, when the need for
extra flexibility arises, it does not do so in a way which matches the
programmer's early expectation.  Kernel developers will routinely submit
patches to remove unused arguments; they should, in general, not be added
in the first place.

Abstraction layers which hide access to hardware - often to allow the bulk
of a driver to be used with multiple operating systems - are especially
frowned upon.  Such layers obscure the code and may impose a performance
penalty; they do not belong in the Linux kernel.

On the other hand, if you find yourself copying significant amounts of code
from another kernel subsystem, it is time to ask whether it would, in fact,
make sense to pull out some of that code into a separate library or to
implement that functionality at a higher level.  There is no value in
replicating the same code throughout the kernel.

#ifdef and preprocessor use in general
**************************************

The C preprocessor seems to present a powerful temptation to some C
programmers, who see it as a way to efficiently encode a great deal of
flexibility into a source file.  But the preprocessor is not C, and heavy
use of it results in code which is much harder for others to read and
harder for the compiler to check for correctness.  Heavy preprocessor use
is almost always a sign of code which needs some cleanup work.

Conditional compilation with #ifdef is, indeed, a powerful feature, and it
is used within the kernel.  But there is little desire to see code which is
sprinkled liberally with #ifdef blocks.  As a general rule, #ifdef use
should be confined to header files whenever possible.
Conditionally-compiled code can be confined to functions which, if the code
is not to be present, simply become empty.  The compiler will then quietly
optimize out the call to the empty function.  The result is far cleaner
code which is easier to follow.

C preprocessor macros present a number of hazards, including possible
multiple evaluation of expressions with side effects and no type safety.
If you are tempted to define a macro, consider creating an inline function
instead.  The code which results will be the same, but inline functions are
easier to read, do not evaluate their arguments multiple times, and allow
the compiler to perform type checking on the arguments and return value.

Inline functions
****************

Inline functions present a hazard of their own, though.  Programmers can
become enamored of the perceived efficiency inherent in avoiding a function
call and fill a source file with inline functions.  Those functions,
however, can actually reduce performance.  Since their code is replicated
at each call site, they end up bloating the size of the compiled kernel.
That, in turn, creates pressure on the processor's memory caches, which can
slow execution dramatically.  Inline functions, as a rule, should be quite
small and relatively rare.  The cost of a function call, after all, is not
that high; the creation of large numbers of inline functions is a classic
example of premature optimization.

In general, kernel programmers ignore cache effects at their peril.  The
classic time/space tradeoff taught in beginning data structures classes
often does not apply to contemporary hardware.  Space *is* time, in that a
larger program will run slower than one which is more compact.

More recent compilers take an increasingly active role in deciding whether
a given function should actually be inlined or not.  So the liberal
placement of "inline" keywords may not just be excessive; it could also be
irrelevant.

Locking
*******

In May, 2006, the "Devicescape" networking stack was, with great
fanfare, released under the GPL and made available for inclusion in the
mainline kernel.  This donation was welcome news; support for wireless
networking in Linux was considered substandard at best, and the Devicescape
stack offered the promise of fixing that situation.  Yet, this code did not
actually make it into the mainline until June, 2007 (2.6.22).  What
happened?

This code showed a number of signs of having been developed behind
corporate doors.  But one large problem in particular was that it was not
designed to work on multiprocessor systems.  Before this networking stack
(now called mac80211) could be merged, a locking scheme needed to be
retrofitted onto it.

Once upon a time, Linux kernel code could be developed without thinking
about the concurrency issues presented by multiprocessor systems.  Now,
however, this document is being written on a dual-core laptop.  Even on
single-processor systems, work being done to improve responsiveness will
raise the level of concurrency within the kernel.  The days when kernel
code could be written without thinking about locking are long past.

Any resource (data structures, hardware registers, etc.) which could be
accessed concurrently by more than one thread must be protected by a lock.
New code should be written with this requirement in mind; retrofitting
locking after the fact is a rather more difficult task.  Kernel developers
should take the time to understand the available locking primitives well
enough to pick the right tool for the job.  Code which shows a lack of
attention to concurrency will have a difficult path into the mainline.

Regressions
***********

One final hazard worth mentioning is this: it can be tempting to make a
change (which may bring big improvements) which causes something to break
for existing users.  This kind of change is called a "regression," and
regressions have become most unwelcome in the mainline kernel.  With few
exceptions, changes which cause regressions will be backed out if the
regression cannot be fixed in a timely manner.  Far better to avoid the
regression in the first place.

It is often argued that a regression can be justified if it causes things
to work for more people than it creates problems for.  Why not make a
change if it brings new functionality to ten systems for each one it
breaks?  The best answer to this question was expressed by Linus in July,
2007:

::

	So we don't fix bugs by introducing new problems.  That way lies
	madness, and nobody ever knows if you actually make any real
	progress at all. Is it two steps forwards, one step back, or one
	step forward and two steps back?

(https://lwn.net/Articles/243460/).

An especially unwelcome type of regression is any sort of change to the
user-space ABI.  Once an interface has been exported to user space, it must
be supported indefinitely.  This fact makes the creation of user-space
interfaces particularly challenging: since they cannot be changed in
incompatible ways, they must be done right the first time.  For this
reason, a great deal of thought, clear documentation, and wide review for
user-space interfaces is always required.

Code checking tools
-------------------

For now, at least, the writing of error-free code remains an ideal that few
of us can reach.  What we can hope to do, though, is to catch and fix as
many of those errors as possible before our code goes into the mainline
kernel.  To that end, the kernel developers have put together an impressive
array of tools which can catch a wide variety of obscure problems in an
automated way.  Any problem caught by the computer is a problem which will
not afflict a user later on, so it stands to reason that the automated
tools should be used whenever possible.

The first step is simply to heed the warnings produced by the compiler.
Contemporary versions of gcc can detect (and warn about) a large number of
potential errors.  Quite often, these warnings point to real problems.
Code submitted for review should, as a rule, not produce any compiler
warnings.  When silencing warnings, take care to understand the real cause
and try to avoid "fixes" which make the warning go away without addressing
its cause.

Note that not all compiler warnings are enabled by default.  Build the
kernel with "make KCFLAGS=-W" to get the full set.

The kernel provides several configuration options which turn on debugging
features; most of these are found in the "kernel hacking" submenu.  Several
of these options should be turned on for any kernel used for development or
testing purposes.  In particular, you should turn on:

 - FRAME_WARN to get warnings for stack frames larger than a given amount.
   The output generated can be verbose, but one need not worry about
   warnings from other parts of the kernel.

 - DEBUG_OBJECTS will add code to track the lifetime of various objects
   created by the kernel and warn when things are done out of order.  If
   you are adding a subsystem which creates (and exports) complex objects
   of its own, consider adding support for the object debugging
   infrastructure.

 - DEBUG_SLAB can find a variety of memory allocation and use errors; it
   should be used on most development kernels.

 - DEBUG_SPINLOCK, DEBUG_ATOMIC_SLEEP, and DEBUG_MUTEXES will find a
   number of common locking errors.

There are quite a few other debugging options, some of which will be
discussed below.  Some of them have a significant performance impact and
should not be used all of the time.  But some time spent learning the
available options will likely be paid back many times over in short order.

One of the heavier debugging tools is the locking checker, or "lockdep."
This tool will track the acquisition and release of every lock (spinlock or
mutex) in the system, the order in which locks are acquired relative to
each other, the current interrupt environment, and more.  It can then
ensure that locks are always acquired in the same order, that the same
interrupt assumptions apply in all situations, and so on.  In other words,
lockdep can find a number of scenarios in which the system could, on rare
occasion, deadlock.  This kind of problem can be painful (for both
developers and users) in a deployed system; lockdep allows them to be found
in an automated manner ahead of time.  Code with any sort of non-trivial
locking should be run with lockdep enabled before being submitted for
inclusion.

As a diligent kernel programmer, you will, beyond doubt, check the return
status of any operation (such as a memory allocation) which can fail.  The
fact of the matter, though, is that the resulting failure recovery paths
are, probably, completely untested.  Untested code tends to be broken code;
you could be much more confident of your code if all those error-handling
paths had been exercised a few times.

The kernel provides a fault injection framework which can do exactly that,
especially where memory allocations are involved.  With fault injection
enabled, a configurable percentage of memory allocations will be made to
fail; these failures can be restricted to a specific range of code.
Running with fault injection enabled allows the programmer to see how the
code responds when things go badly.  See
Documentation/fault-injection/fault-injection.rst for more information on
how to use this facility.

Other kinds of errors can be found with the "sparse" static analysis tool.
With sparse, the programmer can be warned about confusion between
user-space and kernel-space addresses, mixture of big-endian and
small-endian quantities, the passing of integer values where a set of bit
flags is expected, and so on.  Sparse must be installed separately (it can
be found at https://sparse.wiki.kernel.org/index.php/Main_Page if your
distributor does not package it); it can then be run on the code by adding
"C=1" to your make command.

The "Coccinelle" tool (http://coccinelle.lip6.fr/) is able to find a wide
variety of potential coding problems; it can also propose fixes for those
problems.  Quite a few "semantic patches" for the kernel have been packaged
under the scripts/coccinelle directory; running "make coccicheck" will run
through those semantic patches and report on any problems found.  See
:ref:`Documentation/dev-tools/coccinelle.rst <devtools_coccinelle>`
for more information.

Other kinds of portability errors are best found by compiling your code for
other architectures.  If you do not happen to have an S/390 system or a
Blackfin development board handy, you can still perform the compilation
step.  A large set of cross compilers for x86 systems can be found at

	https://www.kernel.org/pub/tools/crosstool/

Some time spent installing and using these compilers will help avoid
embarrassment later.

Documentation
-------------

Documentation has often been more the exception than the rule with kernel
development.  Even so, adequate documentation will help to ease the merging
of new code into the kernel, make life easier for other developers, and
will be helpful for your users.  In many cases, the addition of
documentation has become essentially mandatory.

The first piece of documentation for any patch is its associated
changelog.  Log entries should describe the problem being solved, the form
of the solution, the people who worked on the patch, any relevant
effects on performance, and anything else that might be needed to
understand the patch.  Be sure that the changelog says *why* the patch is
worth applying; a surprising number of developers fail to provide that
information.

Any code which adds a new user-space interface - including new sysfs or
/proc files - should include documentation of that interface which enables
user-space developers to know what they are working with.  See
Documentation/ABI/README for a description of how this documentation should
be formatted and what information needs to be provided.

The file :ref:`Documentation/admin-guide/kernel-parameters.rst
<kernelparameters>` describes all of the kernel's boot-time parameters.
Any patch which adds new parameters should add the appropriate entries to
this file.

Any new configuration options must be accompanied by help text which
clearly explains the options and when the user might want to select them.

Internal API information for many subsystems is documented by way of
specially-formatted comments; these comments can be extracted and formatted
in a number of ways by the "kernel-doc" script.  If you are working within
a subsystem which has kerneldoc comments, you should maintain them and add
them, as appropriate, for externally-available functions.  Even in areas
which have not been so documented, there is no harm in adding kerneldoc
comments for the future; indeed, this can be a useful activity for
beginning kernel developers.  The format of these comments, along with some
information on how to create kerneldoc templates can be found at
:ref:`Documentation/doc-guide/ <doc_guide>`.

Anybody who reads through a significant amount of existing kernel code will
note that, often, comments are most notable by their absence.  Once again,
the expectations for new code are higher than they were in the past;
merging uncommented code will be harder.  That said, there is little desire
for verbosely-commented code.  The code should, itself, be readable, with
comments explaining the more subtle aspects.

Certain things should always be commented.  Uses of memory barriers should
be accompanied by a line explaining why the barrier is necessary.  The
locking rules for data structures generally need to be explained somewhere.
Major data structures need comprehensive documentation in general.
Non-obvious dependencies between separate bits of code should be pointed
out.  Anything which might tempt a code janitor to make an incorrect
"cleanup" needs a comment saying why it is done the way it is.  And so on.

Internal API changes
--------------------

The binary interface provided by the kernel to user space cannot be broken
except under the most severe circumstances.  The kernel's internal
programming interfaces, instead, are highly fluid and can be changed when
the need arises.  If you find yourself having to work around a kernel API,
or simply not using a specific functionality because it does not meet your
needs, that may be a sign that the API needs to change.  As a kernel
developer, you are empowered to make such changes.

There are, of course, some catches.  API changes can be made, but they need
to be well justified.  So any patch making an internal API change should be
accompanied by a description of what the change is and why it is
necessary.  This kind of change should also be broken out into a separate
patch, rather than buried within a larger patch.

The other catch is that a developer who changes an internal API is
generally charged with the task of fixing any code within the kernel tree
which is broken by the change.  For a widely-used function, this duty can
lead to literally hundreds or thousands of changes - many of which are
likely to conflict with work being done by other developers.  Needless to
say, this can be a large job, so it is best to be sure that the
justification is solid.  Note that the Coccinelle tool can help with
wide-ranging API changes.

When making an incompatible API change, one should, whenever possible,
ensure that code which has not been updated is caught by the compiler.
This will help you to be sure that you have found all in-tree uses of that
interface.  It will also alert developers of out-of-tree code that there is
a change that they need to respond to.  Supporting out-of-tree code is not
something that kernel developers need to be worried about, but we also do
not have to make life harder for out-of-tree developers than it needs to
be.


code:
.. _development_posting:

Posting patches
===============

Sooner or later, the time comes when your work is ready to be presented to
the community for review and, eventually, inclusion into the mainline
kernel.  Unsurprisingly, the kernel development community has evolved a set
of conventions and procedures which are used in the posting of patches;
following them will make life much easier for everybody involved.  This
document will attempt to cover these expectations in reasonable detail;
more information can also be found in the files
:ref:`Documentation/process/submitting-patches.rst <submittingpatches>`
and :ref:`Documentation/process/submit-checklist.rst <submitchecklist>`.

When to post
------------

There is a constant temptation to avoid posting patches before they are
completely "ready."  For simple patches, that is not a problem.  If the
work being done is complex, though, there is a lot to be gained by getting
feedback from the community before the work is complete.  So you should
consider posting in-progress work, or even making a git tree available so
that interested developers can catch up with your work at any time.

When posting code which is not yet considered ready for inclusion, it is a
good idea to say so in the posting itself.  Also mention any major work
which remains to be done and any known problems.  Fewer people will look at
patches which are known to be half-baked, but those who do will come in
with the idea that they can help you drive the work in the right direction.

Before creating patches
-----------------------

There are a number of things which should be done before you consider
sending patches to the development community.  These include:

 - Test the code to the extent that you can.  Make use of the kernel's
   debugging tools, ensure that the kernel will build with all reasonable
   combinations of configuration options, use cross-compilers to build for
   different architectures, etc.

 - Make sure your code is compliant with the kernel coding style
   guidelines.

 - Does your change have performance implications?  If so, you should run
   benchmarks showing what the impact (or benefit) of your change is; a
   summary of the results should be included with the patch.

 - Be sure that you have the right to post the code.  If this work was done
   for an employer, the employer likely has a right to the work and must be
   agreeable with its release under the GPL.

As a general rule, putting in some extra thought before posting code almost
always pays back the effort in short order.

Patch preparation
-----------------

The preparation of patches for posting can be a surprising amount of work,
but, once again, attempting to save time here is not generally advisable
even in the short term.

Patches must be prepared against a specific version of the kernel.  As a
general rule, a patch should be based on the current mainline as found in
Linus's git tree.  When basing on mainline, start with a well-known release
point - a stable or -rc release - rather than branching off the mainline at
an arbitrary spot.

It may become necessary to make versions against -mm, linux-next, or a
subsystem tree, though, to facilitate wider testing and review.  Depending
on the area of your patch and what is going on elsewhere, basing a patch
against these other trees can require a significant amount of work
resolving conflicts and dealing with API changes.

Only the most simple changes should be formatted as a single patch;
everything else should be made as a logical series of changes.  Splitting
up patches is a bit of an art; some developers spend a long time figuring
out how to do it in the way that the community expects.  There are a few
rules of thumb, however, which can help considerably:

 - The patch series you post will almost certainly not be the series of
   changes found in your working revision control system.  Instead, the
   changes you have made need to be considered in their final form, then
   split apart in ways which make sense.  The developers are interested in
   discrete, self-contained changes, not the path you took to get to those
   changes.

 - Each logically independent change should be formatted as a separate
   patch.  These changes can be small ("add a field to this structure") or
   large (adding a significant new driver, for example), but they should be
   conceptually small and amenable to a one-line description.  Each patch
   should make a specific change which can be reviewed on its own and
   verified to do what it says it does.

 - As a way of restating the guideline above: do not mix different types of
   changes in the same patch.  If a single patch fixes a critical security
   bug, rearranges a few structures, and reformats the code, there is a
   good chance that it will be passed over and the important fix will be
   lost.

 - Each patch should yield a kernel which builds and runs properly; if your
   patch series is interrupted in the middle, the result should still be a
   working kernel.  Partial application of a patch series is a common
   scenario when the "git bisect" tool is used to find regressions; if the
   result is a broken kernel, you will make life harder for developers and
   users who are engaging in the noble work of tracking down problems.

 - Do not overdo it, though.  One developer once posted a set of edits
   to a single file as 500 separate patches - an act which did not make him
   the most popular person on the kernel mailing list.  A single patch can
   be reasonably large as long as it still contains a single *logical*
   change.

 - It can be tempting to add a whole new infrastructure with a series of
   patches, but to leave that infrastructure unused until the final patch
   in the series enables the whole thing.  This temptation should be
   avoided if possible; if that series adds regressions, bisection will
   finger the last patch as the one which caused the problem, even though
   the real bug is elsewhere.  Whenever possible, a patch which adds new
   code should make that code active immediately.

Working to create the perfect patch series can be a frustrating process
which takes quite a bit of time and thought after the "real work" has been
done.  When done properly, though, it is time well spent.

Patch formatting and changelogs
-------------------------------

So now you have a perfect series of patches for posting, but the work is
not done quite yet.  Each patch needs to be formatted into a message which
quickly and clearly communicates its purpose to the rest of the world.  To
that end, each patch will be composed of the following:

 - An optional "From" line naming the author of the patch.  This line is
   only necessary if you are passing on somebody else's patch via email,
   but it never hurts to add it when in doubt.

 - A one-line description of what the patch does.  This message should be
   enough for a reader who sees it with no other context to figure out the
   scope of the patch; it is the line that will show up in the "short form"
   changelogs.  This message is usually formatted with the relevant
   subsystem name first, followed by the purpose of the patch.  For
   example:

   ::

	gpio: fix build on CONFIG_GPIO_SYSFS=n

 - A blank line followed by a detailed description of the contents of the
   patch.  This description can be as long as is required; it should say
   what the patch does and why it should be applied to the kernel.

 - One or more tag lines, with, at a minimum, one Signed-off-by: line from
   the author of the patch.  Tags will be described in more detail below.

The items above, together, form the changelog for the patch.  Writing good
changelogs is a crucial but often-neglected art; it's worth spending
another moment discussing this issue.  When writing a changelog, you should
bear in mind that a number of different people will be reading your words.
These include subsystem maintainers and reviewers who need to decide
whether the patch should be included, distributors and other maintainers
trying to decide whether a patch should be backported to other kernels, bug
hunters wondering whether the patch is responsible for a problem they are
chasing, users who want to know how the kernel has changed, and more.  A
good changelog conveys the needed information to all of these people in the
most direct and concise way possible.

To that end, the summary line should describe the effects of and motivation
for the change as well as possible given the one-line constraint.  The
detailed description can then amplify on those topics and provide any
needed additional information.  If the patch fixes a bug, cite the commit
which introduced the bug if possible (and please provide both the commit ID
and the title when citing commits).  If a problem is associated with
specific log or compiler output, include that output to help others
searching for a solution to the same problem.  If the change is meant to
support other changes coming in later patch, say so.  If internal APIs are
changed, detail those changes and how other developers should respond.  In
general, the more you can put yourself into the shoes of everybody who will
be reading your changelog, the better that changelog (and the kernel as a
whole) will be.

Needless to say, the changelog should be the text used when committing the
change to a revision control system.  It will be followed by:

 - The patch itself, in the unified ("-u") patch format.  Using the "-p"
   option to diff will associate function names with changes, making the
   resulting patch easier for others to read.

You should avoid including changes to irrelevant files (those generated by
the build process, for example, or editor backup files) in the patch.  The
file "dontdiff" in the Documentation directory can help in this regard;
pass it to diff with the "-X" option.

The tags already briefly mentioned above are used to provide insights how
the patch came into being. They are described in detail in the
:ref:`Documentation/process/submitting-patches.rst <submittingpatches>`
document; what follows here is a brief summary.

One tag is used to refer to earlier commits which introduced problems fixed by
the patch::

	Fixes: 1f2e3d4c5b6a ("The first line of the commit specified by the first 12 characters of its SHA-1 ID")

Another tag is used for linking web pages with additional backgrounds or
details, for example a report about a bug fixed by the patch or a document
with a specification implemented by the patch::

	Link: https://example.com/somewhere.html  optional-other-stuff

Many maintainers when applying a patch also add this tag to link to the
latest public review posting of the patch; often this is automatically done
by tools like b4 or a git hook like the one described in
'Documentation/maintainer/configure-git.rst'.

A third kind of tag is used to document who was involved in the development of
the patch. Each of these uses this format::

	tag: Full Name <email address>  optional-other-stuff

The tags in common use are:

 - Signed-off-by: this is a developer's certification that he or she has
   the right to submit the patch for inclusion into the kernel.  It is an
   agreement to the Developer's Certificate of Origin, the full text of
   which can be found in :ref:`Documentation/process/submitting-patches.rst <submittingpatches>`
   Code without a proper signoff cannot be merged into the mainline.

 - Co-developed-by: states that the patch was co-created by several developers;
   it is a used to give attribution to co-authors (in addition to the author
   attributed by the From: tag) when multiple people work on a single patch.
   Every Co-developed-by: must be immediately followed by a Signed-off-by: of
   the associated co-author.  Details and examples can be found in
   :ref:`Documentation/process/submitting-patches.rst <submittingpatches>`.

 - Acked-by: indicates an agreement by another developer (often a
   maintainer of the relevant code) that the patch is appropriate for
   inclusion into the kernel.

 - Tested-by: states that the named person has tested the patch and found
   it to work.

 - Reviewed-by: the named developer has reviewed the patch for correctness;
   see the reviewer's statement in :ref:`Documentation/process/submitting-patches.rst <submittingpatches>`
   for more detail.

 - Reported-by: names a user who reported a problem which is fixed by this
   patch; this tag is used to give credit to the (often underappreciated)
   people who test our code and let us know when things do not work
   correctly.

 - Cc: the named person received a copy of the patch and had the
   opportunity to comment on it.

Be careful in the addition of tags to your patches, as only Cc: is appropriate
for addition without the explicit permission of the person named; using
Reported-by: is fine most of the time as well, but ask for permission if
the bug was reported in private.

Sending the patch
-----------------

Before you mail your patches, there are a couple of other things you should
take care of:

 - Are you sure that your mailer will not corrupt the patches?  Patches
   which have had gratuitous white-space changes or line wrapping performed
   by the mail client will not apply at the other end, and often will not
   be examined in any detail.  If there is any doubt at all, mail the patch
   to yourself and convince yourself that it shows up intact.

   :ref:`Documentation/process/email-clients.rst <email_clients>` has some
   helpful hints on making specific mail clients work for sending patches.

 - Are you sure your patch is free of silly mistakes?  You should always
   run patches through scripts/checkpatch.pl and address the complaints it
   comes up with.  Please bear in mind that checkpatch.pl, while being the
   embodiment of a fair amount of thought about what kernel patches should
   look like, is not smarter than you.  If fixing a checkpatch.pl complaint
   would make the code worse, don't do it.

Patches should always be sent as plain text.  Please do not send them as
attachments; that makes it much harder for reviewers to quote sections of
the patch in their replies.  Instead, just put the patch directly into your
message.

When mailing patches, it is important to send copies to anybody who might
be interested in it.  Unlike some other projects, the kernel encourages
people to err on the side of sending too many copies; don't assume that the
relevant people will see your posting on the mailing lists.  In particular,
copies should go to:

 - The maintainer(s) of the affected subsystem(s).  As described earlier,
   the MAINTAINERS file is the first place to look for these people.

 - Other developers who have been working in the same area - especially
   those who might be working there now.  Using git to see who else has
   modified the files you are working on can be helpful.

 - If you are responding to a bug report or a feature request, copy the
   original poster as well.

 - Send a copy to the relevant mailing list, or, if nothing else applies,
   the linux-kernel list.

 - If you are fixing a bug, think about whether the fix should go into the
   next stable update.  If so, stable@vger.kernel.org should get a copy of
   the patch.  Also add a "Cc: stable@vger.kernel.org" to the tags within
   the patch itself; that will cause the stable team to get a notification
   when your fix goes into the mainline.

When selecting recipients for a patch, it is good to have an idea of who
you think will eventually accept the patch and get it merged.  While it
is possible to send patches directly to Linus Torvalds and have him merge
them, things are not normally done that way.  Linus is busy, and there are
subsystem maintainers who watch over specific parts of the kernel.  Usually
you will be wanting that maintainer to merge your patches.  If there is no
obvious maintainer, Andrew Morton is often the patch target of last resort.

Patches need good subject lines.  The canonical format for a patch line is
something like:

::

	[PATCH nn/mm] subsys: one-line description of the patch

where "nn" is the ordinal number of the patch, "mm" is the total number of
patches in the series, and "subsys" is the name of the affected subsystem.
Clearly, nn/mm can be omitted for a single, standalone patch.

If you have a significant series of patches, it is customary to send an
introductory description as part zero.  This convention is not universally
followed though; if you use it, remember that information in the
introduction does not make it into the kernel changelogs.  So please ensure
that the patches, themselves, have complete changelog information.

In general, the second and following parts of a multi-part patch should be
sent as a reply to the first part so that they all thread together at the
receiving end.  Tools like git and quilt have commands to mail out a set of
patches with the proper threading.  If you have a long series, though, and
are using git, please stay away from the --chain-reply-to option to avoid
creating exceptionally deep nesting.

repo_name: angery/linux


code:
.. _development_followthrough:

Followthrough
=============

At this point, you have followed the guidelines given so far and, with the
addition of your own engineering skills, have posted a perfect series of
patches.  One of the biggest mistakes that even experienced kernel
developers can make is to conclude that their work is now done.  In truth,
posting patches indicates a transition into the next stage of the process,
with, possibly, quite a bit of work yet to be done.

It is a rare patch which is so good at its first posting that there is no
room for improvement.  The kernel development process recognizes this fact,
and, as a result, is heavily oriented toward the improvement of posted
code.  You, as the author of that code, will be expected to work with the
kernel community to ensure that your code is up to the kernel's quality
standards.  A failure to participate in this process is quite likely to
prevent the inclusion of your patches into the mainline.

Working with reviewers
----------------------

A patch of any significance will result in a number of comments from other
developers as they review the code.  Working with reviewers can be, for
many developers, the most intimidating part of the kernel development
process.  Life can be made much easier, though, if you keep a few things in
mind:

 - If you have explained your patch well, reviewers will understand its
   value and why you went to the trouble of writing it.  But that value
   will not keep them from asking a fundamental question: what will it be
   like to maintain a kernel with this code in it five or ten years later?
   Many of the changes you may be asked to make - from coding style tweaks
   to substantial rewrites - come from the understanding that Linux will
   still be around and under development a decade from now.

 - Code review is hard work, and it is a relatively thankless occupation;
   people remember who wrote kernel code, but there is little lasting fame
   for those who reviewed it.  So reviewers can get grumpy, especially when
   they see the same mistakes being made over and over again.  If you get a
   review which seems angry, insulting, or outright offensive, resist the
   impulse to respond in kind.  Code review is about the code, not about
   the people, and code reviewers are not attacking you personally.

 - Similarly, code reviewers are not trying to promote their employers'
   agendas at the expense of your own.  Kernel developers often expect to
   be working on the kernel years from now, but they understand that their
   employer could change.  They truly are, almost without exception,
   working toward the creation of the best kernel they can; they are not
   trying to create discomfort for their employers' competitors.

What all of this comes down to is that, when reviewers send you comments,
you need to pay attention to the technical observations that they are
making.  Do not let their form of expression or your own pride keep that
from happening.  When you get review comments on a patch, take the time to
understand what the reviewer is trying to say.  If possible, fix the things
that the reviewer is asking you to fix.  And respond back to the reviewer:
thank them, and describe how you will answer their questions.

Note that you do not have to agree with every change suggested by
reviewers.  If you believe that the reviewer has misunderstood your code,
explain what is really going on.  If you have a technical objection to a
suggested change, describe it and justify your solution to the problem.  If
your explanations make sense, the reviewer will accept them.  Should your
explanation not prove persuasive, though, especially if others start to
agree with the reviewer, take some time to think things over again.  It can
be easy to become blinded by your own solution to a problem to the point
that you don't realize that something is fundamentally wrong or, perhaps,
you're not even solving the right problem.

Andrew Morton has suggested that every review comment which does not result
in a code change should result in an additional code comment instead; that
can help future reviewers avoid the questions which came up the first time
around.

One fatal mistake is to ignore review comments in the hope that they will
go away.  They will not go away.  If you repost code without having
responded to the comments you got the time before, you're likely to find
that your patches go nowhere.

Speaking of reposting code: please bear in mind that reviewers are not
going to remember all the details of the code you posted the last time
around.  So it is always a good idea to remind reviewers of previously
raised issues and how you dealt with them; the patch changelog is a good
place for this kind of information.  Reviewers should not have to search
through list archives to familiarize themselves with what was said last
time; if you help them get a running start, they will be in a better mood
when they revisit your code.

What if you've tried to do everything right and things still aren't going
anywhere?  Most technical disagreements can be resolved through discussion,
but there are times when somebody simply has to make a decision.  If you
honestly believe that this decision is going against you wrongly, you can
always try appealing to a higher power.  As of this writing, that higher
power tends to be Andrew Morton.  Andrew has a great deal of respect in the
kernel development community; he can often unjam a situation which seems to
be hopelessly blocked.  Appealing to Andrew should not be done lightly,
though, and not before all other alternatives have been explored.  And bear
in mind, of course, that he may not agree with you either.

What happens next
-----------------

If a patch is considered to be a good thing to add to the kernel, and once
most of the review issues have been resolved, the next step is usually
entry into a subsystem maintainer's tree.  How that works varies from one
subsystem to the next; each maintainer has his or her own way of doing
things.  In particular, there may be more than one tree - one, perhaps,
dedicated to patches planned for the next merge window, and another for
longer-term work.

For patches applying to areas for which there is no obvious subsystem tree
(memory management patches, for example), the default tree often ends up
being -mm.  Patches which affect multiple subsystems can also end up going
through the -mm tree.

Inclusion into a subsystem tree can bring a higher level of visibility to a
patch.  Now other developers working with that tree will get the patch by
default.  Subsystem trees typically feed linux-next as well, making their
contents visible to the development community as a whole.  At this point,
there's a good chance that you will get more comments from a new set of
reviewers; these comments need to be answered as in the previous round.

What may also happen at this point, depending on the nature of your patch,
is that conflicts with work being done by others turn up.  In the worst
case, heavy patch conflicts can result in some work being put on the back
burner so that the remaining patches can be worked into shape and merged.
Other times, conflict resolution will involve working with the other
developers and, possibly, moving some patches between trees to ensure that
everything applies cleanly.  This work can be a pain, but count your
blessings: before the advent of the linux-next tree, these conflicts often
only turned up during the merge window and had to be addressed in a hurry.
Now they can be resolved at leisure, before the merge window opens.

Some day, if all goes well, you'll log on and see that your patch has been
merged into the mainline kernel.  Congratulations!  Once the celebration is
complete (and you have added yourself to the MAINTAINERS file), though, it
is worth remembering an important little fact: the job still is not done.
Merging into the mainline brings its own challenges.

To begin with, the visibility of your patch has increased yet again.  There
may be a new round of comments from developers who had not been aware of
the patch before.  It may be tempting to ignore them, since there is no
longer any question of your code being merged.  Resist that temptation,
though; you still need to be responsive to developers who have questions or
suggestions.

More importantly, though: inclusion into the mainline puts your code into
the hands of a much larger group of testers.  Even if you have contributed
a driver for hardware which is not yet available, you will be surprised by
how many people will build your code into their kernels.  And, of course,
where there are testers, there will be bug reports.

The worst sort of bug reports are regressions.  If your patch causes a
regression, you'll find an uncomfortable number of eyes upon you;
regressions need to be fixed as soon as possible.  If you are unwilling or
unable to fix the regression (and nobody else does it for you), your patch
will almost certainly be removed during the stabilization period.  Beyond
negating all of the work you have done to get your patch into the mainline,
having a patch pulled as the result of a failure to fix a regression could
well make it harder for you to get work merged in the future.

After any regressions have been dealt with, there may be other, ordinary
bugs to deal with.  The stabilization period is your best opportunity to
fix these bugs and ensure that your code's debut in a mainline kernel
release is as solid as possible.  So, please, answer bug reports, and fix
the problems if at all possible.  That's what the stabilization period is
for; you can start creating cool new patches once any problems with the old
ones have been taken care of.

And don't forget that there are other milestones which may also create bug
reports: the next mainline stable release, when prominent distributors pick
up a version of the kernel containing your patch, etc.  Continuing to
respond to these reports is a matter of basic pride in your work.  If that
is insufficient motivation, though, it's also worth considering that the
development community remembers developers who lose interest in their code
after it's merged.  The next time you post a patch, they will be evaluating
it with the assumption that you will not be around to maintain it
afterward.

Other things that can happen
-----------------------------

One day, you may open your mail client and see that somebody has mailed you
a patch to your code.  That is one of the advantages of having your code
out there in the open, after all.  If you agree with the patch, you can
either forward it on to the subsystem maintainer (be sure to include a
proper From: line so that the attribution is correct, and add a signoff of
your own), or send an Acked-by: response back and let the original poster
send it upward.

If you disagree with the patch, send a polite response explaining why.  If
possible, tell the author what changes need to be made to make the patch
acceptable to you.  There is a certain resistance to merging patches which
are opposed by the author and maintainer of the code, but it only goes so
far.  If you are seen as needlessly blocking good work, those patches will
eventually flow around you and get into the mainline anyway.  In the Linux
kernel, nobody has absolute veto power over any code.  Except maybe Linus.

On very rare occasion, you may see something completely different: another
developer posts a different solution to your problem.  At that point,
chances are that one of the two patches will not be merged, and "mine was
here first" is not considered to be a compelling technical argument.  If
somebody else's patch displaces yours and gets into the mainline, there is
really only one way to respond: be pleased that your problem got solved and
get on with your work.  Having one's work shoved aside in this manner can
be hurtful and discouraging, but the community will remember your reaction
long after they have forgotten whose patch actually got merged.


code:
.. _development_advancedtopics:

Advanced topics
===============

At this point, hopefully, you have a handle on how the development process
works.  There is still more to learn, however!  This section will cover a
number of topics which can be helpful for developers wanting to become a
regular part of the Linux kernel development process.

Managing patches with git
-------------------------

The use of distributed version control for the kernel began in early 2002,
when Linus first started playing with the proprietary BitKeeper
application.  While BitKeeper was controversial, the approach to software
version management it embodied most certainly was not.  Distributed version
control enabled an immediate acceleration of the kernel development
project.  In current times, there are several free alternatives to
BitKeeper.  For better or for worse, the kernel project has settled on git
as its tool of choice.

Managing patches with git can make life much easier for the developer,
especially as the volume of those patches grows.  Git also has its rough
edges and poses certain hazards; it is a young and powerful tool which is
still being civilized by its developers.  This document will not attempt to
teach the reader how to use git; that would be sufficient material for a
long document in its own right.  Instead, the focus here will be on how git
fits into the kernel development process in particular.  Developers who
wish to come up to speed with git will find more information at:

	https://git-scm.com/

	https://www.kernel.org/pub/software/scm/git/docs/user-manual.html

and on various tutorials found on the web.

The first order of business is to read the above sites and get a solid
understanding of how git works before trying to use it to make patches
available to others.  A git-using developer should be able to obtain a copy
of the mainline repository, explore the revision history, commit changes to
the tree, use branches, etc.  An understanding of git's tools for the
rewriting of history (such as rebase) is also useful.  Git comes with its
own terminology and concepts; a new user of git should know about refs,
remote branches, the index, fast-forward merges, pushes and pulls, detached
heads, etc.  It can all be a little intimidating at the outset, but the
concepts are not that hard to grasp with a bit of study.

Using git to generate patches for submission by email can be a good
exercise while coming up to speed.

When you are ready to start putting up git trees for others to look at, you
will, of course, need a server that can be pulled from.  Setting up such a
server with git-daemon is relatively straightforward if you have a system
which is accessible to the Internet.  Otherwise, free, public hosting sites
(Github, for example) are starting to appear on the net.  Established
developers can get an account on kernel.org, but those are not easy to come
by; see https://kernel.org/faq/ for more information.

The normal git workflow involves the use of a lot of branches.  Each line
of development can be separated into a separate "topic branch" and
maintained independently.  Branches in git are cheap, there is no reason to
not make free use of them.  And, in any case, you should not do your
development in any branch which you intend to ask others to pull from.
Publicly-available branches should be created with care; merge in patches
from development branches when they are in complete form and ready to go -
not before.

Git provides some powerful tools which can allow you to rewrite your
development history.  An inconvenient patch (one which breaks bisection,
say, or which has some other sort of obvious bug) can be fixed in place or
made to disappear from the history entirely.  A patch series can be
rewritten as if it had been written on top of today's mainline, even though
you have been working on it for months.  Changes can be transparently
shifted from one branch to another.  And so on.  Judicious use of git's
ability to revise history can help in the creation of clean patch sets with
fewer problems.

Excessive use of this capability can lead to other problems, though, beyond
a simple obsession for the creation of the perfect project history.
Rewriting history will rewrite the changes contained in that history,
turning a tested (hopefully) kernel tree into an untested one.  But, beyond
that, developers cannot easily collaborate if they do not have a shared
view of the project history; if you rewrite history which other developers
have pulled into their repositories, you will make life much more difficult
for those developers.  So a simple rule of thumb applies here: history
which has been exported to others should generally be seen as immutable
thereafter.

So, once you push a set of changes to your publicly-available server, those
changes should not be rewritten.  Git will attempt to enforce this rule if
you try to push changes which do not result in a fast-forward merge
(i.e. changes which do not share the same history).  It is possible to
override this check, and there may be times when it is necessary to rewrite
an exported tree.  Moving changesets between trees to avoid conflicts in
linux-next is one example.  But such actions should be rare.  This is one
of the reasons why development should be done in private branches (which
can be rewritten if necessary) and only moved into public branches when
it's in a reasonably advanced state.

As the mainline (or other tree upon which a set of changes is based)
advances, it is tempting to merge with that tree to stay on the leading
edge.  For a private branch, rebasing can be an easy way to keep up with
another tree, but rebasing is not an option once a tree is exported to the
world.  Once that happens, a full merge must be done.  Merging occasionally
makes good sense, but overly frequent merges can clutter the history
needlessly.  Suggested technique in this case is to merge infrequently, and
generally only at specific release points (such as a mainline -rc
release).  If you are nervous about specific changes, you can always
perform test merges in a private branch.  The git "rerere" tool can be
useful in such situations; it remembers how merge conflicts were resolved
so that you don't have to do the same work twice.

One of the biggest recurring complaints about tools like git is this: the
mass movement of patches from one repository to another makes it easy to
slip in ill-advised changes which go into the mainline below the review
radar.  Kernel developers tend to get unhappy when they see that kind of
thing happening; putting up a git tree with unreviewed or off-topic patches
can affect your ability to get trees pulled in the future.  Quoting Linus:

::

	You can send me patches, but for me to pull a git patch from you, I
	need to know that you know what you're doing, and I need to be able
	to trust things *without* then having to go and check every
	individual change by hand.

(https://lwn.net/Articles/224135/).

To avoid this kind of situation, ensure that all patches within a given
branch stick closely to the associated topic; a "driver fixes" branch
should not be making changes to the core memory management code.  And, most
importantly, do not use a git tree to bypass the review process.  Post an
occasional summary of the tree to the relevant list, and, when the time is
right, request that the tree be included in linux-next.

If and when others start to send patches for inclusion into your tree,
don't forget to review them.  Also ensure that you maintain the correct
authorship information; the git "am" tool does its best in this regard, but
you may have to add a "From:" line to the patch if it has been relayed to
you via a third party.

When requesting a pull, be sure to give all the relevant information: where
your tree is, what branch to pull, and what changes will result from the
pull.  The git request-pull command can be helpful in this regard; it will
format the request as other developers expect, and will also check to be
sure that you have remembered to push those changes to the public server.

Reviewing patches
-----------------

Some readers will certainly object to putting this section with "advanced
topics" on the grounds that even beginning kernel developers should be
reviewing patches.  It is certainly true that there is no better way to
learn how to program in the kernel environment than by looking at code
posted by others.  In addition, reviewers are forever in short supply; by
looking at code you can make a significant contribution to the process as a
whole.

Reviewing code can be an intimidating prospect, especially for a new kernel
developer who may well feel nervous about questioning code - in public -
which has been posted by those with more experience.  Even code written by
the most experienced developers can be improved, though.  Perhaps the best
piece of advice for reviewers (all reviewers) is this: phrase review
comments as questions rather than criticisms.  Asking "how does the lock
get released in this path?" will always work better than stating "the
locking here is wrong."

Different developers will review code from different points of view.  Some
are mostly concerned with coding style and whether code lines have trailing
white space.  Others will focus primarily on whether the change implemented
by the patch as a whole is a good thing for the kernel or not.  Yet others
will check for problematic locking, excessive stack usage, possible
security issues, duplication of code found elsewhere, adequate
documentation, adverse effects on performance, user-space ABI changes, etc.
All types of review, if they lead to better code going into the kernel, are
welcome and worthwhile.


code:
.. _development_conclusion:

For more information
====================

There are numerous sources of information on Linux kernel development and
related topics.  First among those will always be the Documentation
directory found in the kernel source distribution.  Start with the
top-level :ref:`process/howto.rst <process_howto>`; also read
:ref:`process/submitting-patches.rst <submittingpatches>`. Many internal
kernel APIs are documented using the kerneldoc mechanism; "make htmldocs"
or "make pdfdocs" can be used to generate those documents in HTML or PDF
format (though the version of TeX shipped by some distributions runs into
internal limits and fails to process the documents properly).

Various web sites discuss kernel development at all levels of detail.  Your
author would like to humbly suggest https://lwn.net/ as a source;
information on many specific kernel topics can be found via the LWN kernel
index at:

	https://lwn.net/Kernel/Index/

Beyond that, a valuable resource for kernel developers is:

	https://kernelnewbies.org/

And, of course, one should not forget https://kernel.org/, the definitive
location for kernel release information.

There are a number of books on kernel development:

	Linux Device Drivers, 3rd Edition (Jonathan Corbet, Alessandro
	Rubini, and Greg Kroah-Hartman).  Online at
	https://lwn.net/Kernel/LDD3/.

	Linux Kernel Development (Robert Love).

	Understanding the Linux Kernel (Daniel Bovet and Marco Cesati).

All of these books suffer from a common fault, though: they tend to be
somewhat obsolete by the time they hit the shelves, and they have been on
the shelves for a while now.  Still, there is quite a bit of good
information to be found there.

Documentation for git can be found at:

	https://www.kernel.org/pub/software/scm/git/docs/

	https://www.kernel.org/pub/software/scm/git/docs/user-manual.html

Conclusion
==========

Congratulations to anybody who has made it through this long-winded
document.  Hopefully it has provided a helpful understanding of how the
Linux kernel is developed and how you can participate in that process.

In the end, it's the participation that matters.  Any open source software
project is no more than the sum of what its contributors put into it.  The
Linux kernel has progressed as quickly and as well as it has because it has
been helped by an impressively large group of developers, all of whom are
working to make it better.  The kernel is a premier example of what can be
done when thousands of people work together toward a common goal.

The kernel can always benefit from a larger developer base, though.  There
is always more work to do.  But, just as importantly, most other
participants in the Linux ecosystem can benefit through contributing to the
kernel.  Getting code into the mainline is the key to higher code quality,
lower maintenance and distribution costs, a higher level of influence over
the direction of kernel development, and more.  It is a situation where
everybody involved wins.  Fire up your editor and come join us; you will be
more than welcome.


code:
.. _addsyscalls:

Adding a New System Call
========================

This document describes what's involved in adding a new system call to the
Linux kernel, over and above the normal submission advice in
:ref:`Documentation/process/submitting-patches.rst <submittingpatches>`.

System Call Alternatives
------------------------

The first thing to consider when adding a new system call is whether one of
the alternatives might be suitable instead.  Although system calls are the
most traditional and most obvious interaction points between userspace and the
kernel, there are other possibilities -- choose what fits best for your
interface.

 - If the operations involved can be made to look like a filesystem-like
   object, it may make more sense to create a new filesystem or device.  This
   also makes it easier to encapsulate the new functionality in a kernel module
   rather than requiring it to be built into the main kernel.

     - If the new functionality involves operations where the kernel notifies
       userspace that something has happened, then returning a new file
       descriptor for the relevant object allows userspace to use
       ``poll``/``select``/``epoll`` to receive that notification.
     - However, operations that don't map to
       :manpage:`read(2)`/:manpage:`write(2)`-like operations
       have to be implemented as :manpage:`ioctl(2)` requests, which can lead
       to a somewhat opaque API.

 - If you're just exposing runtime system information, a new node in sysfs
   (see ``Documentation/filesystems/sysfs.rst``) or the ``/proc`` filesystem may
   be more appropriate.  However, access to these mechanisms requires that the
   relevant filesystem is mounted, which might not always be the case (e.g.
   in a namespaced/sandboxed/chrooted environment).  Avoid adding any API to
   debugfs, as this is not considered a 'production' interface to userspace.
 - If the operation is specific to a particular file or file descriptor, then
   an additional :manpage:`fcntl(2)` command option may be more appropriate.  However,
   :manpage:`fcntl(2)` is a multiplexing system call that hides a lot of complexity, so
   this option is best for when the new function is closely analogous to
   existing :manpage:`fcntl(2)` functionality, or the new functionality is very simple
   (for example, getting/setting a simple flag related to a file descriptor).
 - If the operation is specific to a particular task or process, then an
   additional :manpage:`prctl(2)` command option may be more appropriate.  As
   with :manpage:`fcntl(2)`, this system call is a complicated multiplexor so
   is best reserved for near-analogs of existing ``prctl()`` commands or
   getting/setting a simple flag related to a process.

Designing the API: Planning for Extension
-----------------------------------------

A new system call forms part of the API of the kernel, and has to be supported
indefinitely.  As such, it's a very good idea to explicitly discuss the
interface on the kernel mailing list, and it's important to plan for future
extensions of the interface.

(The syscall table is littered with historical examples where this wasn't done,
together with the corresponding follow-up system calls --
``eventfd``/``eventfd2``, ``dup2``/``dup3``, ``inotify_init``/``inotify_init1``,
``pipe``/``pipe2``, ``renameat``/``renameat2`` -- so
learn from the history of the kernel and plan for extensions from the start.)

For simpler system calls that only take a couple of arguments, the preferred
way to allow for future extensibility is to include a flags argument to the
system call.  To make sure that userspace programs can safely use flags
between kernel versions, check whether the flags value holds any unknown
flags, and reject the system call (with ``EINVAL``) if it does::

    if (flags & ~(THING_FLAG1 | THING_FLAG2 | THING_FLAG3))
        return -EINVAL;

(If no flags values are used yet, check that the flags argument is zero.)

For more sophisticated system calls that involve a larger number of arguments,
it's preferred to encapsulate the majority of the arguments into a structure
that is passed in by pointer.  Such a structure can cope with future extension
by including a size argument in the structure::

    struct xyzzy_params {
        u32 size; /* userspace sets p->size = sizeof(struct xyzzy_params) */
        u32 param_1;
        u64 param_2;
        u64 param_3;
    };

As long as any subsequently added field, say ``param_4``, is designed so that a
zero value gives the previous behaviour, then this allows both directions of
version mismatch:

 - To cope with a later userspace program calling an older kernel, the kernel
   code should check that any memory beyond the size of the structure that it
   expects is zero (effectively checking that ``param_4 == 0``).
 - To cope with an older userspace program calling a newer kernel, the kernel
   code can zero-extend a smaller instance of the structure (effectively
   setting ``param_4 = 0``).

See :manpage:`perf_event_open(2)` and the ``perf_copy_attr()`` function (in
``kernel/events/core.c``) for an example of this approach.

Designing the API: Other Considerations
---------------------------------------

If your new system call allows userspace to refer to a kernel object, it
should use a file descriptor as the handle for that object -- don't invent a
new type of userspace object handle when the kernel already has mechanisms and
well-defined semantics for using file descriptors.

If your new :manpage:`xyzzy(2)` system call does return a new file descriptor,
then the flags argument should include a value that is equivalent to setting
``O_CLOEXEC`` on the new FD.  This makes it possible for userspace to close
the timing window between ``xyzzy()`` and calling
``fcntl(fd, F_SETFD, FD_CLOEXEC)``, where an unexpected ``fork()`` and
``execve()`` in another thread could leak a descriptor to
the exec'ed program. (However, resist the temptation to re-use the actual value
of the ``O_CLOEXEC`` constant, as it is architecture-specific and is part of a
numbering space of ``O_*`` flags that is fairly full.)

If your system call returns a new file descriptor, you should also consider
what it means to use the :manpage:`poll(2)` family of system calls on that file
descriptor. Making a file descriptor ready for reading or writing is the
normal way for the kernel to indicate to userspace that an event has
occurred on the corresponding kernel object.

If your new :manpage:`xyzzy(2)` system call involves a filename argument::

    int sys_xyzzy(const char __user *path, ..., unsigned int flags);

you should also consider whether an :manpage:`xyzzyat(2)` version is more appropriate::

    int sys_xyzzyat(int dfd, const char __user *path, ..., unsigned int flags);

This allows more flexibility for how userspace specifies the file in question;
in particular it allows userspace to request the functionality for an
already-opened file descriptor using the ``AT_EMPTY_PATH`` flag, effectively
giving an :manpage:`fxyzzy(3)` operation for free::

 - xyzzyat(AT_FDCWD, path, ..., 0) is equivalent to xyzzy(path,...)
 - xyzzyat(fd, "", ..., AT_EMPTY_PATH) is equivalent to fxyzzy(fd, ...)

(For more details on the rationale of the \*at() calls, see the
:manpage:`openat(2)` man page; for an example of AT_EMPTY_PATH, see the
:manpage:`fstatat(2)` man page.)

If your new :manpage:`xyzzy(2)` system call involves a parameter describing an
offset within a file, make its type ``loff_t`` so that 64-bit offsets can be
supported even on 32-bit architectures.

If your new :manpage:`xyzzy(2)` system call involves privileged functionality,
it needs to be governed by the appropriate Linux capability bit (checked with
a call to ``capable()``), as described in the :manpage:`capabilities(7)` man
page.  Choose an existing capability bit that governs related functionality,
but try to avoid combining lots of only vaguely related functions together
under the same bit, as this goes against capabilities' purpose of splitting
the power of root.  In particular, avoid adding new uses of the already
overly-general ``CAP_SYS_ADMIN`` capability.

If your new :manpage:`xyzzy(2)` system call manipulates a process other than
the calling process, it should be restricted (using a call to
``ptrace_may_access()``) so that only a calling process with the same
permissions as the target process, or with the necessary capabilities, can
manipulate the target process.

Finally, be aware that some non-x86 architectures have an easier time if
system call parameters that are explicitly 64-bit fall on odd-numbered
arguments (i.e. parameter 1, 3, 5), to allow use of contiguous pairs of 32-bit
registers.  (This concern does not apply if the arguments are part of a
structure that's passed in by pointer.)

Proposing the API
-----------------

To make new system calls easy to review, it's best to divide up the patchset
into separate chunks.  These should include at least the following items as
distinct commits (each of which is described further below):

 - The core implementation of the system call, together with prototypes,
   generic numbering, Kconfig changes and fallback stub implementation.
 - Wiring up of the new system call for one particular architecture, usually
   x86 (including all of x86_64, x86_32 and x32).
 - A demonstration of the use of the new system call in userspace via a
   selftest in ``tools/testing/selftests/``.
 - A draft man-page for the new system call, either as plain text in the
   cover letter, or as a patch to the (separate) man-pages repository.

New system call proposals, like any change to the kernel's API, should always
be cc'ed to linux-api@vger.kernel.org.

Generic System Call Implementation
----------------------------------

The main entry point for your new :manpage:`xyzzy(2)` system call will be called
``sys_xyzzy()``, but you add this entry point with the appropriate
``SYSCALL_DEFINEn()`` macro rather than explicitly.  The 'n' indicates the
number of arguments to the system call, and the macro takes the system call name
followed by the (type, name) pairs for the parameters as arguments.  Using
this macro allows metadata about the new system call to be made available for
other tools.

The new entry point also needs a corresponding function prototype, in
``include/linux/syscalls.h``, marked as asmlinkage to match the way that system
calls are invoked::

    asmlinkage long sys_xyzzy(...);

Some architectures (e.g. x86) have their own architecture-specific syscall
tables, but several other architectures share a generic syscall table. Add your
new system call to the generic list by adding an entry to the list in
``include/uapi/asm-generic/unistd.h``::

    #define __NR_xyzzy 292
    __SYSCALL(__NR_xyzzy, sys_xyzzy)

Also update the __NR_syscalls count to reflect the additional system call, and
note that if multiple new system calls are added in the same merge window,
your new syscall number may get adjusted to resolve conflicts.

The file ``kernel/sys_ni.c`` provides a fallback stub implementation of each
system call, returning ``-ENOSYS``.  Add your new system call here too::

    COND_SYSCALL(xyzzy);

Your new kernel functionality, and the system call that controls it, should
normally be optional, so add a ``CONFIG`` option (typically to
``init/Kconfig``) for it. As usual for new ``CONFIG`` options:

 - Include a description of the new functionality and system call controlled
   by the option.
 - Make the option depend on EXPERT if it should be hidden from normal users.
 - Make any new source files implementing the function dependent on the CONFIG
   option in the Makefile (e.g. ``obj-$(CONFIG_XYZZY_SYSCALL) += xyzzy.o``).
 - Double check that the kernel still builds with the new CONFIG option turned
   off.

To summarize, you need a commit that includes:

 - ``CONFIG`` option for the new function, normally in ``init/Kconfig``
 - ``SYSCALL_DEFINEn(xyzzy, ...)`` for the entry point
 - corresponding prototype in ``include/linux/syscalls.h``
 - generic table entry in ``include/uapi/asm-generic/unistd.h``
 - fallback stub in ``kernel/sys_ni.c``

x86 System Call Implementation
------------------------------

To wire up your new system call for x86 platforms, you need to update the
master syscall tables.  Assuming your new system call isn't special in some
way (see below), this involves a "common" entry (for x86_64 and x32) in
arch/x86/entry/syscalls/syscall_64.tbl::

    333   common   xyzzy     sys_xyzzy

and an "i386" entry in ``arch/x86/entry/syscalls/syscall_32.tbl``::

    380   i386     xyzzy     sys_xyzzy

Again, these numbers are liable to be changed if there are conflicts in the
relevant merge window.

Compatibility System Calls (Generic)
------------------------------------

For most system calls the same 64-bit implementation can be invoked even when
the userspace program is itself 32-bit; even if the system call's parameters
include an explicit pointer, this is handled transparently.

However, there are a couple of situations where a compatibility layer is
needed to cope with size differences between 32-bit and 64-bit.

The first is if the 64-bit kernel also supports 32-bit userspace programs, and
so needs to parse areas of (``__user``) memory that could hold either 32-bit or
64-bit values.  In particular, this is needed whenever a system call argument
is:

 - a pointer to a pointer
 - a pointer to a struct containing a pointer (e.g. ``struct iovec __user *``)
 - a pointer to a varying sized integral type (``time_t``, ``off_t``,
   ``long``, ...)
 - a pointer to a struct containing a varying sized integral type.

The second situation that requires a compatibility layer is if one of the
system call's arguments has a type that is explicitly 64-bit even on a 32-bit
architecture, for example ``loff_t`` or ``__u64``.  In this case, a value that
arrives at a 64-bit kernel from a 32-bit application will be split into two
32-bit values, which then need to be re-assembled in the compatibility layer.

(Note that a system call argument that's a pointer to an explicit 64-bit type
does **not** need a compatibility layer; for example, :manpage:`splice(2)`'s arguments of
type ``loff_t __user *`` do not trigger the need for a ``compat_`` system call.)

The compatibility version of the system call is called ``compat_sys_xyzzy()``,
and is added with the ``COMPAT_SYSCALL_DEFINEn()`` macro, analogously to
SYSCALL_DEFINEn.  This version of the implementation runs as part of a 64-bit
kernel, but expects to receive 32-bit parameter values and does whatever is
needed to deal with them.  (Typically, the ``compat_sys_`` version converts the
values to 64-bit versions and either calls on to the ``sys_`` version, or both of
them call a common inner implementation function.)

The compat entry point also needs a corresponding function prototype, in
``include/linux/compat.h``, marked as asmlinkage to match the way that system
calls are invoked::

    asmlinkage long compat_sys_xyzzy(...);

If the system call involves a structure that is laid out differently on 32-bit
and 64-bit systems, say ``struct xyzzy_args``, then the include/linux/compat.h
header file should also include a compat version of the structure (``struct
compat_xyzzy_args``) where each variable-size field has the appropriate
``compat_`` type that corresponds to the type in ``struct xyzzy_args``.  The
``compat_sys_xyzzy()`` routine can then use this ``compat_`` structure to
parse the arguments from a 32-bit invocation.

For example, if there are fields::

    struct xyzzy_args {
        const char __user *ptr;
        __kernel_long_t varying_val;
        u64 fixed_val;
        /* ... */
    };

in struct xyzzy_args, then struct compat_xyzzy_args would have::

    struct compat_xyzzy_args {
        compat_uptr_t ptr;
        compat_long_t varying_val;
        u64 fixed_val;
        /* ... */
    };

The generic system call list also needs adjusting to allow for the compat
version; the entry in ``include/uapi/asm-generic/unistd.h`` should use
``__SC_COMP`` rather than ``__SYSCALL``::

    #define __NR_xyzzy 292
    __SC_COMP(__NR_xyzzy, sys_xyzzy, compat_sys_xyzzy)

To summarize, you need:

 - a ``COMPAT_SYSCALL_DEFINEn(xyzzy, ...)`` for the compat entry point
 - corresponding prototype in ``include/linux/compat.h``
 - (if needed) 32-bit mapping struct in ``include/linux/compat.h``
 - instance of ``__SC_COMP`` not ``__SYSCALL`` in
   ``include/uapi/asm-generic/unistd.h``

Compatibility System Calls (x86)
--------------------------------

To wire up the x86 architecture of a system call with a compatibility version,
the entries in the syscall tables need to be adjusted.

First, the entry in ``arch/x86/entry/syscalls/syscall_32.tbl`` gets an extra
column to indicate that a 32-bit userspace program running on a 64-bit kernel
should hit the compat entry point::

    380   i386     xyzzy     sys_xyzzy    __ia32_compat_sys_xyzzy

Second, you need to figure out what should happen for the x32 ABI version of
the new system call.  There's a choice here: the layout of the arguments
should either match the 64-bit version or the 32-bit version.

If there's a pointer-to-a-pointer involved, the decision is easy: x32 is
ILP32, so the layout should match the 32-bit version, and the entry in
``arch/x86/entry/syscalls/syscall_64.tbl`` is split so that x32 programs hit
the compatibility wrapper::

    333   64       xyzzy     sys_xyzzy
    ...
    555   x32      xyzzy     __x32_compat_sys_xyzzy

If no pointers are involved, then it is preferable to re-use the 64-bit system
call for the x32 ABI (and consequently the entry in
arch/x86/entry/syscalls/syscall_64.tbl is unchanged).

In either case, you should check that the types involved in your argument
layout do indeed map exactly from x32 (-mx32) to either the 32-bit (-m32) or
64-bit (-m64) equivalents.

System Calls Returning Elsewhere
--------------------------------

For most system calls, once the system call is complete the user program
continues exactly where it left off -- at the next instruction, with the
stack the same and most of the registers the same as before the system call,
and with the same virtual memory space.

However, a few system calls do things differently.  They might return to a
different location (``rt_sigreturn``) or change the memory space
(``fork``/``vfork``/``clone``) or even architecture (``execve``/``execveat``)
of the program.

To allow for this, the kernel implementation of the system call may need to
save and restore additional registers to the kernel stack, allowing complete
control of where and how execution continues after the system call.

This is arch-specific, but typically involves defining assembly entry points
that save/restore additional registers and invoke the real system call entry
point.

For x86_64, this is implemented as a ``stub_xyzzy`` entry point in
``arch/x86/entry/entry_64.S``, and the entry in the syscall table
(``arch/x86/entry/syscalls/syscall_64.tbl``) is adjusted to match::

    333   common   xyzzy     stub_xyzzy

The equivalent for 32-bit programs running on a 64-bit kernel is normally
called ``stub32_xyzzy`` and implemented in ``arch/x86/entry/entry_64_compat.S``,
with the corresponding syscall table adjustment in
``arch/x86/entry/syscalls/syscall_32.tbl``::

    380   i386     xyzzy     sys_xyzzy    stub32_xyzzy

If the system call needs a compatibility layer (as in the previous section)
then the ``stub32_`` version needs to call on to the ``compat_sys_`` version
of the system call rather than the native 64-bit version.  Also, if the x32 ABI
implementation is not common with the x86_64 version, then its syscall
table will also need to invoke a stub that calls on to the ``compat_sys_``
version.

For completeness, it's also nice to set up a mapping so that user-mode Linux
still works -- its syscall table will reference stub_xyzzy, but the UML build
doesn't include ``arch/x86/entry/entry_64.S`` implementation (because UML
simulates registers etc).  Fixing this is as simple as adding a #define to
``arch/x86/um/sys_call_table_64.c``::

    #define stub_xyzzy sys_xyzzy

Other Details
-------------

Most of the kernel treats system calls in a generic way, but there is the
occasional exception that may need updating for your particular system call.

The audit subsystem is one such special case; it includes (arch-specific)
functions that classify some special types of system call -- specifically
file open (``open``/``openat``), program execution (``execve``/``exeveat``) or
socket multiplexor (``socketcall``) operations. If your new system call is
analogous to one of these, then the audit system should be updated.

More generally, if there is an existing system call that is analogous to your
new system call, it's worth doing a kernel-wide grep for the existing system
call to check there are no other special cases.

Testing
-------

A new system call should obviously be tested; it is also useful to provide
reviewers with a demonstration of how user space programs will use the system
call.  A good way to combine these aims is to include a simple self-test
program in a new directory under ``tools/testing/selftests/``.

For a new system call, there will obviously be no libc wrapper function and so
the test will need to invoke it using ``syscall()``; also, if the system call
involves a new userspace-visible structure, the corresponding header will need
to be installed to compile the test.

Make sure the selftest runs successfully on all supported architectures.  For
example, check that it works when compiled as an x86_64 (-m64), x86_32 (-m32)
and x32 (-mx32) ABI program.

For more extensive and thorough testing of new functionality, you should also
consider adding tests to the Linux Test Project, or to the xfstests project
for filesystem-related changes.

 - https://linux-test-project.github.io/
 - git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git

Man Page
--------

All new system calls should come with a complete man page, ideally using groff
markup, but plain text will do.  If groff is used, it's helpful to include a
pre-rendered ASCII version of the man page in the cover email for the
patchset, for the convenience of reviewers.

The man page should be cc'ed to linux-man@vger.kernel.org
For more details, see https://www.kernel.org/doc/man-pages/patches.html

Do not call System Calls in the Kernel
--------------------------------------

System calls are, as stated above, interaction points between userspace and
the kernel.  Therefore, system call functions such as ``sys_xyzzy()`` or
``compat_sys_xyzzy()`` should only be called from userspace via the syscall
table, but not from elsewhere in the kernel.  If the syscall functionality is
useful to be used within the kernel, needs to be shared between an old and a
new syscall, or needs to be shared between a syscall and its compatibility
variant, it should be implemented by means of a "helper" function (such as
``ksys_xyzzy()``).  This kernel function may then be called within the
syscall stub (``sys_xyzzy()``), the compatibility syscall stub
(``compat_sys_xyzzy()``), and/or other kernel code.

At least on 64-bit x86, it will be a hard requirement from v4.17 onwards to not
call system call functions in the kernel.  It uses a different calling
convention for system calls where ``struct pt_regs`` is decoded on-the-fly in a
syscall wrapper which then hands processing over to the actual syscall function.
This means that only those parameters which are actually needed for a specific
syscall are passed on during syscall entry, instead of filling in six CPU
registers with random user space content all the time (which may cause serious
trouble down the call chain).

Moreover, rules on how data may be accessed may differ between kernel data and
user data.  This is another reason why calling ``sys_xyzzy()`` is generally a
bad idea.

Exceptions to this rule are only allowed in architecture-specific overrides,
architecture-specific compatibility wrappers, or other code in arch/.

References and Sources
----------------------

 - LWN article from Michael Kerrisk on use of flags argument in system calls:
   https://lwn.net/Articles/585415/
 - LWN article from Michael Kerrisk on how to handle unknown flags in a system
   call: https://lwn.net/Articles/588444/
 - LWN article from Jake Edge describing constraints on 64-bit system call
   arguments: https://lwn.net/Articles/311630/
 - Pair of LWN articles from David Drysdale that describe the system call
   implementation paths in detail for v3.14:

    - https://lwn.net/Articles/604287/
    - https://lwn.net/Articles/604515/

 - Architecture-specific requirements for system calls are discussed in the
   :manpage:`syscall(2)` man-page:
   http://man7.org/linux/man-pages/man2/syscall.2.html#NOTES
 - Collated emails from Linus Torvalds discussing the problems with ``ioctl()``:
   https://yarchive.net/comp/linux/ioctl.html
 - "How to not invent kernel interfaces", Arnd Bergmann,
   https://www.ukuug.org/events/linux2007/2007/papers/Bergmann.pdf
 - LWN article from Michael Kerrisk on avoiding new uses of CAP_SYS_ADMIN:
   https://lwn.net/Articles/486306/
 - Recommendation from Andrew Morton that all related information for a new
   system call should come in the same email thread:
   https://lore.kernel.org/r/20140724144747.3041b208832bbdf9fbce5d96@linux-foundation.org
 - Recommendation from Michael Kerrisk that a new system call should come with
   a man page: https://lore.kernel.org/r/CAKgNAkgMA39AfoSoA5Pe1r9N+ZzfYQNvNPvcRN7tOvRb8+v06Q@mail.gmail.com
 - Suggestion from Thomas Gleixner that x86 wire-up should be in a separate
   commit: https://lore.kernel.org/r/alpine.DEB.2.11.1411191249560.3909@nanos
 - Suggestion from Greg Kroah-Hartman that it's good for new system calls to
   come with a man-page & selftest: https://lore.kernel.org/r/20140320025530.GA25469@kroah.com
 - Discussion from Michael Kerrisk of new system call vs. :manpage:`prctl(2)` extension:
   https://lore.kernel.org/r/CAHO5Pa3F2MjfTtfNxa8LbnkeeU8=YJ+9tDqxZpw7Gz59E-4AUg@mail.gmail.com
 - Suggestion from Ingo Molnar that system calls that involve multiple
   arguments should encapsulate those arguments in a struct, which includes a
   size field for future extensibility: https://lore.kernel.org/r/20150730083831.GA22182@gmail.com
 - Numbering oddities arising from (re-)use of O_* numbering space flags:

    - commit 75069f2b5bfb ("vfs: renumber FMODE_NONOTIFY and add to uniqueness
      check")
    - commit 12ed2e36c98a ("fanotify: FMODE_NONOTIFY and __O_SYNC in sparc
      conflict")
    - commit bb458c644a59 ("Safer ABI for O_TMPFILE")

 - Discussion from Matthew Wilcox about restrictions on 64-bit arguments:
   https://lore.kernel.org/r/20081212152929.GM26095@parisc-linux.org
 - Recommendation from Greg Kroah-Hartman that unknown flags should be
   policed: https://lore.kernel.org/r/20140717193330.GB4703@kroah.com
 - Recommendation from Linus Torvalds that x32 system calls should prefer
   compatibility with 64-bit versions rather than 32-bit versions:
   https://lore.kernel.org/r/CA+55aFxfmwfB7jbbrXxa=K7VBYPfAvmu3XOkGrLbB1UFjX1+Ew@mail.gmail.com