code:
Reexporting NFS filesystems
===========================

Overview
--------

It is possible to reexport an NFS filesystem over NFS.  However, this
feature comes with a number of limitations.  Before trying it, we
recommend some careful research to determine whether it will work for
your purposes.

A discussion of current known limitations follows.

"fsid=" required, crossmnt broken
---------------------------------

We require the "fsid=" export option on any reexport of an NFS
filesystem.  You can use "uuidgen -r" to generate a unique argument.

The "crossmnt" export does not propagate "fsid=", so it will not allow
traversing into further nfs filesystems; if you wish to export nfs
filesystems mounted under the exported filesystem, you'll need to export
them explicitly, assigning each its own unique "fsid= option.

Reboot recovery
---------------

The NFS protocol's normal reboot recovery mechanisms don't work for the
case when the reexport server reboots.  Clients will lose any locks
they held before the reboot, and further IO will result in errors.
Closing and reopening files should clear the errors.

Filehandle limits
-----------------

If the original server uses an X byte filehandle for a given object, the
reexport server's filehandle for the reexported object will be X+22
bytes, rounded up to the nearest multiple of four bytes.

The result must fit into the RFC-mandated filehandle size limits:

+-------+-----------+
| NFSv2 |  32 bytes |
+-------+-----------+
| NFSv3 |  64 bytes |
+-------+-----------+
| NFSv4 | 128 bytes |
+-------+-----------+

So, for example, you will only be able to reexport a filesystem over
NFSv2 if the original server gives you filehandles that fit in 10
bytes--which is unlikely.

In general there's no way to know the maximum filehandle size given out
by an NFS server without asking the server vendor.

But the following table gives a few examples.  The first column is the
typical length of the filehandle from a Linux server exporting the given
filesystem, the second is the length after that nfs export is reexported
by another Linux host:

+--------+-------------------+----------------+
|        | filehandle length | after reexport |
+========+===================+================+
| ext4:  | 28 bytes          | 52 bytes       |
+--------+-------------------+----------------+
| xfs:   | 32 bytes          | 56 bytes       |
+--------+-------------------+----------------+
| btrfs: | 40 bytes          | 64 bytes       |
+--------+-------------------+----------------+

All will therefore fit in an NFSv3 or NFSv4 filehandle after reexport,
but none are reexportable over NFSv2.

Linux server filehandles are a bit more complicated than this, though;
for example:

        - The (non-default) "subtreecheck" export option generally
          requires another 4 to 8 bytes in the filehandle.
        - If you export a subdirectory of a filesystem (instead of
          exporting the filesystem root), that also usually adds 4 to 8
          bytes.
        - If you export over NFSv2, knfsd usually uses a shorter
          filesystem identifier that saves 8 bytes.
        - The root directory of an export uses a filehandle that is
          shorter.

As you can see, the 128-byte NFSv4 filehandle is large enough that
you're unlikely to have trouble using NFSv4 to reexport any filesystem
exported from a Linux server.  In general, if the original server is
something that also supports NFSv3, you're *probably* OK.  Re-exporting
over NFSv3 may be dicier, and reexporting over NFSv2 will probably
never work.

For more details of Linux filehandle structure, the best reference is
the source code and comments; see in particular:

        - include/linux/exportfs.h:enum fid_type
        - include/uapi/linux/nfsd/nfsfh.h:struct nfs_fhbase_new
        - fs/nfsd/nfsfh.c:set_version_and_fsid_type
        - fs/nfs/export.c:nfs_encode_fh

Open DENY bits ignored
----------------------

NFS since NFSv4 supports ALLOW and DENY bits taken from Windows, which
allow you, for example, to open a file in a mode which forbids other
read opens or write opens. The Linux client doesn't use them, and the
server's support has always been incomplete: they are enforced only
against other NFS users, not against processes accessing the exported
filesystem locally. A reexport server will also not pass them along to
the original server, so they will not be enforced between clients of
different reexport servers.


code:
=========
RPC Cache
=========

This document gives a brief introduction to the caching
mechanisms in the sunrpc layer that is used, in particular,
for NFS authentication.

Caches
======

The caching replaces the old exports table and allows for
a wide variety of values to be caches.

There are a number of caches that are similar in structure though
quite possibly very different in content and use.  There is a corpus
of common code for managing these caches.

Examples of caches that are likely to be needed are:

  - mapping from IP address to client name
  - mapping from client name and filesystem to export options
  - mapping from UID to list of GIDs, to work around NFS's limitation
    of 16 gids.
  - mappings between local UID/GID and remote UID/GID for sites that
    do not have uniform uid assignment
  - mapping from network identify to public key for crypto authentication.

The common code handles such things as:

   - general cache lookup with correct locking
   - supporting 'NEGATIVE' as well as positive entries
   - allowing an EXPIRED time on cache items, and removing
     items after they expire, and are no longer in-use.
   - making requests to user-space to fill in cache entries
   - allowing user-space to directly set entries in the cache
   - delaying RPC requests that depend on as-yet incomplete
     cache entries, and replaying those requests when the cache entry
     is complete.
   - clean out old entries as they expire.

Creating a Cache
----------------

-  A cache needs a datum to store.  This is in the form of a
   structure definition that must contain a struct cache_head
   as an element, usually the first.
   It will also contain a key and some content.
   Each cache element is reference counted and contains
   expiry and update times for use in cache management.
-  A cache needs a "cache_detail" structure that
   describes the cache.  This stores the hash table, some
   parameters for cache management, and some operations detailing how
   to work with particular cache items.

   The operations are:

    struct cache_head \*alloc(void)
      This simply allocates appropriate memory and returns
      a pointer to the cache_detail embedded within the
      structure

    void cache_put(struct kref \*)
      This is called when the last reference to an item is
      dropped.  The pointer passed is to the 'ref' field
      in the cache_head.  cache_put should release any
      references create by 'cache_init' and, if CACHE_VALID
      is set, any references created by cache_update.
      It should then release the memory allocated by
      'alloc'.

    int match(struct cache_head \*orig, struct cache_head \*new)
      test if the keys in the two structures match.  Return
      1 if they do, 0 if they don't.

    void init(struct cache_head \*orig, struct cache_head \*new)
      Set the 'key' fields in 'new' from 'orig'.  This may
      include taking references to shared objects.

    void update(struct cache_head \*orig, struct cache_head \*new)
      Set the 'content' fileds in 'new' from 'orig'.

    int cache_show(struct seq_file \*m, struct cache_detail \*cd, struct cache_head \*h)
      Optional.  Used to provide a /proc file that lists the
      contents of a cache.  This should show one item,
      usually on just one line.

    int cache_request(struct cache_detail \*cd, struct cache_head \*h, char \*\*bpp, int \*blen)
      Format a request to be send to user-space for an item
      to be instantiated.  \*bpp is a buffer of size \*blen.
      bpp should be moved forward over the encoded message,
      and  \*blen should be reduced to show how much free
      space remains.  Return 0 on success or <0 if not
      enough room or other problem.

    int cache_parse(struct cache_detail \*cd, char \*buf, int len)
      A message from user space has arrived to fill out a
      cache entry.  It is in 'buf' of length 'len'.
      cache_parse should parse this, find the item in the
      cache with sunrpc_cache_lookup_rcu, and update the item
      with sunrpc_cache_update.

-  A cache needs to be registered using cache_register().  This
   includes it on a list of caches that will be regularly
   cleaned to discard old data.

Using a cache
-------------

To find a value in a cache, call sunrpc_cache_lookup_rcu passing a pointer
to the cache_head in a sample item with the 'key' fields filled in.
This will be passed to ->match to identify the target entry.  If no
entry is found, a new entry will be create, added to the cache, and
marked as not containing valid data.

The item returned is typically passed to cache_check which will check
if the data is valid, and may initiate an up-call to get fresh data.
cache_check will return -ENOENT in the entry is negative or if an up
call is needed but not possible, -EAGAIN if an upcall is pending,
or 0 if the data is valid;

cache_check can be passed a "struct cache_req\*".  This structure is
typically embedded in the actual request and can be used to create a
deferred copy of the request (struct cache_deferred_req).  This is
done when the found cache item is not uptodate, but the is reason to
believe that userspace might provide information soon.  When the cache
item does become valid, the deferred copy of the request will be
revisited (->revisit).  It is expected that this method will
reschedule the request for processing.

The value returned by sunrpc_cache_lookup_rcu can also be passed to
sunrpc_cache_update to set the content for the item.  A second item is
passed which should hold the content.  If the item found by _lookup
has valid data, then it is discarded and a new item is created.  This
saves any user of an item from worrying about content changing while
it is being inspected.  If the item found by _lookup does not contain
valid data, then the content is copied across and CACHE_VALID is set.

Populating a cache
------------------

Each cache has a name, and when the cache is registered, a directory
with that name is created in /proc/net/rpc

This directory contains a file called 'channel' which is a channel
for communicating between kernel and user for populating the cache.
This directory may later contain other files of interacting
with the cache.

The 'channel' works a bit like a datagram socket. Each 'write' is
passed as a whole to the cache for parsing and interpretation.
Each cache can treat the write requests differently, but it is
expected that a message written will contain:

  - a key
  - an expiry time
  - a content.

with the intention that an item in the cache with the give key
should be create or updated to have the given content, and the
expiry time should be set on that item.

Reading from a channel is a bit more interesting.  When a cache
lookup fails, or when it succeeds but finds an entry that may soon
expire, a request is lodged for that cache item to be updated by
user-space.  These requests appear in the channel file.

Successive reads will return successive requests.
If there are no more requests to return, read will return EOF, but a
select or poll for read will block waiting for another request to be
added.

Thus a user-space helper is likely to::

  open the channel.
    select for readable
    read a request
    write a response
  loop.

If it dies and needs to be restarted, any requests that have not been
answered will still appear in the file and will be read by the new
instance of the helper.

Each cache should define a "cache_parse" method which takes a message
written from user-space and processes it.  It should return an error
(which propagates back to the write syscall) or 0.

Each cache should also define a "cache_request" method which
takes a cache item and encodes a request into the buffer
provided.

.. note::
  If a cache has no active readers on the channel, and has had not
  active readers for more than 60 seconds, further requests will not be
  added to the channel but instead all lookups that do not find a valid
  entry will fail.  This is partly for backward compatibility: The
  previous nfs exports table was deemed to be authoritative and a
  failed lookup meant a definite 'no'.

request/response format
-----------------------

While each cache is free to use its own format for requests
and responses over channel, the following is recommended as
appropriate and support routines are available to help:
Each request or response record should be printable ASCII
with precisely one newline character which should be at the end.
Fields within the record should be separated by spaces, normally one.
If spaces, newlines, or nul characters are needed in a field they
much be quoted.  two mechanisms are available:

-  If a field begins '\x' then it must contain an even number of
   hex digits, and pairs of these digits provide the bytes in the
   field.
-  otherwise a \ in the field must be followed by 3 octal digits
   which give the code for a byte.  Other characters are treated
   as them selves.  At the very least, space, newline, nul, and
   '\' must be quoted in this way.

repo_name: angery/linux


code:
=========================================
rpcsec_gss support for kernel RPC servers
=========================================

This document gives references to the standards and protocols used to
implement RPCGSS authentication in kernel RPC servers such as the NFS
server and the NFS client's NFSv4.0 callback server.  (But note that
NFSv4.1 and higher don't require the client to act as a server for the
purposes of authentication.)

RPCGSS is specified in a few IETF documents:

 - RFC2203 v1: https://tools.ietf.org/rfc/rfc2203.txt
 - RFC5403 v2: https://tools.ietf.org/rfc/rfc5403.txt

There is a third version that we don't currently implement:

 - RFC7861 v3: https://tools.ietf.org/rfc/rfc7861.txt

Background
==========

The RPCGSS Authentication method describes a way to perform GSSAPI
Authentication for NFS.  Although GSSAPI is itself completely mechanism
agnostic, in many cases only the KRB5 mechanism is supported by NFS
implementations.

The Linux kernel, at the moment, supports only the KRB5 mechanism, and
depends on GSSAPI extensions that are KRB5 specific.

GSSAPI is a complex library, and implementing it completely in kernel is
unwarranted. However GSSAPI operations are fundementally separable in 2
parts:

- initial context establishment
- integrity/privacy protection (signing and encrypting of individual
  packets)

The former is more complex and policy-independent, but less
performance-sensitive.  The latter is simpler and needs to be very fast.

Therefore, we perform per-packet integrity and privacy protection in the
kernel, but leave the initial context establishment to userspace.  We
need upcalls to request userspace to perform context establishment.

NFS Server Legacy Upcall Mechanism
==================================

The classic upcall mechanism uses a custom text based upcall mechanism
to talk to a custom daemon called rpc.svcgssd that is provide by the
nfs-utils package.

This upcall mechanism has 2 limitations:

A) It can handle tokens that are no bigger than 2KiB

In some Kerberos deployment GSSAPI tokens can be quite big, up and
beyond 64KiB in size due to various authorization extensions attacked to
the Kerberos tickets, that needs to be sent through the GSS layer in
order to perform context establishment.

B) It does not properly handle creds where the user is member of more
than a few thousand groups (the current hard limit in the kernel is 65K
groups) due to limitation on the size of the buffer that can be send
back to the kernel (4KiB).

NFS Server New RPC Upcall Mechanism
===================================

The newer upcall mechanism uses RPC over a unix socket to a daemon
called gss-proxy, implemented by a userspace program called Gssproxy.

The gss_proxy RPC protocol is currently documented `here
<https://fedorahosted.org/gss-proxy/wiki/ProtocolDocumentation>`_.

This upcall mechanism uses the kernel rpc client and connects to the gssproxy
userspace program over a regular unix socket. The gssproxy protocol does not
suffer from the size limitations of the legacy protocol.

Negotiating Upcall Mechanisms
=============================

To provide backward compatibility, the kernel defaults to using the
legacy mechanism.  To switch to the new mechanism, gss-proxy must bind
to /var/run/gssproxy.sock and then write "1" to
/proc/net/rpc/use-gss-proxy.  If gss-proxy dies, it must repeat both
steps.

Once the upcall mechanism is chosen, it cannot be changed.  To prevent
locking into the legacy mechanisms, the above steps must be performed
before starting nfsd.  Whoever starts nfsd can guarantee this by reading
from /proc/net/rpc/use-gss-proxy and checking that it contains a
"1"--the read will block until gss-proxy has done its write to the file.


code:
.. SPDX-License-Identifier: GPL-2.0

======
NILFS2
======

NILFS2 is a log-structured file system (LFS) supporting continuous
snapshotting.  In addition to versioning capability of the entire file
system, users can even restore files mistakenly overwritten or
destroyed just a few seconds ago.  Since NILFS2 can keep consistency
like conventional LFS, it achieves quick recovery after system
crashes.

NILFS2 creates a number of checkpoints every few seconds or per
synchronous write basis (unless there is no change).  Users can select
significant versions among continuously created checkpoints, and can
change them into snapshots which will be preserved until they are
changed back to checkpoints.

There is no limit on the number of snapshots until the volume gets
full.  Each snapshot is mountable as a read-only file system
concurrently with its writable mount, and this feature is convenient
for online backup.

The userland tools are included in nilfs-utils package, which is
available from the following download page.  At least "mkfs.nilfs2",
"mount.nilfs2", "umount.nilfs2", and "nilfs_cleanerd" (so called
cleaner or garbage collector) are required.  Details on the tools are
described in the man pages included in the package.

:Project web page:    https://nilfs.sourceforge.io/
:Download page:       https://nilfs.sourceforge.io/en/download.html
:List info:           http://vger.kernel.org/vger-lists.html#linux-nilfs

Caveats
=======

Features which NILFS2 does not support yet:

	- atime
	- extended attributes
	- POSIX ACLs
	- quotas
	- fsck
	- defragmentation

Mount options
=============

NILFS2 supports the following mount options:
(*) == default

======================= =======================================================
barrier(*)		This enables/disables the use of write barriers.  This
nobarrier		requires an IO stack which can support barriers, and
			if nilfs gets an error on a barrier write, it will
			disable again with a warning.
errors=continue		Keep going on a filesystem error.
errors=remount-ro(*)	Remount the filesystem read-only on an error.
errors=panic		Panic and halt the machine if an error occurs.
cp=n			Specify the checkpoint-number of the snapshot to be
			mounted.  Checkpoints and snapshots are listed by lscp
			user command.  Only the checkpoints marked as snapshot
			are mountable with this option.  Snapshot is read-only,
			so a read-only mount option must be specified together.
order=relaxed(*)	Apply relaxed order semantics that allows modified data
			blocks to be written to disk without making a
			checkpoint if no metadata update is going.  This mode
			is equivalent to the ordered data mode of the ext3
			filesystem except for the updates on data blocks still
			conserve atomicity.  This will improve synchronous
			write performance for overwriting.
order=strict		Apply strict in-order semantics that preserves sequence
			of all file operations including overwriting of data
			blocks.  That means, it is guaranteed that no
			overtaking of events occurs in the recovered file
			system after a crash.
norecovery		Disable recovery of the filesystem on mount.
			This disables every write access on the device for
			read-only mounts or snapshots.  This option will fail
			for r/w mounts on an unclean volume.
discard			This enables/disables the use of discard/TRIM commands.
nodiscard(*)		The discard/TRIM commands are sent to the underlying
			block device when blocks are freed.  This is useful
			for SSD devices and sparse/thinly-provisioned LUNs.
======================= =======================================================

Ioctls
======

There is some NILFS2 specific functionality which can be accessed by applications
through the system call interfaces. The list of all NILFS2 specific ioctls are
shown in the table below.

Table of NILFS2 specific ioctls:

 ============================== ===============================================
 Ioctl			        Description
 ============================== ===============================================
 NILFS_IOCTL_CHANGE_CPMODE      Change mode of given checkpoint between
			        checkpoint and snapshot state. This ioctl is
			        used in chcp and mkcp utilities.

 NILFS_IOCTL_DELETE_CHECKPOINT  Remove checkpoint from NILFS2 file system.
			        This ioctl is used in rmcp utility.

 NILFS_IOCTL_GET_CPINFO         Return info about requested checkpoints. This
			        ioctl is used in lscp utility and by
			        nilfs_cleanerd daemon.

 NILFS_IOCTL_GET_CPSTAT         Return checkpoints statistics. This ioctl is
			        used by lscp, rmcp utilities and by
			        nilfs_cleanerd daemon.

 NILFS_IOCTL_GET_SUINFO         Return segment usage info about requested
			        segments. This ioctl is used in lssu,
			        nilfs_resize utilities and by nilfs_cleanerd
			        daemon.

 NILFS_IOCTL_SET_SUINFO         Modify segment usage info of requested
				segments. This ioctl is used by
				nilfs_cleanerd daemon to skip unnecessary
				cleaning operation of segments and reduce
				performance penalty or wear of flash device
				due to redundant move of in-use blocks.

 NILFS_IOCTL_GET_SUSTAT         Return segment usage statistics. This ioctl
			        is used in lssu, nilfs_resize utilities and
			        by nilfs_cleanerd daemon.

 NILFS_IOCTL_GET_VINFO          Return information on virtual block addresses.
			        This ioctl is used by nilfs_cleanerd daemon.

 NILFS_IOCTL_GET_BDESCS         Return information about descriptors of disk
			        block numbers. This ioctl is used by
			        nilfs_cleanerd daemon.

 NILFS_IOCTL_CLEAN_SEGMENTS     Do garbage collection operation in the
			        environment of requested parameters from
			        userspace. This ioctl is used by
			        nilfs_cleanerd daemon.

 NILFS_IOCTL_SYNC               Make a checkpoint. This ioctl is used in
			        mkcp utility.

 NILFS_IOCTL_RESIZE             Resize NILFS2 volume. This ioctl is used
			        by nilfs_resize utility.

 NILFS_IOCTL_SET_ALLOC_RANGE    Define lower limit of segments in bytes and
			        upper limit of segments in bytes. This ioctl
			        is used by nilfs_resize utility.
 ============================== ===============================================

NILFS2 usage
============

To use nilfs2 as a local file system, simply::

 # mkfs -t nilfs2 /dev/block_device
 # mount -t nilfs2 /dev/block_device /dir

This will also invoke the cleaner through the mount helper program
(mount.nilfs2).

Checkpoints and snapshots are managed by the following commands.
Their manpages are included in the nilfs-utils package above.

  ====     ===========================================================
  lscp     list checkpoints or snapshots.
  mkcp     make a checkpoint or a snapshot.
  chcp     change an existing checkpoint to a snapshot or vice versa.
  rmcp     invalidate specified checkpoint(s).
  ====     ===========================================================

To mount a snapshot::

 # mount -t nilfs2 -r -o cp=<cno> /dev/block_device /snap_dir

where <cno> is the checkpoint number of the snapshot.

To unmount the NILFS2 mount point or snapshot, simply::

 # umount /dir

Then, the cleaner daemon is automatically shut down by the umount
helper program (umount.nilfs2).

Disk format
===========

A nilfs2 volume is equally divided into a number of segments except
for the super block (SB) and segment #0.  A segment is the container
of logs.  Each log is composed of summary information blocks, payload
blocks, and an optional super root block (SR)::

   ______________________________________________________
  | |SB| | Segment | Segment | Segment | ... | Segment | |
  |_|__|_|____0____|____1____|____2____|_____|____N____|_|
  0 +1K +4K       +8M       +16M      +24M  +(8MB x N)
       .             .            (Typical offsets for 4KB-block)
    .                  .
  .______________________.
  | log | log |... | log |
  |__1__|__2__|____|__m__|
        .       .
      .               .
    .                       .
  .______________________________.
  | Summary | Payload blocks  |SR|
  |_blocks__|_________________|__|

The payload blocks are organized per file, and each file consists of
data blocks and B-tree node blocks::

    |<---       File-A        --->|<---       File-B        --->|
   _______________________________________________________________
    | Data blocks | B-tree blocks | Data blocks | B-tree blocks | ...
   _|_____________|_______________|_____________|_______________|_

Since only the modified blocks are written in the log, it may have
files without data blocks or B-tree node blocks.

The organization of the blocks is recorded in the summary information
blocks, which contains a header structure (nilfs_segment_summary), per
file structures (nilfs_finfo), and per block structures (nilfs_binfo)::

  _________________________________________________________________________
 | Summary | finfo | binfo | ... | binfo | finfo | binfo | ... | binfo |...
 |_blocks__|___A___|_(A,1)_|_____|(A,Na)_|___B___|_(B,1)_|_____|(B,Nb)_|___

The logs include regular files, directory files, symbolic link files
and several meta data files.  The mata data files are the files used
to maintain file system meta data.  The current version of NILFS2 uses
the following meta data files::

 1) Inode file (ifile)             -- Stores on-disk inodes
 2) Checkpoint file (cpfile)       -- Stores checkpoints
 3) Segment usage file (sufile)    -- Stores allocation state of segments
 4) Data address translation file  -- Maps virtual block numbers to usual
    (DAT)                             block numbers.  This file serves to
                                      make on-disk blocks relocatable.

The following figure shows a typical organization of the logs::

  _________________________________________________________________________
 | Summary | regular file | file  | ... | ifile | cpfile | sufile | DAT |SR|
 |_blocks__|_or_directory_|_______|_____|_______|________|________|_____|__|

To stride over segment boundaries, this sequence of files may be split
into multiple logs.  The sequence of logs that should be treated as
logically one log, is delimited with flags marked in the segment
summary.  The recovery code of nilfs2 looks this boundary information
to ensure atomicity of updates.

The super root block is inserted for every checkpoints.  It includes
three special inodes, inodes for the DAT, cpfile, and sufile.  Inodes
of regular files, directories, symlinks and other special files, are
included in the ifile.  The inode of ifile itself is included in the
corresponding checkpoint entry in the cpfile.  Thus, the hierarchy
among NILFS2 files can be depicted as follows::

  Super block (SB)
       |
       v
  Super root block (the latest cno=xx)
       |-- DAT
       |-- sufile
       `-- cpfile
              |-- ifile (cno=c1)
              |-- ifile (cno=c2) ---- file (ino=i1)
              :        :          |-- file (ino=i2)
              `-- ifile (cno=xx)  |-- file (ino=i3)
                                  :        :
                                  `-- file (ino=yy)
                                    ( regular file, directory, or symlink )

For detail on the format of each file, please see nilfs2_ondisk.h
located at include/uapi/linux directory.

There are no patents or other intellectual property that we protect
with regard to the design of NILFS2.  It is allowed to replicate the
design in hopes that other operating systems could share (mount, read,
write, etc.) data stored in this format.


code:
.. SPDX-License-Identifier: GPL-2.0

================================
The Linux NTFS filesystem driver
================================

.. Table of contents

   - Overview
   - Web site
   - Features
   - Supported mount options
   - Known bugs and (mis-)features
   - Using NTFS volume and stripe sets
     - The Device-Mapper driver
     - The Software RAID / MD driver
     - Limitations when using the MD driver

Overview
========

Linux-NTFS comes with a number of user-space programs known as ntfsprogs.
These include mkntfs, a full-featured ntfs filesystem format utility,
ntfsundelete used for recovering files that were unintentionally deleted
from an NTFS volume and ntfsresize which is used to resize an NTFS partition.
See the web site for more information.

To mount an NTFS 1.2/3.x (Windows NT4/2000/XP/2003) volume, use the file
system type 'ntfs'.  The driver currently supports read-only mode (with no
fault-tolerance, encryption or journalling) and very limited, but safe, write
support.

For fault tolerance and raid support (i.e. volume and stripe sets), you can
use the kernel's Software RAID / MD driver.  See section "Using Software RAID
with NTFS" for details.

Web site
========

There is plenty of additional information on the linux-ntfs web site
at http://www.linux-ntfs.org/

The web site has a lot of additional information, such as a comprehensive
FAQ, documentation on the NTFS on-disk format, information on the Linux-NTFS
userspace utilities, etc.

Features
========

- This is a complete rewrite of the NTFS driver that used to be in the 2.4 and
  earlier kernels.  This new driver implements NTFS read support and is
  functionally equivalent to the old ntfs driver and it also implements limited
  write support.  The biggest limitation at present is that files/directories
  cannot be created or deleted.  See below for the list of write features that
  are so far supported.  Another limitation is that writing to compressed files
  is not implemented at all.  Also, neither read nor write access to encrypted
  files is so far implemented.
- The new driver has full support for sparse files on NTFS 3.x volumes which
  the old driver isn't happy with.
- The new driver supports execution of binaries due to mmap() now being
  supported.
- The new driver supports loopback mounting of files on NTFS which is used by
  some Linux distributions to enable the user to run Linux from an NTFS
  partition by creating a large file while in Windows and then loopback
  mounting the file while in Linux and creating a Linux filesystem on it that
  is used to install Linux on it.
- A comparison of the two drivers using::

	time find . -type f -exec md5sum "{}" \;

  run three times in sequence with each driver (after a reboot) on a 1.4GiB
  NTFS partition, showed the new driver to be 20% faster in total time elapsed
  (from 9:43 minutes on average down to 7:53).  The time spent in user space
  was unchanged but the time spent in the kernel was decreased by a factor of
  2.5 (from 85 CPU seconds down to 33).
- The driver does not support short file names in general.  For backwards
  compatibility, we implement access to files using their short file names if
  they exist.  The driver will not create short file names however, and a
  rename will discard any existing short file name.
- The new driver supports exporting of mounted NTFS volumes via NFS.
- The new driver supports async io (aio).
- The new driver supports fsync(2), fdatasync(2), and msync(2).
- The new driver supports readv(2) and writev(2).
- The new driver supports access time updates (including mtime and ctime).
- The new driver supports truncate(2) and open(2) with O_TRUNC.  But at present
  only very limited support for highly fragmented files, i.e. ones which have
  their data attribute split across multiple extents, is included.  Another
  limitation is that at present truncate(2) will never create sparse files,
  since to mark a file sparse we need to modify the directory entry for the
  file and we do not implement directory modifications yet.
- The new driver supports write(2) which can both overwrite existing data and
  extend the file size so that you can write beyond the existing data.  Also,
  writing into sparse regions is supported and the holes are filled in with
  clusters.  But at present only limited support for highly fragmented files,
  i.e. ones which have their data attribute split across multiple extents, is
  included.  Another limitation is that write(2) will never create sparse
  files, since to mark a file sparse we need to modify the directory entry for
  the file and we do not implement directory modifications yet.

Supported mount options
=======================

In addition to the generic mount options described by the manual page for the
mount command (man 8 mount, also see man 5 fstab), the NTFS driver supports the
following mount options:

======================= =======================================================
iocharset=name		Deprecated option.  Still supported but please use
			nls=name in the future.  See description for nls=name.

nls=name		Character set to use when returning file names.
			Unlike VFAT, NTFS suppresses names that contain
			unconvertible characters.  Note that most character
			sets contain insufficient characters to represent all
			possible Unicode characters that can exist on NTFS.
			To be sure you are not missing any files, you are
			advised to use nls=utf8 which is capable of
			representing all Unicode characters.

utf8=<bool>		Option no longer supported.  Currently mapped to
			nls=utf8 but please use nls=utf8 in the future and
			make sure utf8 is compiled either as module or into
			the kernel.  See description for nls=name.

uid=
gid=
umask=			Provide default owner, group, and access mode mask.
			These options work as documented in mount(8).  By
			default, the files/directories are owned by root and
			he/she has read and write permissions, as well as
			browse permission for directories.  No one else has any
			access permissions.  I.e. the mode on all files is by
			default rw------- and for directories rwx------, a
			consequence of the default fmask=0177 and dmask=0077.
			Using a umask of zero will grant all permissions to
			everyone, i.e. all files and directories will have mode
			rwxrwxrwx.

fmask=
dmask=			Instead of specifying umask which applies both to
			files and directories, fmask applies only to files and
			dmask only to directories.

sloppy=<BOOL>		If sloppy is specified, ignore unknown mount options.
			Otherwise the default behaviour is to abort mount if
			any unknown options are found.

show_sys_files=<BOOL>	If show_sys_files is specified, show the system files
			in directory listings.  Otherwise the default behaviour
			is to hide the system files.
			Note that even when show_sys_files is specified, "$MFT"
			will not be visible due to bugs/mis-features in glibc.
			Further, note that irrespective of show_sys_files, all
			files are accessible by name, i.e. you can always do
			"ls -l \$UpCase" for example to specifically show the
			system file containing the Unicode upcase table.

case_sensitive=<BOOL>	If case_sensitive is specified, treat all file names as
			case sensitive and create file names in the POSIX
			namespace.  Otherwise the default behaviour is to treat
			file names as case insensitive and to create file names
			in the WIN32/LONG name space.  Note, the Linux NTFS
			driver will never create short file names and will
			remove them on rename/delete of the corresponding long
			file name.
			Note that files remain accessible via their short file
			name, if it exists.  If case_sensitive, you will need
			to provide the correct case of the short file name.

disable_sparse=<BOOL>	If disable_sparse is specified, creation of sparse
			regions, i.e. holes, inside files is disabled for the
			volume (for the duration of this mount only).  By
			default, creation of sparse regions is enabled, which
			is consistent with the behaviour of traditional Unix
			filesystems.

errors=opt		What to do when critical filesystem errors are found.
			Following values can be used for "opt":

			  ========  =========================================
			  continue  DEFAULT, try to clean-up as much as
				    possible, e.g. marking a corrupt inode as
				    bad so it is no longer accessed, and then
				    continue.
			  recover   At present only supported is recovery of
				    the boot sector from the backup copy.
				    If read-only mount, the recovery is done
				    in memory only and not written to disk.
			  ========  =========================================

			Note that the options are additive, i.e. specifying::

			   errors=continue,errors=recover

			means the driver will attempt to recover and if that
			fails it will clean-up as much as possible and
			continue.

mft_zone_multiplier=	Set the MFT zone multiplier for the volume (this
			setting is not persistent across mounts and can be
			changed from mount to mount but cannot be changed on
			remount).  Values of 1 to 4 are allowed, 1 being the
			default.  The MFT zone multiplier determines how much
			space is reserved for the MFT on the volume.  If all
			other space is used up, then the MFT zone will be
			shrunk dynamically, so this has no impact on the
			amount of free space.  However, it can have an impact
			on performance by affecting fragmentation of the MFT.
			In general use the default.  If you have a lot of small
			files then use a higher value.  The values have the
			following meaning:

			      =====	    =================================
			      Value	     MFT zone size (% of volume size)
			      =====	    =================================
				1		12.5%
				2		25%
				3		37.5%
				4		50%
			      =====	    =================================

			Note this option is irrelevant for read-only mounts.
======================= =======================================================

Known bugs and (mis-)features
=============================

- The link count on each directory inode entry is set to 1, due to Linux not
  supporting directory hard links.  This may well confuse some user space
  applications, since the directory names will have the same inode numbers.
  This also speeds up ntfs_read_inode() immensely.  And we haven't found any
  problems with this approach so far.  If you find a problem with this, please
  let us know.

Please send bug reports/comments/feedback/abuse to the Linux-NTFS development
list at sourceforge: linux-ntfs-dev@lists.sourceforge.net

Using NTFS volume and stripe sets
=================================

For support of volume and stripe sets, you can either use the kernel's
Device-Mapper driver or the kernel's Software RAID / MD driver.  The former is
the recommended one to use for linear raid.  But the latter is required for
raid level 5.  For striping and mirroring, either driver should work fine.

The Device-Mapper driver
------------------------

You will need to create a table of the components of the volume/stripe set and
how they fit together and load this into the kernel using the dmsetup utility
(see man 8 dmsetup).

Linear volume sets, i.e. linear raid, has been tested and works fine.  Even
though untested, there is no reason why stripe sets, i.e. raid level 0, and
mirrors, i.e. raid level 1 should not work, too.  Stripes with parity, i.e.
raid level 5, unfortunately cannot work yet because the current version of the
Device-Mapper driver does not support raid level 5.  You may be able to use the
Software RAID / MD driver for raid level 5, see the next section for details.

To create the table describing your volume you will need to know each of its
components and their sizes in sectors, i.e. multiples of 512-byte blocks.

For NT4 fault tolerant volumes you can obtain the sizes using fdisk.  So for
example if one of your partitions is /dev/hda2 you would do::

    $ fdisk -ul /dev/hda

    Disk /dev/hda: 81.9 GB, 81964302336 bytes
    255 heads, 63 sectors/track, 9964 cylinders, total 160086528 sectors
    Units = sectors of 1 * 512 = 512 bytes

	Device Boot      Start         End      Blocks   Id  System
	/dev/hda1   *          63     4209029     2104483+  83  Linux
	/dev/hda2         4209030    37768814    16779892+  86  NTFS
	/dev/hda3        37768815    46170809     4200997+  83  Linux

And you would know that /dev/hda2 has a size of 37768814 - 4209030 + 1 =
33559785 sectors.

For Win2k and later dynamic disks, you can for example use the ldminfo utility
which is part of the Linux LDM tools (the latest version at the time of
writing is linux-ldm-0.0.8.tar.bz2).  You can download it from:

	http://www.linux-ntfs.org/

Simply extract the downloaded archive (tar xvjf linux-ldm-0.0.8.tar.bz2), go
into it (cd linux-ldm-0.0.8) and change to the test directory (cd test).  You
will find the precompiled (i386) ldminfo utility there.  NOTE: You will not be
able to compile this yourself easily so use the binary version!

Then you would use ldminfo in dump mode to obtain the necessary information::

    $ ./ldminfo --dump /dev/hda

This would dump the LDM database found on /dev/hda which describes all of your
dynamic disks and all the volumes on them.  At the bottom you will see the
VOLUME DEFINITIONS section which is all you really need.  You may need to look
further above to determine which of the disks in the volume definitions is
which device in Linux.  Hint: Run ldminfo on each of your dynamic disks and
look at the Disk Id close to the top of the output for each (the PRIVATE HEADER
section).  You can then find these Disk Ids in the VBLK DATABASE section in the
<Disk> components where you will get the LDM Name for the disk that is found in
the VOLUME DEFINITIONS section.

Note you will also need to enable the LDM driver in the Linux kernel.  If your
distribution did not enable it, you will need to recompile the kernel with it
enabled.  This will create the LDM partitions on each device at boot time.  You
would then use those devices (for /dev/hda they would be /dev/hda1, 2, 3, etc)
in the Device-Mapper table.

You can also bypass using the LDM driver by using the main device (e.g.
/dev/hda) and then using the offsets of the LDM partitions into this device as
the "Start sector of device" when creating the table.  Once again ldminfo would
give you the correct information to do this.

Assuming you know all your devices and their sizes things are easy.

For a linear raid the table would look like this (note all values are in
512-byte sectors)::

    # Offset into	Size of this	Raid type	Device		Start sector
    # volume	device						of device
    0		1028161		linear		/dev/hda1	0
    1028161		3903762		linear		/dev/hdb2	0
    4931923		2103211		linear		/dev/hdc1	0

For a striped volume, i.e. raid level 0, you will need to know the chunk size
you used when creating the volume.  Windows uses 64kiB as the default, so it
will probably be this unless you changes the defaults when creating the array.

For a raid level 0 the table would look like this (note all values are in
512-byte sectors)::

    # Offset   Size	    Raid     Number   Chunk  1st        Start	2nd	  Start
    # into     of the   type     of	      size   Device	in	Device	  in
    # volume   volume	     stripes			device		  device
    0	   2056320  striped  2	      128    /dev/hda1	0	/dev/hdb1 0

If there are more than two devices, just add each of them to the end of the
line.

Finally, for a mirrored volume, i.e. raid level 1, the table would look like
this (note all values are in 512-byte sectors)::

    # Ofs Size   Raid   Log  Number Region Should Number Source  Start Target Start
    # in  of the type   type of log size   sync?  of     Device  in    Device in
    # vol volume		 params		     mirrors	     Device	  Device
    0    2056320 mirror core 2	16     nosync 2	   /dev/hda1 0   /dev/hdb1 0

If you are mirroring to multiple devices you can specify further targets at the
end of the line.

Note the "Should sync?" parameter "nosync" means that the two mirrors are
already in sync which will be the case on a clean shutdown of Windows.  If the
mirrors are not clean, you can specify the "sync" option instead of "nosync"
and the Device-Mapper driver will then copy the entirety of the "Source Device"
to the "Target Device" or if you specified multiple target devices to all of
them.

Once you have your table, save it in a file somewhere (e.g. /etc/ntfsvolume1),
and hand it over to dmsetup to work with, like so::

    $ dmsetup create myvolume1 /etc/ntfsvolume1

You can obviously replace "myvolume1" with whatever name you like.

If it all worked, you will now have the device /dev/device-mapper/myvolume1
which you can then just use as an argument to the mount command as usual to
mount the ntfs volume.  For example::

    $ mount -t ntfs -o ro /dev/device-mapper/myvolume1 /mnt/myvol1

(You need to create the directory /mnt/myvol1 first and of course you can use
anything you like instead of /mnt/myvol1 as long as it is an existing
directory.)

It is advisable to do the mount read-only to see if the volume has been setup
correctly to avoid the possibility of causing damage to the data on the ntfs
volume.

The Software RAID / MD driver
-----------------------------

An alternative to using the Device-Mapper driver is to use the kernel's
Software RAID / MD driver.  For which you need to set up your /etc/raidtab
appropriately (see man 5 raidtab).

Linear volume sets, i.e. linear raid, as well as stripe sets, i.e. raid level
0, have been tested and work fine (though see section "Limitations when using
the MD driver with NTFS volumes" especially if you want to use linear raid).
Even though untested, there is no reason why mirrors, i.e. raid level 1, and
stripes with parity, i.e. raid level 5, should not work, too.

You have to use the "persistent-superblock 0" option for each raid-disk in the
NTFS volume/stripe you are configuring in /etc/raidtab as the persistent
superblock used by the MD driver would damage the NTFS volume.

Windows by default uses a stripe chunk size of 64k, so you probably want the
"chunk-size 64k" option for each raid-disk, too.

For example, if you have a stripe set consisting of two partitions /dev/hda5
and /dev/hdb1 your /etc/raidtab would look like this::

    raiddev /dev/md0
	    raid-level	0
	    nr-raid-disks	2
	    nr-spare-disks	0
	    persistent-superblock	0
	    chunk-size	64k
	    device		/dev/hda5
	    raid-disk	0
	    device		/dev/hdb1
	    raid-disk	1

For linear raid, just change the raid-level above to "raid-level linear", for
mirrors, change it to "raid-level 1", and for stripe sets with parity, change
it to "raid-level 5".

Note for stripe sets with parity you will also need to tell the MD driver
which parity algorithm to use by specifying the option "parity-algorithm
which", where you need to replace "which" with the name of the algorithm to
use (see man 5 raidtab for available algorithms) and you will have to try the
different available algorithms until you find one that works.  Make sure you
are working read-only when playing with this as you may damage your data
otherwise.  If you find which algorithm works please let us know (email the
linux-ntfs developers list linux-ntfs-dev@lists.sourceforge.net or drop in on
IRC in channel #ntfs on the irc.freenode.net network) so we can update this
documentation.

Once the raidtab is setup, run for example raid0run -a to start all devices or
raid0run /dev/md0 to start a particular md device, in this case /dev/md0.

Then just use the mount command as usual to mount the ntfs volume using for
example::

    mount -t ntfs -o ro /dev/md0 /mnt/myntfsvolume

It is advisable to do the mount read-only to see if the md volume has been
setup correctly to avoid the possibility of causing damage to the data on the
ntfs volume.

Limitations when using the Software RAID / MD driver
-----------------------------------------------------

Using the md driver will not work properly if any of your NTFS partitions have
an odd number of sectors.  This is especially important for linear raid as all
data after the first partition with an odd number of sectors will be offset by
one or more sectors so if you mount such a partition with write support you
will cause massive damage to the data on the volume which will only become
apparent when you try to use the volume again under Windows.

So when using linear raid, make sure that all your partitions have an even
number of sectors BEFORE attempting to use it.  You have been warned!

Even better is to simply use the Device-Mapper for linear raid and then you do
not have this problem with odd numbers of sectors.


code:
.. SPDX-License-Identifier: GPL-2.0

=====
NTFS3
=====

Summary and Features
====================

NTFS3 is fully functional NTFS Read-Write driver. The driver works with NTFS
versions up to 3.1. File system type to use on mount is *ntfs3*.

- This driver implements NTFS read/write support for normal, sparse and
  compressed files.
- Supports native journal replaying.
- Supports NFS export of mounted NTFS volumes.
- Supports extended attributes. Predefined extended attributes:

	- *system.ntfs_security* gets/sets security

		Descriptor: SECURITY_DESCRIPTOR_RELATIVE

	- *system.ntfs_attrib* gets/sets ntfs file/dir attributes.

	  Note: Applied to empty files, this allows to switch type between
	  sparse(0x200), compressed(0x800) and normal.

Mount Options
=============

The list below describes mount options supported by NTFS3 driver in addition to
generic ones. You can use every mount option with **no** option. If it is in
this table marked with no it means default is without **no**.

.. flat-table::
   :widths: 1 5
   :fill-cells:

   * - iocharset=name
     - This option informs the driver how to interpret path strings and
       translate them to Unicode and back. If this option is not set, the
       default codepage will be used (CONFIG_NLS_DEFAULT).

       Example: iocharset=utf8

   * - uid=
     - :rspan:`1`
   * - gid=

   * - umask=
     - Controls the default permissions for files/directories created after
       the NTFS volume is mounted.

   * - dmask=
     - :rspan:`1` Instead of specifying umask which applies both to files and
       directories, fmask applies only to files and dmask only to directories.
   * - fmask=

   * - noacsrules
     - "No access rules" mount option sets access rights for files/folders to
       777 and owner/group to root. This mount option absorbs all other
       permissions.

       - Permissions change for files/folders will be reported as successful,
	 but they will remain 777.

       - Owner/group change will be reported as successful, butthey will stay
	 as root.

   * - nohidden
     - Files with the Windows-specific HIDDEN (FILE_ATTRIBUTE_HIDDEN) attribute
       will not be shown under Linux.

   * - sys_immutable
     - Files with the Windows-specific SYSTEM (FILE_ATTRIBUTE_SYSTEM) attribute
       will be marked as system immutable files.

   * - discard
     - Enable support of the TRIM command for improved performance on delete
       operations, which is recommended for use with the solid-state drives
       (SSD).

   * - force
     - Forces the driver to mount partitions even if volume is marked dirty.
       Not recommended for use.

   * - sparse
     - Create new files as sparse.

   * - showmeta
     - Use this parameter to show all meta-files (System Files) on a mounted
       NTFS partition. By default, all meta-files are hidden.

   * - prealloc
     - Preallocate space for files excessively when file size is increasing on
       writes. Decreases fragmentation in case of parallel write operations to
       different files.

   * - acl
     - Support POSIX ACLs (Access Control Lists). Effective if supported by
       Kernel. Not to be confused with NTFS ACLs. The option specified as acl
       enables support for POSIX ACLs.

Todo list
=========
- Full journaling support over JBD. Currently journal replaying is supported
  which is not necessarily as effectice as JBD would be.

References
==========
- Commercial version of the NTFS driver for Linux.
	https://www.paragon-software.com/home/ntfs-linux-professional/

- Direct e-mail address for feedback and requests on the NTFS3 implementation.
	almaz.alexandrovich@paragon-software.com


code:
.. SPDX-License-Identifier: GPL-2.0

=====================================
OCFS2 file system - online file check
=====================================

This document will describe OCFS2 online file check feature.

Introduction
============
OCFS2 is often used in high-availability systems. However, OCFS2 usually
converts the filesystem to read-only when encounters an error. This may not be
necessary, since turning the filesystem read-only would affect other running
processes as well, decreasing availability.
Then, a mount option (errors=continue) is introduced, which would return the
-EIO errno to the calling process and terminate further processing so that the
filesystem is not corrupted further. The filesystem is not converted to
read-only, and the problematic file's inode number is reported in the kernel
log. The user can try to check/fix this file via online filecheck feature.

Scope
=====
This effort is to check/fix small issues which may hinder day-to-day operations
of a cluster filesystem by turning the filesystem read-only. The scope of
checking/fixing is at the file level, initially for regular files and eventually
to all files (including system files) of the filesystem.

In case of directory to file links is incorrect, the directory inode is
reported as erroneous.

This feature is not suited for extravagant checks which involve dependency of
other components of the filesystem, such as but not limited to, checking if the
bits for file blocks in the allocation has been set. In case of such an error,
the offline fsck should/would be recommended.

Finally, such an operation/feature should not be automated lest the filesystem
may end up with more damage than before the repair attempt. So, this has to
be performed using user interaction and consent.

User interface
==============
When there are errors in the OCFS2 filesystem, they are usually accompanied
by the inode number which caused the error. This inode number would be the
input to check/fix the file.

There is a sysfs directory for each OCFS2 file system mounting::

  /sys/fs/ocfs2/<devname>/filecheck

Here, <devname> indicates the name of OCFS2 volume device which has been already
mounted. The file above would accept inode numbers. This could be used to
communicate with kernel space, tell which file(inode number) will be checked or
fixed. Currently, three operations are supported, which includes checking
inode, fixing inode and setting the size of result record history.

1. If you want to know what error exactly happened to <inode> before fixing, do::

    # echo "<inode>" > /sys/fs/ocfs2/<devname>/filecheck/check
    # cat /sys/fs/ocfs2/<devname>/filecheck/check

The output is like this::

    INO		DONE	ERROR
    39502		1	GENERATION

    <INO> lists the inode numbers.
    <DONE> indicates whether the operation has been finished.
    <ERROR> says what kind of errors was found. For the detailed error numbers,
    please refer to the file linux/fs/ocfs2/filecheck.h.

2. If you determine to fix this inode, do::

    # echo "<inode>" > /sys/fs/ocfs2/<devname>/filecheck/fix
    # cat /sys/fs/ocfs2/<devname>/filecheck/fix

The output is like this:::

    INO		DONE	ERROR
    39502		1	SUCCESS

This time, the <ERROR> column indicates whether this fix is successful or not.

3. The record cache is used to store the history of check/fix results. It's
default size is 10, and can be adjust between the range of 10 ~ 100. You can
adjust the size like this::

  # echo "<size>" > /sys/fs/ocfs2/<devname>/filecheck/set

Fixing stuff
============
On receiving the inode, the filesystem would read the inode and the
file metadata. In case of errors, the filesystem would fix the errors
and report the problems it fixed in the kernel log. As a precautionary measure,
the inode must first be checked for errors before performing a final fix.

The inode and the result history will be maintained temporarily in a
small linked list buffer which would contain the last (N) inodes
fixed/checked, the detailed errors which were fixed/checked are printed in the
kernel log.


code:
.. SPDX-License-Identifier: GPL-2.0

================
OCFS2 filesystem
================

OCFS2 is a general purpose extent based shared disk cluster file
system with many similarities to ext3. It supports 64 bit inode
numbers, and has automatically extending metadata groups which may
also make it attractive for non-clustered use.

You'll want to install the ocfs2-tools package in order to at least
get "mount.ocfs2" and "ocfs2_hb_ctl".

Project web page:    http://ocfs2.wiki.kernel.org
Tools git tree:      https://github.com/markfasheh/ocfs2-tools
OCFS2 mailing lists: https://oss.oracle.com/projects/ocfs2/mailman/

All code copyright 2005 Oracle except when otherwise noted.

Credits
=======

Lots of code taken from ext3 and other projects.

Authors in alphabetical order:

- Joel Becker   <joel.becker@oracle.com>
- Zach Brown    <zach.brown@oracle.com>
- Mark Fasheh   <mfasheh@suse.com>
- Kurt Hackel   <kurt.hackel@oracle.com>
- Tao Ma        <tao.ma@oracle.com>
- Sunil Mushran <sunil.mushran@oracle.com>
- Manish Singh  <manish.singh@oracle.com>
- Tiger Yang    <tiger.yang@oracle.com>

Caveats
=======
Features which OCFS2 does not support yet:

	- Directory change notification (F_NOTIFY)
	- Distributed Caching (F_SETLEASE/F_GETLEASE/break_lease)

Mount options
=============

OCFS2 supports the following mount options:

(*) == default

======================= ========================================================
barrier=1		This enables/disables barriers. barrier=0 disables it,
			barrier=1 enables it.
errors=remount-ro(*)	Remount the filesystem read-only on an error.
errors=panic		Panic and halt the machine if an error occurs.
intr		(*)	Allow signals to interrupt cluster operations.
nointr			Do not allow signals to interrupt cluster
			operations.
noatime			Do not update access time.
relatime(*)		Update atime if the previous atime is older than
			mtime or ctime
strictatime		Always update atime, but the minimum update interval
			is specified by atime_quantum.
atime_quantum=60(*)	OCFS2 will not update atime unless this number
			of seconds has passed since the last update.
			Set to zero to always update atime. This option need
			work with strictatime.
data=ordered	(*)	All data are forced directly out to the main file
			system prior to its metadata being committed to the
			journal.
data=writeback		Data ordering is not preserved, data may be written
			into the main file system after its metadata has been
			committed to the journal.
preferred_slot=0(*)	During mount, try to use this filesystem slot first. If
			it is in use by another node, the first empty one found
			will be chosen. Invalid values will be ignored.
commit=nrsec	(*)	Ocfs2 can be told to sync all its data and metadata
			every 'nrsec' seconds. The default value is 5 seconds.
			This means that if you lose your power, you will lose
			as much as the latest 5 seconds of work (your
			filesystem will not be damaged though, thanks to the
			journaling).  This default value (or any low value)
			will hurt performance, but it's good for data-safety.
			Setting it to 0 will have the same effect as leaving
			it at the default (5 seconds).
			Setting it to very large values will improve
			performance.
localalloc=8(*)		Allows custom localalloc size in MB. If the value is too
			large, the fs will silently revert it to the default.
localflocks		This disables cluster aware flock.
inode64			Indicates that Ocfs2 is allowed to create inodes at
			any location in the filesystem, including those which
			will result in inode numbers occupying more than 32
			bits of significance.
user_xattr	(*)	Enables Extended User Attributes.
nouser_xattr		Disables Extended User Attributes.
acl			Enables POSIX Access Control Lists support.
noacl		(*)	Disables POSIX Access Control Lists support.
resv_level=2	(*)	Set how aggressive allocation reservations will be.
			Valid values are between 0 (reservations off) to 8
			(maximum space for reservations).
dir_resv_level=	(*)	By default, directory reservations will scale with file
			reservations - users should rarely need to change this
			value. If allocation reservations are turned off, this
			option will have no effect.
coherency=full  (*)	Disallow concurrent O_DIRECT writes, cluster inode
			lock will be taken to force other nodes drop cache,
			therefore full cluster coherency is guaranteed even
			for O_DIRECT writes.
coherency=buffered	Allow concurrent O_DIRECT writes without EX lock among
			nodes, which gains high performance at risk of getting
			stale data on other nodes.
journal_async_commit	Commit block can be written to disk without waiting
			for descriptor blocks. If enabled older kernels cannot
			mount the device. This will enable 'journal_checksum'
			internally.
======================= ========================================================


code:
.. SPDX-License-Identifier: GPL-2.0

================================
Optimized MPEG Filesystem (OMFS)
================================

Overview
========

OMFS is a filesystem created by SonicBlue for use in the ReplayTV DVR
and Rio Karma MP3 player.  The filesystem is extent-based, utilizing
block sizes from 2k to 8k, with hash-based directories.  This
filesystem driver may be used to read and write disks from these
devices.

Note, it is not recommended that this FS be used in place of a general
filesystem for your own streaming media device.  Native Linux filesystems
will likely perform better.

More information is available at:

    http://linux-karma.sf.net/

Various utilities, including mkomfs and omfsck, are included with
omfsprogs, available at:

    https://bobcopeland.com/karma/

Instructions are included in its README.

Options
=======

OMFS supports the following mount-time options:

    ============   ========================================
    uid=n          make all files owned by specified user
    gid=n          make all files owned by specified group
    umask=xxx      set permission umask to xxx
    fmask=xxx      set umask to xxx for files
    dmask=xxx      set umask to xxx for directories
    ============   ========================================

Disk format
===========

OMFS discriminates between "sysblocks" and normal data blocks.  The sysblock
group consists of super block information, file metadata, directory structures,
and extents.  Each sysblock has a header containing CRCs of the entire
sysblock, and may be mirrored in successive blocks on the disk.  A sysblock may
have a smaller size than a data block, but since they are both addressed by the
same 64-bit block number, any remaining space in the smaller sysblock is
unused.

Sysblock header information::

    struct omfs_header {
	    __be64 h_self;                  /* FS block where this is located */
	    __be32 h_body_size;             /* size of useful data after header */
	    __be16 h_crc;                   /* crc-ccitt of body_size bytes */
	    char h_fill1[2];
	    u8 h_version;                   /* version, always 1 */
	    char h_type;                    /* OMFS_INODE_X */
	    u8 h_magic;                     /* OMFS_IMAGIC */
	    u8 h_check_xor;                 /* XOR of header bytes before this */
	    __be32 h_fill2;
    };

Files and directories are both represented by omfs_inode::

    struct omfs_inode {
	    struct omfs_header i_head;      /* header */
	    __be64 i_parent;                /* parent containing this inode */
	    __be64 i_sibling;               /* next inode in hash bucket */
	    __be64 i_ctime;                 /* ctime, in milliseconds */
	    char i_fill1[35];
	    char i_type;                    /* OMFS_[DIR,FILE] */
	    __be32 i_fill2;
	    char i_fill3[64];
	    char i_name[OMFS_NAMELEN];      /* filename */
	    __be64 i_size;                  /* size of file, in bytes */
    };

Directories in OMFS are implemented as a large hash table.  Filenames are
hashed then prepended into the bucket list beginning at OMFS_DIR_START.
Lookup requires hashing the filename, then seeking across i_sibling pointers
until a match is found on i_name.  Empty buckets are represented by block
pointers with all-1s (~0).

A file is an omfs_inode structure followed by an extent table beginning at
OMFS_EXTENT_START::

    struct omfs_extent_entry {
	    __be64 e_cluster;               /* start location of a set of blocks */
	    __be64 e_blocks;                /* number of blocks after e_cluster */
    };

    struct omfs_extent {
	    __be64 e_next;                  /* next extent table location */
	    __be32 e_extent_count;          /* total # extents in this table */
	    __be32 e_fill;
	    struct omfs_extent_entry e_entry;       /* start of extent entries */
    };

Each extent holds the block offset followed by number of blocks allocated to
the extent.  The final extent in each table is a terminator with e_cluster
being ~0 and e_blocks being ones'-complement of the total number of blocks
in the table.

If this table overflows, a continuation inode is written and pointed to by
e_next.  These have a header but lack the rest of the inode structure.


code:
.. SPDX-License-Identifier: GPL-2.0

========
ORANGEFS
========

OrangeFS is an LGPL userspace scale-out parallel storage system. It is ideal
for large storage problems faced by HPC, BigData, Streaming Video,
Genomics, Bioinformatics.

Orangefs, originally called PVFS, was first developed in 1993 by
Walt Ligon and Eric Blumer as a parallel file system for Parallel
Virtual Machine (PVM) as part of a NASA grant to study the I/O patterns
of parallel programs.

Orangefs features include:

  * Distributes file data among multiple file servers
  * Supports simultaneous access by multiple clients
  * Stores file data and metadata on servers using local file system
    and access methods
  * Userspace implementation is easy to install and maintain
  * Direct MPI support
  * Stateless

Mailing List Archives
=====================

http://lists.orangefs.org/pipermail/devel_lists.orangefs.org/

Mailing List Submissions
========================

devel@lists.orangefs.org

Documentation
=============

http://www.orangefs.org/documentation/

Running ORANGEFS On a Single Server
===================================

OrangeFS is usually run in large installations with multiple servers and
clients, but a complete filesystem can be run on a single machine for
development and testing.

On Fedora, install orangefs and orangefs-server::

    dnf -y install orangefs orangefs-server

There is an example server configuration file in
/etc/orangefs/orangefs.conf.  Change localhost to your hostname if
necessary.

To generate a filesystem to run xfstests against, see below.

There is an example client configuration file in /etc/pvfs2tab.  It is a
single line.  Uncomment it and change the hostname if necessary.  This
controls clients which use libpvfs2.  This does not control the
pvfs2-client-core.

Create the filesystem::

    pvfs2-server -f /etc/orangefs/orangefs.conf

Start the server::

    systemctl start orangefs-server

Test the server::

    pvfs2-ping -m /pvfsmnt

Start the client.  The module must be compiled in or loaded before this
point::

    systemctl start orangefs-client

Mount the filesystem::

    mount -t pvfs2 tcp://localhost:3334/orangefs /pvfsmnt

Userspace Filesystem Source
===========================

http://www.orangefs.org/download

Orangefs versions prior to 2.9.3 would not be compatible with the
upstream version of the kernel client.

Building ORANGEFS on a Single Server
====================================

Where OrangeFS cannot be installed from distribution packages, it may be
built from source.

You can omit --prefix if you don't care that things are sprinkled around
in /usr/local.  As of version 2.9.6, OrangeFS uses Berkeley DB by
default, we will probably be changing the default to LMDB soon.

::

    ./configure --prefix=/opt/ofs --with-db-backend=lmdb --disable-usrint

    make

    make install

Create an orangefs config file by running pvfs2-genconfig and
specifying a target config file. Pvfs2-genconfig will prompt you
through. Generally it works fine to take the defaults, but you
should use your server's hostname, rather than "localhost" when
it comes to that question::

    /opt/ofs/bin/pvfs2-genconfig /etc/pvfs2.conf

Create an /etc/pvfs2tab file (localhost is fine)::

    echo tcp://localhost:3334/orangefs /pvfsmnt pvfs2 defaults,noauto 0 0 > \
	/etc/pvfs2tab

Create the mount point you specified in the tab file if needed::

    mkdir /pvfsmnt

Bootstrap the server::

    /opt/ofs/sbin/pvfs2-server -f /etc/pvfs2.conf

Start the server::

    /opt/ofs/sbin/pvfs2-server /etc/pvfs2.conf

Now the server should be running. Pvfs2-ls is a simple
test to verify that the server is running::

    /opt/ofs/bin/pvfs2-ls /pvfsmnt

If stuff seems to be working, load the kernel module and
turn on the client core::

    /opt/ofs/sbin/pvfs2-client -p /opt/ofs/sbin/pvfs2-client-core

Mount your filesystem::

    mount -t pvfs2 tcp://`hostname`:3334/orangefs /pvfsmnt

Running xfstests
================

It is useful to use a scratch filesystem with xfstests.  This can be
done with only one server.

Make a second copy of the FileSystem section in the server configuration
file, which is /etc/orangefs/orangefs.conf.  Change the Name to scratch.
Change the ID to something other than the ID of the first FileSystem
section (2 is usually a good choice).

Then there are two FileSystem sections: orangefs and scratch.

This change should be made before creating the filesystem.

::

    pvfs2-server -f /etc/orangefs/orangefs.conf

To run xfstests, create /etc/xfsqa.config::

    TEST_DIR=/orangefs
    TEST_DEV=tcp://localhost:3334/orangefs
    SCRATCH_MNT=/scratch
    SCRATCH_DEV=tcp://localhost:3334/scratch

Then xfstests can be run::

    ./check -pvfs2

Options
=======

The following mount options are accepted:

  acl
    Allow the use of Access Control Lists on files and directories.

  intr
    Some operations between the kernel client and the user space
    filesystem can be interruptible, such as changes in debug levels
    and the setting of tunable parameters.

  local_lock
    Enable posix locking from the perspective of "this" kernel. The
    default file_operations lock action is to return ENOSYS. Posix
    locking kicks in if the filesystem is mounted with -o local_lock.
    Distributed locking is being worked on for the future.

Debugging
=========

If you want the debug (GOSSIP) statements in a particular
source file (inode.c for example) go to syslog::

  echo inode > /sys/kernel/debug/orangefs/kernel-debug

No debugging (the default)::

  echo none > /sys/kernel/debug/orangefs/kernel-debug

Debugging from several source files::

  echo inode,dir > /sys/kernel/debug/orangefs/kernel-debug

All debugging::

  echo all > /sys/kernel/debug/orangefs/kernel-debug

Get a list of all debugging keywords::

  cat /sys/kernel/debug/orangefs/debug-help

Protocol between Kernel Module and Userspace
============================================

Orangefs is a user space filesystem and an associated kernel module.
We'll just refer to the user space part of Orangefs as "userspace"
from here on out. Orangefs descends from PVFS, and userspace code
still uses PVFS for function and variable names. Userspace typedefs
many of the important structures. Function and variable names in
the kernel module have been transitioned to "orangefs", and The Linux
Coding Style avoids typedefs, so kernel module structures that
correspond to userspace structures are not typedefed.

The kernel module implements a pseudo device that userspace
can read from and write to. Userspace can also manipulate the
kernel module through the pseudo device with ioctl.

The Bufmap
----------

At startup userspace allocates two page-size-aligned (posix_memalign)
mlocked memory buffers, one is used for IO and one is used for readdir
operations. The IO buffer is 41943040 bytes and the readdir buffer is
4194304 bytes. Each buffer contains logical chunks, or partitions, and
a pointer to each buffer is added to its own PVFS_dev_map_desc structure
which also describes its total size, as well as the size and number of
the partitions.

A pointer to the IO buffer's PVFS_dev_map_desc structure is sent to a
mapping routine in the kernel module with an ioctl. The structure is
copied from user space to kernel space with copy_from_user and is used
to initialize the kernel module's "bufmap" (struct orangefs_bufmap), which
then contains:

  * refcnt
    - a reference counter
  * desc_size - PVFS2_BUFMAP_DEFAULT_DESC_SIZE (4194304) - the IO buffer's
    partition size, which represents the filesystem's block size and
    is used for s_blocksize in super blocks.
  * desc_count - PVFS2_BUFMAP_DEFAULT_DESC_COUNT (10) - the number of
    partitions in the IO buffer.
  * desc_shift - log2(desc_size), used for s_blocksize_bits in super blocks.
  * total_size - the total size of the IO buffer.
  * page_count - the number of 4096 byte pages in the IO buffer.
  * page_array - a pointer to ``page_count * (sizeof(struct page*))`` bytes
    of kcalloced memory. This memory is used as an array of pointers
    to each of the pages in the IO buffer through a call to get_user_pages.
  * desc_array - a pointer to ``desc_count * (sizeof(struct orangefs_bufmap_desc))``
    bytes of kcalloced memory. This memory is further intialized:

      user_desc is the kernel's copy of the IO buffer's ORANGEFS_dev_map_desc
      structure. user_desc->ptr points to the IO buffer.

      ::

	pages_per_desc = bufmap->desc_size / PAGE_SIZE
	offset = 0

        bufmap->desc_array[0].page_array = &bufmap->page_array[offset]
        bufmap->desc_array[0].array_count = pages_per_desc = 1024
        bufmap->desc_array[0].uaddr = (user_desc->ptr) + (0 * 1024 * 4096)
        offset += 1024
                           .
                           .
                           .
        bufmap->desc_array[9].page_array = &bufmap->page_array[offset]
        bufmap->desc_array[9].array_count = pages_per_desc = 1024
        bufmap->desc_array[9].uaddr = (user_desc->ptr) +
                                               (9 * 1024 * 4096)
        offset += 1024

  * buffer_index_array - a desc_count sized array of ints, used to
    indicate which of the IO buffer's partitions are available to use.
  * buffer_index_lock - a spinlock to protect buffer_index_array during update.
  * readdir_index_array - a five (ORANGEFS_READDIR_DEFAULT_DESC_COUNT) element
    int array used to indicate which of the readdir buffer's partitions are
    available to use.
  * readdir_index_lock - a spinlock to protect readdir_index_array during
    update.

Operations
----------

The kernel module builds an "op" (struct orangefs_kernel_op_s) when it
needs to communicate with userspace. Part of the op contains the "upcall"
which expresses the request to userspace. Part of the op eventually
contains the "downcall" which expresses the results of the request.

The slab allocator is used to keep a cache of op structures handy.

At init time the kernel module defines and initializes a request list
and an in_progress hash table to keep track of all the ops that are
in flight at any given time.

Ops are stateful:

 * unknown
	    - op was just initialized
 * waiting
	    - op is on request_list (upward bound)
 * inprogr
	    - op is in progress (waiting for downcall)
 * serviced
	    - op has matching downcall; ok
 * purged
	    - op has to start a timer since client-core
              exited uncleanly before servicing op
 * given up
	    - submitter has given up waiting for it

When some arbitrary userspace program needs to perform a
filesystem operation on Orangefs (readdir, I/O, create, whatever)
an op structure is initialized and tagged with a distinguishing ID
number. The upcall part of the op is filled out, and the op is
passed to the "service_operation" function.

Service_operation changes the op's state to "waiting", puts
it on the request list, and signals the Orangefs file_operations.poll
function through a wait queue. Userspace is polling the pseudo-device
and thus becomes aware of the upcall request that needs to be read.

When the Orangefs file_operations.read function is triggered, the
request list is searched for an op that seems ready-to-process.
The op is removed from the request list. The tag from the op and
the filled-out upcall struct are copy_to_user'ed back to userspace.

If any of these (and some additional protocol) copy_to_users fail,
the op's state is set to "waiting" and the op is added back to
the request list. Otherwise, the op's state is changed to "in progress",
and the op is hashed on its tag and put onto the end of a list in the
in_progress hash table at the index the tag hashed to.

When userspace has assembled the response to the upcall, it
writes the response, which includes the distinguishing tag, back to
the pseudo device in a series of io_vecs. This triggers the Orangefs
file_operations.write_iter function to find the op with the associated
tag and remove it from the in_progress hash table. As long as the op's
state is not "canceled" or "given up", its state is set to "serviced".
The file_operations.write_iter function returns to the waiting vfs,
and back to service_operation through wait_for_matching_downcall.

Service operation returns to its caller with the op's downcall
part (the response to the upcall) filled out.

The "client-core" is the bridge between the kernel module and
userspace. The client-core is a daemon. The client-core has an
associated watchdog daemon. If the client-core is ever signaled
to die, the watchdog daemon restarts the client-core. Even though
the client-core is restarted "right away", there is a period of
time during such an event that the client-core is dead. A dead client-core
can't be triggered by the Orangefs file_operations.poll function.
Ops that pass through service_operation during a "dead spell" can timeout
on the wait queue and one attempt is made to recycle them. Obviously,
if the client-core stays dead too long, the arbitrary userspace processes
trying to use Orangefs will be negatively affected. Waiting ops
that can't be serviced will be removed from the request list and
have their states set to "given up". In-progress ops that can't
be serviced will be removed from the in_progress hash table and
have their states set to "given up".

Readdir and I/O ops are atypical with respect to their payloads.

  - readdir ops use the smaller of the two pre-allocated pre-partitioned
    memory buffers. The readdir buffer is only available to userspace.
    The kernel module obtains an index to a free partition before launching
    a readdir op. Userspace deposits the results into the indexed partition
    and then writes them to back to the pvfs device.

  - io (read and write) ops use the larger of the two pre-allocated
    pre-partitioned memory buffers. The IO buffer is accessible from
    both userspace and the kernel module. The kernel module obtains an
    index to a free partition before launching an io op. The kernel module
    deposits write data into the indexed partition, to be consumed
    directly by userspace. Userspace deposits the results of read
    requests into the indexed partition, to be consumed directly
    by the kernel module.

Responses to kernel requests are all packaged in pvfs2_downcall_t
structs. Besides a few other members, pvfs2_downcall_t contains a
union of structs, each of which is associated with a particular
response type.

The several members outside of the union are:

 ``int32_t type``
    - type of operation.
 ``int32_t status``
    - return code for the operation.
 ``int64_t trailer_size``
    - 0 unless readdir operation.
 ``char *trailer_buf``
    - initialized to NULL, used during readdir operations.

The appropriate member inside the union is filled out for any
particular response.

  PVFS2_VFS_OP_FILE_IO
    fill a pvfs2_io_response_t

  PVFS2_VFS_OP_LOOKUP
    fill a PVFS_object_kref

  PVFS2_VFS_OP_CREATE
    fill a PVFS_object_kref

  PVFS2_VFS_OP_SYMLINK
    fill a PVFS_object_kref

  PVFS2_VFS_OP_GETATTR
    fill in a PVFS_sys_attr_s (tons of stuff the kernel doesn't need)
    fill in a string with the link target when the object is a symlink.

  PVFS2_VFS_OP_MKDIR
    fill a PVFS_object_kref

  PVFS2_VFS_OP_STATFS
    fill a pvfs2_statfs_response_t with useless info <g>. It is hard for
    us to know, in a timely fashion, these statistics about our
    distributed network filesystem.

  PVFS2_VFS_OP_FS_MOUNT
    fill a pvfs2_fs_mount_response_t which is just like a PVFS_object_kref
    except its members are in a different order and "__pad1" is replaced
    with "id".

  PVFS2_VFS_OP_GETXATTR
    fill a pvfs2_getxattr_response_t

  PVFS2_VFS_OP_LISTXATTR
    fill a pvfs2_listxattr_response_t

  PVFS2_VFS_OP_PARAM
    fill a pvfs2_param_response_t

  PVFS2_VFS_OP_PERF_COUNT
    fill a pvfs2_perf_count_response_t

  PVFS2_VFS_OP_FSKEY
    file a pvfs2_fs_key_response_t

  PVFS2_VFS_OP_READDIR
    jamb everything needed to represent a pvfs2_readdir_response_t into
    the readdir buffer descriptor specified in the upcall.

Userspace uses writev() on /dev/pvfs2-req to pass responses to the requests
made by the kernel side.

A buffer_list containing:

  - a pointer to the prepared response to the request from the
    kernel (struct pvfs2_downcall_t).
  - and also, in the case of a readdir request, a pointer to a
    buffer containing descriptors for the objects in the target
    directory.

... is sent to the function (PINT_dev_write_list) which performs
the writev.

PINT_dev_write_list has a local iovec array: struct iovec io_array[10];

The first four elements of io_array are initialized like this for all
responses::

  io_array[0].iov_base = address of local variable "proto_ver" (int32_t)
  io_array[0].iov_len = sizeof(int32_t)

  io_array[1].iov_base = address of global variable "pdev_magic" (int32_t)
  io_array[1].iov_len = sizeof(int32_t)

  io_array[2].iov_base = address of parameter "tag" (PVFS_id_gen_t)
  io_array[2].iov_len = sizeof(int64_t)

  io_array[3].iov_base = address of out_downcall member (pvfs2_downcall_t)
                         of global variable vfs_request (vfs_request_t)
  io_array[3].iov_len = sizeof(pvfs2_downcall_t)

Readdir responses initialize the fifth element io_array like this::

  io_array[4].iov_base = contents of member trailer_buf (char *)
                         from out_downcall member of global variable
                         vfs_request
  io_array[4].iov_len = contents of member trailer_size (PVFS_size)
                        from out_downcall member of global variable
                        vfs_request

Orangefs exploits the dcache in order to avoid sending redundant
requests to userspace. We keep object inode attributes up-to-date with
orangefs_inode_getattr. Orangefs_inode_getattr uses two arguments to
help it decide whether or not to update an inode: "new" and "bypass".
Orangefs keeps private data in an object's inode that includes a short
timeout value, getattr_time, which allows any iteration of
orangefs_inode_getattr to know how long it has been since the inode was
updated. When the object is not new (new == 0) and the bypass flag is not
set (bypass == 0) orangefs_inode_getattr returns without updating the inode
if getattr_time has not timed out. Getattr_time is updated each time the
inode is updated.

Creation of a new object (file, dir, sym-link) includes the evaluation of
its pathname, resulting in a negative directory entry for the object.
A new inode is allocated and associated with the dentry, turning it from
a negative dentry into a "productive full member of society". Orangefs
obtains the new inode from Linux with new_inode() and associates
the inode with the dentry by sending the pair back to Linux with
d_instantiate().

The evaluation of a pathname for an object resolves to its corresponding
dentry. If there is no corresponding dentry, one is created for it in
the dcache. Whenever a dentry is modified or verified Orangefs stores a
short timeout value in the dentry's d_time, and the dentry will be trusted
for that amount of time. Orangefs is a network filesystem, and objects
can potentially change out-of-band with any particular Orangefs kernel module
instance, so trusting a dentry is risky. The alternative to trusting
dentries is to always obtain the needed information from userspace - at
least a trip to the client-core, maybe to the servers. Obtaining information
from a dentry is cheap, obtaining it from userspace is relatively expensive,
hence the motivation to use the dentry when possible.

The timeout values d_time and getattr_time are jiffy based, and the
code is designed to avoid the jiffy-wrap problem::

    "In general, if the clock may have wrapped around more than once, there
    is no way to tell how much time has elapsed. However, if the times t1
    and t2 are known to be fairly close, we can reliably compute the
    difference in a way that takes into account the possibility that the
    clock may have wrapped between times."

from course notes by instructor Andy Wang