code:
.. SPDX-License-Identifier: GPL-2.0

.. UBIFS Authentication
.. sigma star gmbh
.. 2018

============================
UBIFS Authentication Support
============================

Introduction
============

UBIFS utilizes the fscrypt framework to provide confidentiality for file
contents and file names. This prevents attacks where an attacker is able to
read contents of the filesystem on a single point in time. A classic example
is a lost smartphone where the attacker is unable to read personal data stored
on the device without the filesystem decryption key.

At the current state, UBIFS encryption however does not prevent attacks where
the attacker is able to modify the filesystem contents and the user uses the
device afterwards. In such a scenario an attacker can modify filesystem
contents arbitrarily without the user noticing. One example is to modify a
binary to perform a malicious action when executed [DMC-CBC-ATTACK]. Since
most of the filesystem metadata of UBIFS is stored in plain, this makes it
fairly easy to swap files and replace their contents.

Other full disk encryption systems like dm-crypt cover all filesystem metadata,
which makes such kinds of attacks more complicated, but not impossible.
Especially, if the attacker is given access to the device multiple points in
time. For dm-crypt and other filesystems that build upon the Linux block IO
layer, the dm-integrity or dm-verity subsystems [DM-INTEGRITY, DM-VERITY]
can be used to get full data authentication at the block layer.
These can also be combined with dm-crypt [CRYPTSETUP2].

This document describes an approach to get file contents _and_ full metadata
authentication for UBIFS. Since UBIFS uses fscrypt for file contents and file
name encryption, the authentication system could be tied into fscrypt such that
existing features like key derivation can be utilized. It should however also
be possible to use UBIFS authentication without using encryption.

MTD, UBI & UBIFS
----------------

On Linux, the MTD (Memory Technology Devices) subsystem provides a uniform
interface to access raw flash devices. One of the more prominent subsystems that
work on top of MTD is UBI (Unsorted Block Images). It provides volume management
for flash devices and is thus somewhat similar to LVM for block devices. In
addition, it deals with flash-specific wear-leveling and transparent I/O error
handling. UBI offers logical erase blocks (LEBs) to the layers on top of it
and maps them transparently to physical erase blocks (PEBs) on the flash.

UBIFS is a filesystem for raw flash which operates on top of UBI. Thus, wear
leveling and some flash specifics are left to UBI, while UBIFS focuses on
scalability, performance and recoverability.

::

	+------------+ +*******+ +-----------+ +-----+
	|            | * UBIFS * | UBI-BLOCK | | ... |
	| JFFS/JFFS2 | +*******+ +-----------+ +-----+
	|            | +-----------------------------+ +-----------+ +-----+
	|            | |              UBI            | | MTD-BLOCK | | ... |
	+------------+ +-----------------------------+ +-----------+ +-----+
	+------------------------------------------------------------------+
	|                  MEMORY TECHNOLOGY DEVICES (MTD)                 |
	+------------------------------------------------------------------+
	+-----------------------------+ +--------------------------+ +-----+
	|         NAND DRIVERS        | |        NOR DRIVERS       | | ... |
	+-----------------------------+ +--------------------------+ +-----+

            Figure 1: Linux kernel subsystems for dealing with raw flash

Internally, UBIFS maintains multiple data structures which are persisted on
the flash:

- *Index*: an on-flash B+ tree where the leaf nodes contain filesystem data
- *Journal*: an additional data structure to collect FS changes before updating
  the on-flash index and reduce flash wear.
- *Tree Node Cache (TNC)*: an in-memory B+ tree that reflects the current FS
  state to avoid frequent flash reads. It is basically the in-memory
  representation of the index, but contains additional attributes.
- *LEB property tree (LPT)*: an on-flash B+ tree for free space accounting per
  UBI LEB.

In the remainder of this section we will cover the on-flash UBIFS data
structures in more detail. The TNC is of less importance here since it is never
persisted onto the flash directly. More details on UBIFS can also be found in
[UBIFS-WP].

UBIFS Index & Tree Node Cache
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Basic on-flash UBIFS entities are called *nodes*. UBIFS knows different types
of nodes. Eg. data nodes (``struct ubifs_data_node``) which store chunks of file
contents or inode nodes (``struct ubifs_ino_node``) which represent VFS inodes.
Almost all types of nodes share a common header (``ubifs_ch``) containing basic
information like node type, node length, a sequence number, etc. (see
``fs/ubifs/ubifs-media.h`` in kernel source). Exceptions are entries of the LPT
and some less important node types like padding nodes which are used to pad
unusable content at the end of LEBs.

To avoid re-writing the whole B+ tree on every single change, it is implemented
as *wandering tree*, where only the changed nodes are re-written and previous
versions of them are obsoleted without erasing them right away. As a result,
the index is not stored in a single place on the flash, but *wanders* around
and there are obsolete parts on the flash as long as the LEB containing them is
not reused by UBIFS. To find the most recent version of the index, UBIFS stores
a special node called *master node* into UBI LEB 1 which always points to the
most recent root node of the UBIFS index. For recoverability, the master node
is additionally duplicated to LEB 2. Mounting UBIFS is thus a simple read of
LEB 1 and 2 to get the current master node and from there get the location of
the most recent on-flash index.

The TNC is the in-memory representation of the on-flash index. It contains some
additional runtime attributes per node which are not persisted. One of these is
a dirty-flag which marks nodes that have to be persisted the next time the
index is written onto the flash. The TNC acts as a write-back cache and all
modifications of the on-flash index are done through the TNC. Like other caches,
the TNC does not have to mirror the full index into memory, but reads parts of
it from flash whenever needed. A *commit* is the UBIFS operation of updating the
on-flash filesystem structures like the index. On every commit, the TNC nodes
marked as dirty are written to the flash to update the persisted index.

Journal
~~~~~~~

To avoid wearing out the flash, the index is only persisted (*commited*) when
certain conditions are met (eg. ``fsync(2)``). The journal is used to record
any changes (in form of inode nodes, data nodes etc.) between commits
of the index. During mount, the journal is read from the flash and replayed
onto the TNC (which will be created on-demand from the on-flash index).

UBIFS reserves a bunch of LEBs just for the journal called *log area*. The
amount of log area LEBs is configured on filesystem creation (using
``mkfs.ubifs``) and stored in the superblock node. The log area contains only
two types of nodes: *reference nodes* and *commit start nodes*. A commit start
node is written whenever an index commit is performed. Reference nodes are
written on every journal update. Each reference node points to the position of
other nodes (inode nodes, data nodes etc.) on the flash that are part of this
journal entry. These nodes are called *buds* and describe the actual filesystem
changes including their data.

The log area is maintained as a ring. Whenever the journal is almost full,
a commit is initiated. This also writes a commit start node so that during
mount, UBIFS will seek for the most recent commit start node and just replay
every reference node after that. Every reference node before the commit start
node will be ignored as they are already part of the on-flash index.

When writing a journal entry, UBIFS first ensures that enough space is
available to write the reference node and buds part of this entry. Then, the
reference node is written and afterwards the buds describing the file changes.
On replay, UBIFS will record every reference node and inspect the location of
the referenced LEBs to discover the buds. If these are corrupt or missing,
UBIFS will attempt to recover them by re-reading the LEB. This is however only
done for the last referenced LEB of the journal. Only this can become corrupt
because of a power cut. If the recovery fails, UBIFS will not mount. An error
for every other LEB will directly cause UBIFS to fail the mount operation.

::

       | ----    LOG AREA     ---- | ----------    MAIN AREA    ------------ |

        -----+------+-----+--------+----   ------+-----+-----+---------------
        \    |      |     |        |   /  /      |     |     |               \
        / CS |  REF | REF |        |   \  \ DENT | INO | INO |               /
        \    |      |     |        |   /  /      |     |     |               \
         ----+------+-----+--------+---   -------+-----+-----+----------------
                 |     |                  ^            ^
                 |     |                  |            |
                 +------------------------+            |
                       |                               |
                       +-------------------------------+

                Figure 2: UBIFS flash layout of log area with commit start nodes
                          (CS) and reference nodes (REF) pointing to main area
                          containing their buds

LEB Property Tree/Table
~~~~~~~~~~~~~~~~~~~~~~~

The LEB property tree is used to store per-LEB information. This includes the
LEB type and amount of free and *dirty* (old, obsolete content) space [1]_ on
the LEB. The type is important, because UBIFS never mixes index nodes with data
nodes on a single LEB and thus each LEB has a specific purpose. This again is
useful for free space calculations. See [UBIFS-WP] for more details.

The LEB property tree again is a B+ tree, but it is much smaller than the
index. Due to its smaller size it is always written as one chunk on every
commit. Thus, saving the LPT is an atomic operation.

.. [1] Since LEBs can only be appended and never overwritten, there is a
   difference between free space ie. the remaining space left on the LEB to be
   written to without erasing it and previously written content that is obsolete
   but can't be overwritten without erasing the full LEB.

UBIFS Authentication
====================

This chapter introduces UBIFS authentication which enables UBIFS to verify
the authenticity and integrity of metadata and file contents stored on flash.

Threat Model
------------

UBIFS authentication enables detection of offline data modification. While it
does not prevent it, it enables (trusted) code to check the integrity and
authenticity of on-flash file contents and filesystem metadata. This covers
attacks where file contents are swapped.

UBIFS authentication will not protect against rollback of full flash contents.
Ie. an attacker can still dump the flash and restore it at a later time without
detection. It will also not protect against partial rollback of individual
index commits. That means that an attacker is able to partially undo changes.
This is possible because UBIFS does not immediately overwrites obsolete
versions of the index tree or the journal, but instead marks them as obsolete
and garbage collection erases them at a later time. An attacker can use this by
erasing parts of the current tree and restoring old versions that are still on
the flash and have not yet been erased. This is possible, because every commit
will always write a new version of the index root node and the master node
without overwriting the previous version. This is further helped by the
wear-leveling operations of UBI which copies contents from one physical
eraseblock to another and does not atomically erase the first eraseblock.

UBIFS authentication does not cover attacks where an attacker is able to
execute code on the device after the authentication key was provided.
Additional measures like secure boot and trusted boot have to be taken to
ensure that only trusted code is executed on a device.

Authentication
--------------

To be able to fully trust data read from flash, all UBIFS data structures
stored on flash are authenticated. That is:

- The index which includes file contents, file metadata like extended
  attributes, file length etc.
- The journal which also contains file contents and metadata by recording changes
  to the filesystem
- The LPT which stores UBI LEB metadata which UBIFS uses for free space accounting

Index Authentication
~~~~~~~~~~~~~~~~~~~~

Through UBIFS' concept of a wandering tree, it already takes care of only
updating and persisting changed parts from leaf node up to the root node
of the full B+ tree. This enables us to augment the index nodes of the tree
with a hash over each node's child nodes. As a result, the index basically also
a Merkle tree. Since the leaf nodes of the index contain the actual filesystem
data, the hashes of their parent index nodes thus cover all the file contents
and file metadata. When a file changes, the UBIFS index is updated accordingly
from the leaf nodes up to the root node including the master node. This process
can be hooked to recompute the hash only for each changed node at the same time.
Whenever a file is read, UBIFS can verify the hashes from each leaf node up to
the root node to ensure the node's integrity.

To ensure the authenticity of the whole index, the UBIFS master node stores a
keyed hash (HMAC) over its own contents and a hash of the root node of the index
tree. As mentioned above, the master node is always written to the flash whenever
the index is persisted (ie. on index commit).

Using this approach only UBIFS index nodes and the master node are changed to
include a hash. All other types of nodes will remain unchanged. This reduces
the storage overhead which is precious for users of UBIFS (ie. embedded
devices).

::

                             +---------------+
                             |  Master Node  |
                             |    (hash)     |
                             +---------------+
                                     |
                                     v
                            +-------------------+
                            |  Index Node #1    |
                            |                   |
                            | branch0   branchn |
                            | (hash)    (hash)  |
                            +-------------------+
                               |    ...   |  (fanout: 8)
                               |          |
                       +-------+          +------+
                       |                         |
                       v                         v
            +-------------------+       +-------------------+
            |  Index Node #2    |       |  Index Node #3    |
            |                   |       |                   |
            | branch0   branchn |       | branch0   branchn |
            | (hash)    (hash)  |       | (hash)    (hash)  |
            +-------------------+       +-------------------+
                 |   ...                     |   ...   |
                 v                           v         v
               +-----------+         +----------+  +-----------+
               | Data Node |         | INO Node |  | DENT Node |
               +-----------+         +----------+  +-----------+

           Figure 3: Coverage areas of index node hash and master node HMAC

The most important part for robustness and power-cut safety is to atomically
persist the hash and file contents. Here the existing UBIFS logic for how
changed nodes are persisted is already designed for this purpose such that
UBIFS can safely recover if a power-cut occurs while persisting. Adding
hashes to index nodes does not change this since each hash will be persisted
atomically together with its respective node.

Journal Authentication
~~~~~~~~~~~~~~~~~~~~~~

The journal is authenticated too. Since the journal is continuously written
it is necessary to also add authentication information frequently to the
journal so that in case of a powercut not too much data can't be authenticated.
This is done by creating a continuous hash beginning from the commit start node
over the previous reference nodes, the current reference node, and the bud
nodes. From time to time whenever it is suitable authentication nodes are added
between the bud nodes. This new node type contains a HMAC over the current state
of the hash chain. That way a journal can be authenticated up to the last
authentication node. The tail of the journal which may not have a authentication
node cannot be authenticated and is skipped during journal replay.

We get this picture for journal authentication::

    ,,,,,,,,
    ,......,...........................................
    ,. CS  ,               hash1.----.           hash2.----.
    ,.  |  ,                    .    |hmac            .    |hmac
    ,.  v  ,                    .    v                .    v
    ,.REF#0,-> bud -> bud -> bud.-> auth -> bud -> bud.-> auth ...
    ,..|...,...........................................
    ,  |   ,
    ,  |   ,,,,,,,,,,,,,,,
    .  |            hash3,----.
    ,  |                 ,    |hmac
    ,  v                 ,    v
    , REF#1 -> bud -> bud,-> auth ...
    ,,,|,,,,,,,,,,,,,,,,,,
       v
      REF#2 -> ...
       |
       V
      ...

Since the hash also includes the reference nodes an attacker cannot reorder or
skip any journal heads for replay. An attacker can only remove bud nodes or
reference nodes from the end of the journal, effectively rewinding the
filesystem at maximum back to the last commit.

The location of the log area is stored in the master node. Since the master
node is authenticated with a HMAC as described above, it is not possible to
tamper with that without detection. The size of the log area is specified when
the filesystem is created using `mkfs.ubifs` and stored in the superblock node.
To avoid tampering with this and other values stored there, a HMAC is added to
the superblock struct. The superblock node is stored in LEB 0 and is only
modified on feature flag or similar changes, but never on file changes.

LPT Authentication
~~~~~~~~~~~~~~~~~~

The location of the LPT root node on the flash is stored in the UBIFS master
node. Since the LPT is written and read atomically on every commit, there is
no need to authenticate individual nodes of the tree. It suffices to
protect the integrity of the full LPT by a simple hash stored in the master
node. Since the master node itself is authenticated, the LPTs authenticity can
be verified by verifying the authenticity of the master node and comparing the
LTP hash stored there with the hash computed from the read on-flash LPT.

Key Management
--------------

For simplicity, UBIFS authentication uses a single key to compute the HMACs
of superblock, master, commit start and reference nodes. This key has to be
available on creation of the filesystem (`mkfs.ubifs`) to authenticate the
superblock node. Further, it has to be available on mount of the filesystem
to verify authenticated nodes and generate new HMACs for changes.

UBIFS authentication is intended to operate side-by-side with UBIFS encryption
(fscrypt) to provide confidentiality and authenticity. Since UBIFS encryption
has a different approach of encryption policies per directory, there can be
multiple fscrypt master keys and there might be folders without encryption.
UBIFS authentication on the other hand has an all-or-nothing approach in the
sense that it either authenticates everything of the filesystem or nothing.
Because of this and because UBIFS authentication should also be usable without
encryption, it does not share the same master key with fscrypt, but manages
a dedicated authentication key.

The API for providing the authentication key has yet to be defined, but the
key can eg. be provided by userspace through a keyring similar to the way it
is currently done in fscrypt. It should however be noted that the current
fscrypt approach has shown its flaws and the userspace API will eventually
change [FSCRYPT-POLICY2].

Nevertheless, it will be possible for a user to provide a single passphrase
or key in userspace that covers UBIFS authentication and encryption. This can
be solved by the corresponding userspace tools which derive a second key for
authentication in addition to the derived fscrypt master key used for
encryption.

To be able to check if the proper key is available on mount, the UBIFS
superblock node will additionally store a hash of the authentication key. This
approach is similar to the approach proposed for fscrypt encryption policy v2
[FSCRYPT-POLICY2].

Future Extensions
=================

In certain cases where a vendor wants to provide an authenticated filesystem
image to customers, it should be possible to do so without sharing the secret
UBIFS authentication key. Instead, in addition the each HMAC a digital
signature could be stored where the vendor shares the public key alongside the
filesystem image. In case this filesystem has to be modified afterwards,
UBIFS can exchange all digital signatures with HMACs on first mount similar
to the way the IMA/EVM subsystem deals with such situations. The HMAC key
will then have to be provided beforehand in the normal way.

References
==========

[CRYPTSETUP2]        https://www.saout.de/pipermail/dm-crypt/2017-November/005745.html

[DMC-CBC-ATTACK]     https://www.jakoblell.com/blog/2013/12/22/practical-malleability-attack-against-cbc-encrypted-luks-partitions/

[DM-INTEGRITY]       https://www.kernel.org/doc/Documentation/device-mapper/dm-integrity.rst

[DM-VERITY]          https://www.kernel.org/doc/Documentation/device-mapper/verity.rst

[FSCRYPT-POLICY2]    https://www.spinics.net/lists/linux-ext4/msg58710.html

[UBIFS-WP]           http://www.linux-mtd.infradead.org/doc/ubifs_whitepaper.pdf


code:
.. SPDX-License-Identifier: GPL-2.0

===============
UBI File System
===============

Introduction
============

UBIFS file-system stands for UBI File System. UBI stands for "Unsorted
Block Images". UBIFS is a flash file system, which means it is designed
to work with flash devices. It is important to understand, that UBIFS
is completely different to any traditional file-system in Linux, like
Ext2, XFS, JFS, etc. UBIFS represents a separate class of file-systems
which work with MTD devices, not block devices. The other Linux
file-system of this class is JFFS2.

To make it more clear, here is a small comparison of MTD devices and
block devices.

1 MTD devices represent flash devices and they consist of eraseblocks of
  rather large size, typically about 128KiB. Block devices consist of
  small blocks, typically 512 bytes.
2 MTD devices support 3 main operations - read from some offset within an
  eraseblock, write to some offset within an eraseblock, and erase a whole
  eraseblock. Block  devices support 2 main operations - read a whole
  block and write a whole block.
3 The whole eraseblock has to be erased before it becomes possible to
  re-write its contents. Blocks may be just re-written.
4 Eraseblocks become worn out after some number of erase cycles -
  typically 100K-1G for SLC NAND and NOR flashes, and 1K-10K for MLC
  NAND flashes. Blocks do not have the wear-out property.
5 Eraseblocks may become bad (only on NAND flashes) and software should
  deal with this. Blocks on hard drives typically do not become bad,
  because hardware has mechanisms to substitute bad blocks, at least in
  modern LBA disks.

It should be quite obvious why UBIFS is very different to traditional
file-systems.

UBIFS works on top of UBI. UBI is a separate software layer which may be
found in drivers/mtd/ubi. UBI is basically a volume management and
wear-leveling layer. It provides so called UBI volumes which is a higher
level abstraction than a MTD device. The programming model of UBI devices
is very similar to MTD devices - they still consist of large eraseblocks,
they have read/write/erase operations, but UBI devices are devoid of
limitations like wear and bad blocks (items 4 and 5 in the above list).

In a sense, UBIFS is a next generation of JFFS2 file-system, but it is
very different and incompatible to JFFS2. The following are the main
differences.

* JFFS2 works on top of MTD devices, UBIFS depends on UBI and works on
  top of UBI volumes.
* JFFS2 does not have on-media index and has to build it while mounting,
  which requires full media scan. UBIFS maintains the FS indexing
  information on the flash media and does not require full media scan,
  so it mounts many times faster than JFFS2.
* JFFS2 is a write-through file-system, while UBIFS supports write-back,
  which makes UBIFS much faster on writes.

Similarly to JFFS2, UBIFS supports on-the-fly compression which makes
it possible to fit quite a lot of data to the flash.

Similarly to JFFS2, UBIFS is tolerant of unclean reboots and power-cuts.
It does not need stuff like fsck.ext2. UBIFS automatically replays its
journal and recovers from crashes, ensuring that the on-flash data
structures are consistent.

UBIFS scales logarithmically (most of the data structures it uses are
trees), so the mount time and memory consumption do not linearly depend
on the flash size, like in case of JFFS2. This is because UBIFS
maintains the FS index on the flash media. However, UBIFS depends on
UBI, which scales linearly. So overall UBI/UBIFS stack scales linearly.
Nevertheless, UBI/UBIFS scales considerably better than JFFS2.

The authors of UBIFS believe, that it is possible to develop UBI2 which
would scale logarithmically as well. UBI2 would support the same API as UBI,
but it would be binary incompatible to UBI. So UBIFS would not need to be
changed to use UBI2

Mount options
=============

(*) == default.

====================	=======================================================
bulk_read		read more in one go to take advantage of flash
			media that read faster sequentially
no_bulk_read (*)	do not bulk-read
no_chk_data_crc (*)	skip checking of CRCs on data nodes in order to
			improve read performance. Use this option only
			if the flash media is highly reliable. The effect
			of this option is that corruption of the contents
			of a file can go unnoticed.
chk_data_crc		do not skip checking CRCs on data nodes
compr=none              override default compressor and set it to "none"
compr=lzo               override default compressor and set it to "lzo"
compr=zlib              override default compressor and set it to "zlib"
auth_key=		specify the key used for authenticating the filesystem.
			Passing this option makes authentication mandatory.
			The passed key must be present in the kernel keyring
			and must be of type 'logon'
auth_hash_name=		The hash algorithm used for authentication. Used for
			both hashing and for creating HMACs. Typical values
			include "sha256" or "sha512"
====================	=======================================================

Quick usage instructions
========================

The UBI volume to mount is specified using "ubiX_Y" or "ubiX:NAME" syntax,
where "X" is UBI device number, "Y" is UBI volume number, and "NAME" is
UBI volume name.

Mount volume 0 on UBI device 0 to /mnt/ubifs::

    $ mount -t ubifs ubi0_0 /mnt/ubifs

Mount "rootfs" volume of UBI device 0 to /mnt/ubifs ("rootfs" is volume
name)::

    $ mount -t ubifs ubi0:rootfs /mnt/ubifs

The following is an example of the kernel boot arguments to attach mtd0
to UBI and mount volume "rootfs":
ubi.mtd=0 root=ubi0:rootfs rootfstype=ubifs

References
==========

UBIFS documentation and FAQ/HOWTO at the MTD web site:

- http://www.linux-mtd.infradead.org/doc/ubifs.html
- http://www.linux-mtd.infradead.org/faq/ubifs.html

repo_name: angery/linux


code:
.. SPDX-License-Identifier: GPL-2.0

===============
UDF file system
===============

If you encounter problems with reading UDF discs using this driver,
please report them according to MAINTAINERS file.

Write support requires a block driver which supports writing.  Currently
dvd+rw drives and media support true random sector writes, and so a udf
filesystem on such devices can be directly mounted read/write.  CD-RW
media however, does not support this.  Instead the media can be formatted
for packet mode using the utility cdrwtool, then the pktcdvd driver can
be bound to the underlying cd device to provide the required buffering
and read-modify-write cycles to allow the filesystem random sector writes
while providing the hardware with only full packet writes.  While not
required for dvd+rw media, use of the pktcdvd driver often enhances
performance due to very poor read-modify-write support supplied internally
by drive firmware.

-------------------------------------------------------------------------------

The following mount options are supported:

	===========	======================================
	gid=		Set the default group.
	umask=		Set the default umask.
	mode=		Set the default file permissions.
	dmode=		Set the default directory permissions.
	uid=		Set the default user.
	bs=		Set the block size.
	unhide		Show otherwise hidden files.
	undelete	Show deleted files in lists.
	adinicb		Embed data in the inode (default)
	noadinicb	Don't embed data in the inode
	shortad		Use short ad's
	longad		Use long ad's (default)
	nostrict	Unset strict conformance
	iocharset=	Set the NLS character set
	===========	======================================

The uid= and gid= options need a bit more explaining.  They will accept a
decimal numeric value and all inodes on that mount will then appear as
belonging to that uid and gid.  Mount options also accept the string "forget".
The forget option causes all IDs to be written to disk as -1 which is a way
of UDF standard to indicate that IDs are not supported for these files .

For typical desktop use of removable media, you should set the ID to that of
the interactively logged on user, and also specify the forget option.  This way
the interactive user will always see the files on the disk as belonging to him.

The remaining are for debugging and disaster recovery:

	=====		================================
	novrs		Skip volume sequence recognition
	=====		================================

The following expect a offset from 0.

	==========	=================================================
	session=	Set the CDROM session (default= last session)
	anchor=		Override standard anchor location. (default= 256)
	lastblock=	Set the last block of the filesystem/
	==========	=================================================

-------------------------------------------------------------------------------

For the latest version and toolset see:
	https://github.com/pali/udftools

Documentation on UDF and ECMA 167 is available FREE from:
	- http://www.osta.org/
	- https://www.ecma-international.org/


code:
====
VFAT
====

USING VFAT
==========

To use the vfat filesystem, use the filesystem type 'vfat'.  i.e.::

  mount -t vfat /dev/fd0 /mnt

No special partition formatter is required,
'mkdosfs' will work fine if you want to format from within Linux.

VFAT MOUNT OPTIONS
==================

**uid=###**
	Set the owner of all files on this filesystem.
	The default is the uid of current process.

**gid=###**
	Set the group of all files on this filesystem.
	The default is the gid of current process.

**umask=###**
	The permission mask (for files and directories, see *umask(1)*).
	The default is the umask of current process.

**dmask=###**
	The permission mask for the directory.
	The default is the umask of current process.

**fmask=###**
	The permission mask for files.
	The default is the umask of current process.

**allow_utime=###**
	This option controls the permission check of mtime/atime.

		**-20**: If current process is in group of file's group ID,
                you can change timestamp.

		**-2**: Other users can change timestamp.

	The default is set from dmask option. If the directory is
	writable, utime(2) is also allowed. i.e. ~dmask & 022.

	Normally utime(2) checks current process is owner of
	the file, or it has CAP_FOWNER capability. But FAT
	filesystem doesn't have uid/gid on disk, so normal
	check is too unflexible. With this option you can
	relax it.

**codepage=###**
	Sets the codepage number for converting to shortname
	characters on FAT filesystem.
	By default, FAT_DEFAULT_CODEPAGE setting is used.

**iocharset=<name>**
	Character set to use for converting between the
	encoding is used for user visible filename and 16 bit
	Unicode characters. Long filenames are stored on disk
	in Unicode format, but Unix for the most part doesn't
	know how to deal with Unicode.
	By default, FAT_DEFAULT_IOCHARSET setting is used.

	There is also an option of doing UTF-8 translations
	with the utf8 option.

.. note:: ``iocharset=utf8`` is not recommended. If unsure, you should consider
	  the utf8 option instead.

**utf8=<bool>**
	UTF-8 is the filesystem safe version of Unicode that
	is used by the console. It can be enabled or disabled
	for the filesystem with this option.
	If 'uni_xlate' gets set, UTF-8 gets disabled.
	By default, FAT_DEFAULT_UTF8 setting is used.

**uni_xlate=<bool>**
	Translate unhandled Unicode characters to special
	escaped sequences.  This would let you backup and
	restore filenames that are created with any Unicode
	characters.  Until Linux supports Unicode for real,
	this gives you an alternative.  Without this option,
	a '?' is used when no translation is possible.  The
	escape character is ':' because it is otherwise
	illegal on the vfat filesystem.  The escape sequence
	that gets used is ':' and the four digits of hexadecimal
	unicode.

**nonumtail=<bool>**
	When creating 8.3 aliases, normally the alias will
	end in '~1' or tilde followed by some number.  If this
	option is set, then if the filename is
	"longfilename.txt" and "longfile.txt" does not
	currently exist in the directory, longfile.txt will
	be the short alias instead of longfi~1.txt.

**usefree**
	Use the "free clusters" value stored on FSINFO. It will
	be used to determine number of free clusters without
	scanning disk. But it's not used by default, because
	recent Windows don't update it correctly in some
	case. If you are sure the "free clusters" on FSINFO is
	correct, by this option you can avoid scanning disk.

**quiet**
	Stops printing certain warning messages.

**check=s|r|n**
	Case sensitivity checking setting.

	**s**: strict, case sensitive

	**r**: relaxed, case insensitive

	**n**: normal, default setting, currently case insensitive

**nocase**
	This was deprecated for vfat. Use ``shortname=win95`` instead.

**shortname=lower|win95|winnt|mixed**
	Shortname display/create setting.

	**lower**: convert to lowercase for display,
	emulate the Windows 95 rule for create.

	**win95**: emulate the Windows 95 rule for display/create.

	**winnt**: emulate the Windows NT rule for display/create.

	**mixed**: emulate the Windows NT rule for display,
	emulate the Windows 95 rule for create.

	Default setting is `mixed`.

**tz=UTC**
	Interpret timestamps as UTC rather than local time.
	This option disables the conversion of timestamps
	between local time (as used by Windows on FAT) and UTC
	(which Linux uses internally).  This is particularly
	useful when mounting devices (like digital cameras)
	that are set to UTC in order to avoid the pitfalls of
	local time.

**time_offset=minutes**
	Set offset for conversion of timestamps from local time
	used by FAT to UTC. I.e. <minutes> minutes will be subtracted
	from each timestamp to convert it to UTC used internally by
	Linux. This is useful when time zone set in ``sys_tz`` is
	not the time zone used by the filesystem. Note that this
	option still does not provide correct time stamps in all
	cases in presence of DST - time stamps in a different DST
	setting will be off by one hour.

**showexec**
	If set, the execute permission bits of the file will be
	allowed only if the extension part of the name is .EXE,
	.COM, or .BAT. Not set by default.

**debug**
	Can be set, but unused by the current implementation.

**sys_immutable**
	If set, ATTR_SYS attribute on FAT is handled as
	IMMUTABLE flag on Linux. Not set by default.

**flush**
	If set, the filesystem will try to flush to disk more
	early than normal. Not set by default.

**rodir**
	FAT has the ATTR_RO (read-only) attribute. On Windows,
	the ATTR_RO of the directory will just be ignored,
	and is used only by applications as a flag (e.g. it's set
	for the customized folder).

	If you want to use ATTR_RO as read-only flag even for
	the directory, set this option.

**errors=panic|continue|remount-ro**
	specify FAT behavior on critical errors: panic, continue
	without doing anything or remount the partition in
	read-only mode (default behavior).

**discard**
	If set, issues discard/TRIM commands to the block
	device when blocks are freed. This is useful for SSD devices
	and sparse/thinly-provisioned LUNs.

**nfs=stale_rw|nostale_ro**
	Enable this only if you want to export the FAT filesystem
	over NFS.

		**stale_rw**: This option maintains an index (cache) of directory
		*inodes* by *i_logstart* which is used by the nfs-related code to
		improve look-ups. Full file operations (read/write) over NFS is
		supported but with cache eviction at NFS server, this could
		result in ESTALE issues.

		**nostale_ro**: This option bases the *inode* number and filehandle
		on the on-disk location of a file in the MS-DOS directory entry.
		This ensures that ESTALE will not be returned after a file is
		evicted from the inode cache. However, it means that operations
		such as rename, create and unlink could cause filehandles that
		previously pointed at one file to point at a different file,
		potentially causing data corruption. For this reason, this
		option also mounts the filesystem readonly.

	To maintain backward compatibility, ``'-o nfs'`` is also accepted,
	defaulting to "stale_rw".

**dos1xfloppy  <bool>: 0,1,yes,no,true,false**
	If set, use a fallback default BIOS Parameter Block
	configuration, determined by backing device size. These static
	parameters match defaults assumed by DOS 1.x for 160 kiB,
	180 kiB, 320 kiB, and 360 kiB floppies and floppy images.

LIMITATION
==========

The fallocated region of file is discarded at umount/evict time
when using fallocate with FALLOC_FL_KEEP_SIZE.
So, User should assume that fallocated region can be discarded at
last close if there is memory pressure resulting in eviction of
the inode from the memory. As a result, for any dependency on
the fallocated region, user should make sure to recheck fallocate
after reopening the file.

TODO
====
Need to get rid of the raw scanning stuff.  Instead, always use
a get next directory entry approach.  The only thing left that uses
raw scanning is the directory renaming code.

POSSIBLE PROBLEMS
=================

- vfat_valid_longname does not properly checked reserved names.
- When a volume name is the same as a directory name in the root
  directory of the filesystem, the directory name sometimes shows
  up as an empty file.
- autoconv option does not work correctly.

TEST SUITE
==========
If you plan to make any modifications to the vfat filesystem, please
get the test suite that comes with the vfat distribution at

`<http://web.archive.org/web/*/http://bmrc.berkeley.edu/people/chaffee/vfat.html>`_

This tests quite a few parts of the vfat filesystem and additional
tests for new features or untested features would be appreciated.

NOTES ON THE STRUCTURE OF THE VFAT FILESYSTEM
=============================================
This documentation was provided by Galen C. Hunt gchunt@cs.rochester.edu and
lightly annotated by Gordon Chaffee.

This document presents a very rough, technical overview of my
knowledge of the extended FAT file system used in Windows NT 3.5 and
Windows 95.  I don't guarantee that any of the following is correct,
but it appears to be so.

The extended FAT file system is almost identical to the FAT
file system used in DOS versions up to and including *6.223410239847*
:-).  The significant change has been the addition of long file names.
These names support up to 255 characters including spaces and lower
case characters as opposed to the traditional 8.3 short names.

Here is the description of the traditional FAT entry in the current
Windows 95 filesystem::

        struct directory { // Short 8.3 names
                unsigned char name[8];          // file name
                unsigned char ext[3];           // file extension
                unsigned char attr;             // attribute byte
		unsigned char lcase;		// Case for base and extension
		unsigned char ctime_ms;		// Creation time, milliseconds
		unsigned char ctime[2];		// Creation time
		unsigned char cdate[2];		// Creation date
		unsigned char adate[2];		// Last access date
		unsigned char reserved[2];	// reserved values (ignored)
                unsigned char time[2];          // time stamp
                unsigned char date[2];          // date stamp
                unsigned char start[2];         // starting cluster number
                unsigned char size[4];          // size of the file
        };

The lcase field specifies if the base and/or the extension of an 8.3
name should be capitalized.  This field does not seem to be used by
Windows 95 but it is used by Windows NT.  The case of filenames is not
completely compatible from Windows NT to Windows 95.  It is not completely
compatible in the reverse direction, however.  Filenames that fit in
the 8.3 namespace and are written on Windows NT to be lowercase will
show up as uppercase on Windows 95.

.. note:: Note that the ``start`` and ``size`` values are actually little
          endian integer values.  The descriptions of the fields in this
          structure are public knowledge and can be found elsewhere.

With the extended FAT system, Microsoft has inserted extra
directory entries for any files with extended names.  (Any name which
legally fits within the old 8.3 encoding scheme does not have extra
entries.)  I call these extra entries slots.  Basically, a slot is a
specially formatted directory entry which holds up to 13 characters of
a file's extended name.  Think of slots as additional labeling for the
directory entry of the file to which they correspond.  Microsoft
prefers to refer to the 8.3 entry for a file as its alias and the
extended slot directory entries as the file name.

The C structure for a slot directory entry follows::

        struct slot { // Up to 13 characters of a long name
                unsigned char id;               // sequence number for slot
                unsigned char name0_4[10];      // first 5 characters in name
                unsigned char attr;             // attribute byte
                unsigned char reserved;         // always 0
                unsigned char alias_checksum;   // checksum for 8.3 alias
                unsigned char name5_10[12];     // 6 more characters in name
                unsigned char start[2];         // starting cluster number
                unsigned char name11_12[4];     // last 2 characters in name
        };

If the layout of the slots looks a little odd, it's only
because of Microsoft's efforts to maintain compatibility with old
software.  The slots must be disguised to prevent old software from
panicking.  To this end, a number of measures are taken:

        1) The attribute byte for a slot directory entry is always set
           to 0x0f.  This corresponds to an old directory entry with
           attributes of "hidden", "system", "read-only", and "volume
           label".  Most old software will ignore any directory
           entries with the "volume label" bit set.  Real volume label
           entries don't have the other three bits set.

        2) The starting cluster is always set to 0, an impossible
           value for a DOS file.

Because the extended FAT system is backward compatible, it is
possible for old software to modify directory entries.  Measures must
be taken to ensure the validity of slots.  An extended FAT system can
verify that a slot does in fact belong to an 8.3 directory entry by
the following:

        1) Positioning.  Slots for a file always immediately proceed
           their corresponding 8.3 directory entry.  In addition, each
           slot has an id which marks its order in the extended file
           name.  Here is a very abbreviated view of an 8.3 directory
           entry and its corresponding long name slots for the file
           "My Big File.Extension which is long"::

                <proceeding files...>
                <slot #3, id = 0x43, characters = "h is long">
                <slot #2, id = 0x02, characters = "xtension whic">
                <slot #1, id = 0x01, characters = "My Big File.E">
                <directory entry, name = "MYBIGFIL.EXT">

           .. note:: Note that the slots are stored from last to first.  Slots
		     are numbered from 1 to N.  The Nth slot is ``or'ed`` with
		     0x40 to mark it as the last one.

        2) Checksum.  Each slot has an alias_checksum value.  The
           checksum is calculated from the 8.3 name using the
           following algorithm::

                for (sum = i = 0; i < 11; i++) {
                        sum = (((sum&1)<<7)|((sum&0xfe)>>1)) + name[i]
                }

	3) If there is free space in the final slot, a Unicode ``NULL (0x0000)``
	   is stored after the final character.  After that, all unused
	   characters in the final slot are set to Unicode 0xFFFF.

Finally, note that the extended name is stored in Unicode.  Each Unicode
character takes either two or four bytes, UTF-16LE encoded.