The Unix File System
Only One Kind of File: Byte Streams
A file is a stream (a linear array) of 8-bit bytes.
Any byte or contiguous sequence of bytes may be read or written,
regardless of any underlying block, track or cylinder structure in the
hardware. A single read or write can access any number of bytes:
there is no maximum size, and it needn't correspond to the underlying
An file opened by a process is identified by a file
descriptor which is private to that process. Associated with
each file descriptor is a file pointer which records the
current location in the file. A read will read bytes
starting at the file pointer, and will increment the file pointer by
the number of bytes read. A write will write bytes to the
location specified by the file pointer and likewise increment it. The
file pointer can be adjusted to any location (even beyond the end of
the file) by a seek operation.
Text files and binary files are the same. Whether or not a program
can handle binary files just depends on that particular program. All
files are stored the same way (as a stream of bytes): text files,
database files, executable programs, source code, etc.
No Record Orientation That You Don't Impose Yourself
Unix files have no record orientation. A program can impose any
record orientation it likes: to work with 80-column records, simply
read and write 80-byte chunks. However, very few Unix programs use
fixed-length records, because they're so inflexible, and it's nearly
as easy to deal with variable length lines in text files.
Text files are simply a byte stream where a particular ASCII value,
the newline or linefeed character (octal 012; decimal 10) is used to
separate lines. Note that, conventionally, Unix does not use carriage
return / linefeed pairs to separate lines. Just a single newline is
used, which makes programming much simpler. Devices that require the
concept of a carriage return (terminals, printers) simply map newline
to whatever is required.
End of File
There is no end of file character in a Unix file. The file system
records the length of every file and simply reports an end of file
state when end of file is reached.
Directories and Filenames
Files don't have names, they have I-numbers. I-numbers
index an array of inodes stored in each file system.
Inodes contain useful information about files:
- Type of file
- Plain file
- Named pipe or fifo
- Symbolic link
- Link count
- Owner permissions
- Group permissions
- Other permissions
- Size in bytes
- Last access time
- Last modification time
- Last inode change time
- Data block pointers
A directory is a mapping from I-number to a filename.
Directories are stored in ordinary byte-stream files, but they are
marked as directories in the inode so that only the kernel can write
to them (any process can read a directory).
Since directories are ordinary files, they have inodes and hence
I-numbers. This means a directory can contain the I-number of another
directory. This provides a graph-structured file system.
Since general cycles in the file system would make file system
traversal difficult, the kernel disallows them. However, the kernel
actually maintains two restricted types of cycles in each directory:
The kernel assures that every directory contains two entries, one
mapping the filename . to the directory's own I-number, and one
mapping the filename .. to the directory's parent's I-number.
This means that every directory can be reached by traversing other
directories in the file system; in other words, the graph of the file
system is connected.
- A link to the current directory
- A link to the parent directory
There is a distinguished directory called the root
directory; all other directories are descendants of this directory.
The name of the root directory is /.
Users usually identify files not by I-number, but rather by
pathname. A pathname is conceptually a sequence of one or
more directory names, always starting with the root directory, and
ending (optionally) with the name of a non-directory. (If the
pathname doesn't end with a non-directory, it names a directory rather
than a plain file.) These directory names and the final optional file
name are components of the pathname. The pathname
specifies a path to the file of interest from the root, naming
explicitly every directory that must be traversed to reach it.
Pathnames are spelled with slashes (/) separating
the path components. Since the name of the root directory is
/, a leading slash is taken to name the root
directory, and all the other slashes are separtors.
To simplify the use of pathnames, there is an alternate form called
relative pathnames (the form already described is called
absolute pathnames). A relative pathname is any pathname
which does not start with a slash. Since it doesn't start
with a slash, it doesn't (necessarily) start at the root: how do we
decide where it starts? Simple: a relative pathname starts at the
current working directory.
Current Working Directory
Every process has a current working directory (CWD), which is used to
resolve relative pathnames. The CWD of a process is inherited by it's
child processes, but any process can change its CWD at will. The
login program sets the CWD of the shell to the user's home
directory. Other than this behavior, and the fact that a user's
home directory is arranged to be owned by and writable by that user,
there is nothing special about the home directory (though certain
programs may assume that they can find certain files (typically
configuration files) there).
If the presence of an I-number in a directory names that file, what's
to prevent an I-number from appearing in that directory multiple
times, or, from appearing in other directories? Nothing: Unix files
can have many names. Each directory entry is called a
link, and so a file can have many links.
Special File Descriptors
Every process already has three files open when it is created. Hence,
it has three file descriptors in use. These open files are called
standard input (
standard output (
standard error (
This allows a style of program which functions as a simple
filter, reading bytes from standard input, processing them,
and writing the result to standard output. Standard error is used to
report errors, so that they aren't mixed up with the output.
No program is required to be written in acordance with this model: a
process is free to ignore these open files and process named files
instead. But many Unix programs are written as filters and this
allows them to be connected in a pipeline, standard output
to standard input, to build complex applications out of a simple chain
Files as Abstractions
A file is an abstraction through which we can access data on disk.
The kernel requires us to use a small number of system calls to access
files; these system calls define the file abstraction. The basic
operations on files are:
Creating A File
A file is created by creating its inode. This can only be done via
the kernel (through a system call). The kernel insists that a
pathname (absolute or relative) be specified when creating an inode,
so creating an inode also creates a link in a directory. This means
that every file has at least one name.
Removing A File
There is no way to directly remove an inode. The kernel provides a
system call to remove a link from a given directory. This does not
necessarily remove the inode or free the data blocks associated with
the file, because the file may have more then one name and thus there
may still be other links to the file in other directories.
For this reason the kernel maintains a link count in the
inode, which serves as a reference count for the inode and data
blocks. Whenever a file is unlinked, it's link count is decremented.
When the link count goes to zero, the inode and data blocks are freed.
This is another reason why general cycles are not allowed in the file
system: a cycle would throw off the link count and result in unused
structures that couldn't be freed.
Since directories contain the all important links, they can't be
simply removed (unlinked) like plain files. A special system call
must be used to remove a directory, and it will only succeed if the
only links in the directory are
Opening a File
A file is opened by specifying a pathname and a mode. The mode can be
An optional parameter allows the file to be created if it doesn't
exist. In addition, one can specify truncation on open, synchronous
writes, etc. The open system call returns a file descriptor.
- read only
- write only
- read / write
Closing a File
All file descriptors are automatically closed when a process exits,
but the close system call can also be called explicitly. It takes a
file descriptor as a parameter. Closing a file frees locks, shuts
down network connections, etc, as appropriate.
Reading a File
The read system call takes a file descriptor, a pointer to a buffer,
and a count of the number of bytes to read. Any number of bytes can
be read with one read. If you are reading from a pipe, fifo, or
socket, the read will (by default) block if there are not yet as many
bytes to read as you requested.
Writing a File
The write system call takes the same arguments as the read system
call, but assumes the buffer contains data to be written and writes as
many bytes as requested.
Every inode has a set of access permissions associated with it. Unix
access permissions are very simple; the kernel doesn't support
anything like access control lists (ACLs). This simplicity makes it
easy to program and understand file permissions, and easy to predict
the ramifications of combining different levels of access with
different owners -- something which is difficult on more complex
systems. ACLs, when needed, are implemented with user mode code.
There are three levels of access permissions:
These are combined with three possible actions:
These combine to make 512 possible combinations, which is already a
lot: imagine the complexity of systems that use more elaborate access
User permissions specify what the owner of the inode can do
to the file, group permissions specify what anyone in the
group of the inode can do, and other permissions cover
Read access allows you to read the contents of the files
data blocks, write access allows you to modify the contents
of those data blocks, and execute permission allows you to
execute the contents of a file as a program. (It's possible to be
able to execute a file without being able to read it.)
Some operating systems also have the notion of file create
and file delete access: in Unix, this is simply controlled
by permissions on the containing directory. To do either operation
requires write access to the containing directory (because creating or
removing a link really does require write access to the data blocks of
the directory file).
Similarly, Unix doesn't need list directory access, because
this is controlled by read permission on the directory itself.
However, execute permission doesn't make sense for a directory (which
can never contain executable code), so this bit is overloaded with
another meaning: it controls the ability to use a directory as a
component of a pathname.
These Unix file permissions (sometimes known as the file
mode) are stored in the inode as nine bits:
a read, a write and an execute bit for each of user, group and other.
ls program (when invoked with the
option) actually displays the permissions as
with hyphens replacing the bits that are off. The permission bits are
also often expressed as three octal digits, since it's so easy to
compute the bits that way. (Honest!)
File System as Name Space
A name space is a mapping from names (strings) to objects
of some kind. A file system is a name space, where the names map to
files on the disk. But operating systems need name spaces for other
things as well: processes, network connections, terminals, tape
drives, printers, etc. All these things need names or identifiers.
One of the most innovatove and successful aspects of the design of
Unix is the use of the file system as a unifying name space for things
other than disk files.
In Unix, all devices exist in the file system, traditionally located
/dev. In addition, some interprocess communication
mechanisms (fifos and Unix domain sockets) are located in the file
system. This means that the file abstraction can be used to
manipulate these devices, simplifying programming. In Unix, writing
programs to manipulate disk files, tape drives, or network connections
are made as much alike as possible.
Earlier I said that Unix only has one type of file. But the inode has
bits for the file type. Now we see how this apparent contradiction is
resolved. The other types of files are actually other objects in the
file systems name space, which can be be manipulated with the file
abstraction. So even though these things are not disk files, but tape
drives, terminals and network connections, they can be manipulated as
files by opening and closing them, and reading and writing them.
Character Special Files
Character special files represent devices that can be
accessed a byte at a time, like a terminal, disk partition, tape
drive, etc. These are also known as raw devices.
Block Special Files
Block special files represent devices that can be accessed
by blocks, which are buffered by the kernel. These are also known as
cooked devices (because they're not raw). These are
typically disk partitions.
Note that disk partitions are represented in three ways under Unix: as
a collection of plain files in the file system, as character special
devicess, and as block special devices. You can use whatever method
of access is approproate. Usually, we access the disk partition
through the file system, exploited the kernel-provided name space,
caching, access control, etc. But disk backups are usually performed
by opening the raw device for each disk partition and dumping it to
tapes, to gain extra speed by bypassing the overhead of going through
the kernel. In addition, special purpose file systems (say for a high
performance DBMS) can be built on a disk partition and accessed
(portably) via character or block special files.
Note that the permission to access the raw disk partition directly is
controlled, simply and elegantly, by the usual Unix access permissions
on the special files.
Named Pipes (FIFOs)
A named pipe or fifo is an extremely simple and
elegant interprocess communication (IPC) mechanism. When opened for
writing by a process, it causes that process to block until another
process opens the same fifo for reading. Then the two processes have
successfully rendezvoused and whatever bytes are written to the fifo
by the writer can be read by the reader. The bytes aren't stored on
the disk, but rather in kernel buffers. This makes Unix IPC extremely
The University of Chicago Library
This page last updated: Mon Aug 8 17:46:07 CDT 1994
This page was generated from Extended HTML by xhtml.