Chapter 2. Class #2 - Linux Filesystem Management

Table of Contents

The Key Filesystem Concept
Linux Filesystem Hierarchy Standard
Working with Filesystem Partitions
Creating & Formatting Partitions
Mounting & Managing Filesystems
Working With Swap
LUKS Encryption
LVM (Logical Volume Management)
File System Permissions
Viewing & Changing Permissions
Critical File Permission Commands
Umask
Access Control Lists
Reference Material for this Chapter

The term "filesystem" can create confusion. There are several meanings for the word "filesystem". Let's take a minute to disambuguate this concept. Please consider the following:

The way files are physically written to storage devices, as in the ext3, FAT32, NTFS filesystems, etc.
Example: "Create a VFAT filesystem on a USB drive if you want a device that works for both Windows and Linux."
The collection of files and directories stored on a particular storage device.
Example: "On any device using Ext 2/3/4, you should find a "lost+found" directory at the root level of the filesystem."
The unified directory structure which logically organizes files.
Example: "In contrast to Windows, which accesses drives with various drive letters, on Linux all storage devices are mounted into a single filesystem." [3]
The role which defines how directories should be structured and utilized in Linux.
Example: "In a Linux filesystem, third party applications should generally be installed in /opt."

The Key Filesystem Concept

When considering filesystems, it is wise to keep in mind the distinctions made above. Even further, no matter what the level of filesystem you're working with, the following layered concept must be kept in mind. First one must have a DISK or CONTAINER of some type. Then one must create a PARTITION on the disk. Finally, one FORMATS the partition with a FILESYSTEM of some type.

[Note] A Filesystem on a Partition

A "filesystem" as it is created on a "partition" is only one of the "types" of filesystems listed above. This is an important distinction, and one must be able to remain clear about the meaning of the word "filesystem" by the context in which the word is used.

Linux Filesystem Hierarchy Standard

The directory structure of a Linux system is standardized through the Filesystem Hierarchy Standard . The Linux Manual system has an abbreviated reference. Enter `man 7 hier` at the command line to reference locally installed material about the Filesystem Hierarchy Standard . Red Hat has a more complete description, along with Red Hat-specific implementation decisions in their Storage Administration Guide at Red Hat Filesystem Hierarchy Standard .

Figure 2.1. FHS 1

FHS 1

[Important] What the FHS Means Practically

The Filesystem Hierarchy Standard is a very important concept in Linux. It means that once a person becomes familiar with the locations of critical operating system elements in Linux, these locations should be consistent across Linux distributions. This practice, of course, is only as effective as the distribution chooses to implement the Filesystem Hierarchy Standard .

Working with Filesystem Partitions

Listed below is an overview of the process for using Basic Storage Devices. The remainder of this chapter will expand on this series of events in detail. We will also diverge from "Basic Storage Devices" later to use Logical Volumes and Encrypted Partitions.

  1. Install the device or otherwise make it available to the system.
  2. Create a partition with `fdisk` or `parted`.
  3. Create a filesystem on the partition with `mkfs` or other tools.
  4. Choose or create a directory to serve as a mount point.
  5. Mount the partition.
  6. Add an entry to /etc/fstab to make it persistent.

Creating & Formatting Partitions

The tools listed below are used for creating, formatting, and manipulating disks and partitions. Please note that these tools are command line utilities. We will use these tools to create partitions in the exercises that follow.

  • `fdisk` or `parted` - Used to partition hard disks or other block devices.
  • `mkfs` and variants - Used to create filesystems on block devices (actually a front-end for a variety of FS-specific tools).
  • `fsck` and variants - Used to run filesystem checks (a front-end to FS specific tools).
  • `mount` - Used to mount a filesystem to a specific location in the directory structure.
  • /etc/fstab - Configuration file used to describe the filesystems that should be persistently mounted.
  • `blkid` - used to identify filesystems or other in-use devices by UUID or filesystem labels.
  • `df` - used to display the capacity and utilization % of mounted filesystems.
  • `partx` - used to force implementation of a new partition table on an in-use device w/o the need to reboot.
  • `partprobe` - precursor to `partx`, performs the same function.
  • `tune2fs` - adjust tunable filesystem parameters on ext2/ext3/ext4 filesystems.
  • `resize2fs` - ext2/ext3/ext4 file system resizer.
  • e2fsprogs - see the e2fsprogs definition website for more info.

Mounting & Managing Filesystems

Once a disk has been formatted, partitioned and a filesystem has been created, it must be mounted in order to be used. There are several methods of mounting partitions, namely using UUIDs, filesystem labels, or referring directly to the partition.

The following listing shows the output of the command `cat /etc/fstab` which gives examples of the above:

    /dev/mapper/luks-e37f48b1-9d4b-4512-8f89-90e34c668ed6 /                       ext4    defaults,x-systemd.device-timeout=0 1 1
    UUID=ffcde308-6b25-4dc9-9715-7decabf9362d /boot                   ext4    defaults        1 2
    UUID=1BE9-0EAF          /boot/efi               vfat    umask=0077,shortname=winnt 0 0
    /dev/mapper/luks-6c2d8527-7da8-4cef-a0df-a8e37ec9f60f /home                   ext4    defaults,x-systemd.device-timeout=0 1 2
    /dev/mapper/luks-98a8da86-474c-46fe-bf9e-c41837a5d9a4 /mnt/bernate            ext4    defaults,x-systemd.device-timeout=0 1 2
    /dev/mapper/luks-79710f5d-fc6e-4e4a-b0fb-333f45845436 /mnt/vm_images          ext4    defaults,x-systemd.device-timeout=0 1 2
    UUID=0d3c723f-c650-4164-b7db-d9bd0c69bea6 swap                    swap    defaults        0 0
  

According to the objectives listed on Red Hat website, candidates for the RHCE Information exam should be able to accomplish the following:

  • Configure systems to mount file systems at boot by Universally Unique ID (UUID) or label.
  • Add partitions to create additional storage space.
  • Add new partitions, logical volumes, and swap to a system non-destructively.
  • Create, mount, unmount and use ext2, ext3 and ext4 file systems.
  • Extend existing unencrypted ext4-formatted logical volumes.

Working With Swap

Swap space can be manually built and added to the system in two ways. We will first look at how to create swap space from disk partition.

How to Add SWAP Using a Partition

  1. Create a type 82 partition
  2. Initialize as swap with `mkswap /dev/<partition>`
  3. Identify the UUID with `blkid`
  4. Add an /etc/fstab line: UUID=<UUID> swap swap defaults 0 0
  5. Activate the new swap space with: `swapon -a`

How to Add SWAP Using a File

  1. Create a pre-allocated file of the desired size: `dd if=/dev/zero of=/path/to/<swapfile> bs=1M count=<size in MB>`
  2. Initialize as swap with `mkswap /path/to/<swapfile>`
  3. Add an /etc/fstab line: /path/to/<swapfile> swap swap defaults 0 0
  4. Activate the new swap space with: `swapon -a`

LUKS Encryption

LUKS is the default encryption scheme used in Red Hat (and derivatives). To see if the tools are installed, run the command `rpm -qa | grep crypt`. In order to install the tools needed, run the command `yum -y install cryptsetup`. This should install a package named cryptsetup-luks-1.1.2-2.el6.x86_64 (or something to that effect depending on the version, etc.). With the pre-game necessities in place, we can look at the process of creating an encrypted partition on disk.

[Note] This Encryption Scheme Works Everywhere

It's worth noting that the following steps can be used to encrypt a partition on a physical hard disk or on a USB drive.

  1. Create a new partition.
  2. Encrypt it with `cryptsetup luksFormat /dev/<partition>`.
  3. Open the encrypted device and assign it a name with `cryptsetup luksOpen /dev/<partition> <name>`.
  4. Create a filesystem on the named device (/dev/mapper/<name>) - `mkfs -t ext4 /dev/mapper/<namne>`.
  5. Create a mountpoint for the device.
  6. Mount the device.

To lock the volume (put the encryption layer back in place):

  1. Unmount it.
  2. Use `cryptsetup luksClose <name>` to remove the decryption mapping.

To persistently mount the LUKS device:

  1. Create an entry in /etc/crypttab thus: <name> /dev/<partition> <password (none|<blank>|<path/to/file/with/password>)>
  2. If the password field is "none" or left blank, the system will prompt for a password.
  3. Create an entry in /etc/fstab

LVM (Logical Volume Management)

Logical Volume Management is a way to "aggregate" physical disks into "volumes" and create "logical" partitions that span the disks. Red Hat (and derivatives) have an excellent system for achieving this method of disk management. Although this system is a bit confusing at first, it's an excellent way to manage storage space. It's important to keep in mind the following series of events:

  1. Physical Disks are created, then incorporated into...
  2. ...Volume Groups, which then are subdivided into...
  3. ..Logical Volumes.
  4. The Logical Volumes are then manipulated in the same manner as any other physical partition: formatted, encrypted, mounted, etc.

Here is an overview of the process for using Logical Volume Management:

  1. Install the physical device(s) or otherwise make them available to the system.
  2. Create a type 8e partition with `fdisk` or `parted`.
  3. Initialize the partition as a Physical Volume with `pvcreate`.
  4. Add the storage of the PV to a Volume Group with `vgcreate`.
  5. Allocate storage from the volume group to a logical volume with `lvcreate`.
  6. Create a filesystem on the Logical Volume with `mkfs` or other tools.
  7. Choose or create a directory to serve as a mount point.
  8. Mount the partition.
  9. Add an entry to /etc/fstab to make it persistent.

Removing a Logical Volume (and a Volume Group) is essentially the opposite process as creating one. Here's an overview of removing Logical Volume structures:

  1. Unmount the LV you want to remove.
  2. Edit /etc/fstab to remove its entry.
  3. Remove the Logical Volume: `lvremove /dev/<vg>/<LV>`
  4. Before removing a VG, ensure there are no more LVs within it.
  5. Remove the Volume Group: `vgremove /dev/<vg>`
  6. Remove the LVM signature from the partitions: `pvremove /dev/<partition>`

File System Permissions

We will consider file system permissions in two categories: Basic & Extended.

Basic File System Permissions

Linux permissions are organized around three sets of permissions. Any given file or directory can be owned by one (and only one) user and one (and only one) group. Three different sets of permissions can be assigned.

  1. User - User permissions apply to the individual user who owns the file or directory.
  2. Group - Group permissions apply to any user who is a member of the group that owns the file or directory.
  3. Other - Other permissions apply to any user account with access to the system that does not fall into the previous categories.

For the three types of permissions, there are three possible types for each set:

  1. Read ("r") - On a file, allows reading; on a directory, allows listing.
  2. Write ("w") - On a file, allows editing; on a directory, allows creation and deletion of files
  3. Execute ("x") - On a file, allows execution if the file is otherwise executable (script or binary); on a directory, allows entry or traversal (`# cd <dirname>`)

Extended File System Permissions

Further, each of the permissions can be "extended" with and additional attribute:

  1. SUID (Set User ID) - On an executable, runs a process under the UID of the file owner rather than that of the user executing it.
  2. SGID (Set Group ID) - On a directory, causes any files created in the directory to belong to the group owning the directory. On an executable, runs a process under the GID of the group owning the file rather the logged-in group of the user executing it.
  3. Stickybit - On a directory, ensures that only the owner of a file or the owner of the directory can delete it, even if all users or other members of a group have write access to the directory.

Listed below are the steps for using SGID and Sticky Bit to create a collaborative directory:

  1. Create a group for collaboration.
  2. Add users to the group.
  3. Create a directory for collaboration.
  4. Set its group ownership to the intended group.
  5. Set its group permissions appropriately.
  6. Recursively set the SGID and sticky bits on the directory.

This ensures that:

  • All files created in this directory will be owned by the intended group (SGID effect).
  • All files created in this directory can only be deleted by the user who owns the file or the user who owns the directory (stickybit effect).
Extended Attributes in Directory Listings
    -rwxrwxrwx  Normal Permissions, All permissions granted or 777 permission
    -rwSrwxrwx  Indicates SUID set
    -rwsrwxrwx  Indicates SUID and execute permission set
    -rwxrwSrwx  Indicates SGID set
    -rwxrwsrwx  Indicates SGID and execute permission set
    -rwxrwxrwT  Indicates Stickybit set
    -rwxrwxrwt  Indicates Stickybit and execute permission set
  
[Warning] Danger of 777 Permissions

The 777 permission, when set on a directory, gives EVERYONE full permission (particularly WRITE and EXECUTE) on the directory. While - in a very few instances, this is desirable. However, for most cases it is very much not what is desired in the long term. It's often a short term solution to a problem that creates long term problems that can be much bigger.

Viewing & Changing Permissions

The series of images below describe the wealth of information that is contained in file system listings. Some of this information is about the files and some is about the permissions applied to those files. Remember, in Linux "everything is a file".

Figure 2.2. Directory Listing Showing Permissions

Directory Listing Showing Permissions

The figure above shows the output of the command `ls -al`. Note the wealth of information that is displayed over and above a simple `ls` command.

Figure 2.3. "Hidden" Files & Soft Links

"Hidden" Files & Soft Links

This figure shows hidden files as well as "soft" links.

Figure 2.4. SUID Bit Shown in Listing

SUID Bit Shown in Listing

The figure above shows the file with SUID extended attribute in the listing.

Figure 2.5. Block & Character Devices and the "Sticky Bit"

Block & Character Devices and the "Sticky Bit"

This figure shows "block" and "character" devices as well as the "sticky bit" set on a directory.

Figure 2.6. SUID Bit on the `ping` Command

SUID Bit on the `ping` Command

A perfect example of using SUID is with the `ping` command, which must be run as root, but executed by anyone on the system.

Figure 2.7. Directories with the "Sticky Bit"

Directories with the "Sticky Bit"

Further examples of directories with the "sticky bit".

Figure 2.8. Permissions in a Nutshell

Permissions in a Nutshell

The image above condenses all that we've covered concerning file permissions.

Critical File Permission Commands

The following list shows some of the commands that can be used to alter pernissions on files and directories:

  1. chmod a+x </path/to/file/or/directory> - adds the "execute" bit to a file.
  2. chmod u=rw,g=r,o-rwx ./myfile.txt - multiple operations in one pass.
  3. chmod 755 </path/to/directory> - numeric method of changing the permissions
  4. chmod 3775 </path/to/directory> - numeric method of adding extended permissions (SGID + Sticky Bit) to a directory

Umask

The umask value determines the permissions that will be applied to newly created files and directories. As a "mask" it is subtractive, representing the value of the permissions you DO NOT want to grant. Execute rights are automatically withheld (w/o regard for the umask) for files but not for directories. Extended attributes are not addressed, even though a umask is four characters. The default umask value is set in /etc/bashrc and can be modified (non-persistently!) with the bash built-in command `umask`.

[Note] umask Oddity

A umask of 0002 yields permissions of 0775 on new directories and 0664 on new files. Changing the umask value does not retroactively change permissions on the existing files and directories in the system.

Access Control Lists

File access control lists provide more granular control of user and group permissions. The filesystem must be mounted with the 'acl' option or be configured with that option by default. Here's the "quick list" sequence of events for using access control lists:

  1. Pass the `acl` option to `mount` (non-persistently) with ACLs enabled.
  2. Add acl in the options field of /etc/fstab to persistently enable ACLs.
  3. Or use `tune2fs -o user_xattr,acl /path/to/device` to configure those attributes as default mount options that are held in the filesystem accounting table.

Access Control List Commands

Here are the two crucial commands for manipulating filesystem access control lists:

  1. `getfacl <directory or file>` - Used to view file ACLs.
  2. `setfacl <directory or file>` - Used to establish or change file ACLs.

Example output of the command `getfacl <acldir>`:

    # file: acldir
    # owner: frank
    # group: frank
    user::rwx
    user:bob:-wx
    user:mary:rw-
    group::rwx
    mask::rwx
    other::r-x
  

Example output of the command `ls -l <acldir>`:

    drwxrwxr-x+ 2 frank frank 4096 2009-05-27 14:15 acldir
  

Reference Material for this Chapter

For this chapter's supporting material, please reference Chapters 6 & 8 in the RHCSA/RHCE Linux Certification Study Guide text book.