A brief Introduction to Cgroups in Linux

What is it?

cgroups (abbreviated from control groups) is a Linux kernel feature¹ that limits, accounts for, and isolates the resource (i.e CPU, memory, disk I/O, and network) usage of a collection of processes. It is essential for managing resource allocation in modern systems, especially in environments that require efficiency and isolation like containers, virtual machines, and multi-user servers.

We can create multiple cgroups with parent-child relationships, as it is organized in a hierarchical structure. Each cgroup can inherit or override resource settings from its parent. To manage specific resources, subsystems (also called controllers) are used. Here are some common subsystems:

• CPU (cpu, cpuacct): Control CPU usage.
• Memory (memory): Limit and monitor memory usage.
• Block I/O (blkio): Manage disk I/O.
• Devices (devices): Control access to devices.
• Network (net_cls, net_prio): Manage network bandwidth and prioritize network traffic.

As of writing, there are two versions of cgroups.

• cgroups v1: The original version with a separate hierarchy for each resource type.
• cgroups v2: A more unified and simplified interface, providing a single hierarchy for all controllers.

Currently, most Linux distributions use cgroups v2 by default. Below is how we could check which cgroup version is in use on a Linux system.

ls /sys/fs/cgroup

As the output of above, if we see a single file called cgroup.controllers and other files like cgroup.procs, cpu.max, cgroups v2 is in use. If we see a set of directories like cpu, memory, blkio, etc., it means our system is using cgroups v1.

How does it work?

cgroups enable the grouping of processes and the enforcement of limits on how much of a system’s resources these groups can use. This is achieved by organizing processes into hierarchical groups, where each group has a set of resource rules or limits. Each cgroup can then be configured independently, with resource usage tracked and controlled at the group level.

Below are some of the features that cgroups offer.

• Limit the amount of resources a process group can use. For example, we can limit the CPU usage to 50% for a group of background processes, ensuring they don’t slow down more critical tasks.
• Set priority levels to ensure that some groups of processes receive more CPU or I/O resources than others. This is useful for ensuring critical applications get the resources they need.
• Track how much resources each group is using. This helps in monitoring, performance tuning, and understanding resource usage patterns.
• Create isolated environments where groups of processes cannot interfere with each other’s resources. This is a crucial feature for containers, where applications run independently on shared hardware.
• Easily freeze, stop, or restart groups of processes. This can be particularly handy for pausing resource-heavy tasks.

Where is it used?

cgroups is a key component in enabling containerization technologies like Docker, Kubernetes, and LXC (Linux Containers). They use cgroups to isolate and manage containers, ensuring each container gets a defined share of the host’s resources. Also, in multi-tenant environments, where multiple users or applications share the same server, cgroups help allocate resources fairly, preventing any single user or application from monopolizing system resources. Additionally, cgroups is crucial for system performance tuning, allowing administrators to improve responsiveness by assigning lower CPU priority to non-critical background processes, ensuring that essential tasks receive the resources they need.

A quick example?

To manage cgroups, we interact with files in a virtual filesystem called cgroupfs. On modern systems (with cgroups version 2), this is typically mounted under /sys/fs/cgroup.

1. Creating a cgroup:

   # Create a new cgroup under the 'memory' subsystem
   mkdir /sys/fs/cgroup/memory/my_cgroup
   

2. Setting a Limit:

   # Limit memory usage to 512MB for the cgroup
   echo 536870912 > /sys/fs/cgroup/memory/my_cgroup/memory.limit_in_bytes
   

3. Adding a Process to a cgroup:

   # Add a process (PID 1234) to the cgroup
   echo 1234 > /sys/fs/cgroup/memory/my_cgroup/cgroup.procs
   

Learning about cgroups can come in handy when working with containers, virtual environments, or managing server resources.