Introduction to Cgroups, the Linux Control Group

Cgroups is present in the official Linux kernel 2.6.24 (late 2007), still he’s not much know or used (at least for what i know).
In this article I’ll give you an overview of this powerful Linux tool to control how much CPU, memory, disk I/O or network I/O each process or user can use in your server.

So in short cgroups it’s a feature to limit, account and isolate resource usage (CPU, memory, disk I/O, etc.) of process groups.
Let’s see how.

Theory

A control group is a collection of processes that are bound by the same criteria. These groups can be hierarchical, where each group inherits limits from its parent group. The kernel provides access to multiple controllers (subsystems) through the cgroups interface.

The infrastructure of Cgroups provides only the features of grouping the tasks, while the various Cgroups subsystem implements the specific control policies for each resource. This framework it’s really powerful and allow you to set rules on resources not only based on users and groups.

For example you could:

To keep a Web server from using all the memory on a system that’s also running a data base.
To allocate system resources among user groups of different priority (e.g. guests 10% of the CPU, users 40% of the CPU, powerusers 50% of the CPU )

But Cgroups can be used also to isolate and give special command to groups of processes, so we can say that there are 2 kind of subsystems

1. The Isolation and Special Control Subsystems
2. The Resource Control Subsystems

I’ll focus on the control of resources so this is a small overview of the Isolation and Special Control Subsystems:

• The CPUset : assigns individual CPUs and memory nodes to cgroups.
• The Namespace : provides a private view of the system to the processes in a cgroup, and is used primarily for OS-level virtualization.
• The Freezer : stops all the processes in a cgroup from executing by removing them from the kernel task scheduler.
• The Device : allows or denies access to devices to processes in a cgroup.
• The Checkpoint / Restart : Save the state of all processes in a cgroup to a dump file. Restart it later (or just save the state and continue).

The administration of Cgroups is performed by the use of a special virtual file system (sort of procfs or sysfs), trivially called cgroupfs.

Cgropus in practice

Unless you have a server release of some years ago (red hat and centos 5.X for example), you should have a kernel able to use Cgroups, in general check if your kernel is >= of 2.6.24 with the command uname -a this is the requisite to use cgroups.

Than you need the tools in user-space, i’m doing the tests on my Ubuntu, on this idistribution the files to be installed are cgroup-bin and libcgroup1 :

sudo aptitude install cgroup-bin libcgroup1

Once installed you’ll have a new filesystem mounted:

# ls /sys/fs/cgroup
cpu  cpuacct  devices  memory

On other Linux distributions you could need to mount it manually with a command like :

#sudo mount -t cgroup none /mnt

You can also mount only certain Cgroups subsystem by adding the -o cgroup_name option to the mount command.


Back on Ubuntu (and Debian) you have a file that can be used to choose which cgroups subsystem mount, it’s /etc/cgconfig.conf , inside you’ll find the standard configuration:

mount {
        cpu = /sys/fs//cgroup/cpu;
        cpuacct = /sys/fs/cgroup/cpuacct;
        devices = /sys/fs/cgroup/devices;
        memory = /sys/fs/cgroup/memory;
}

This basically tell which subsystem use and where to mount them.
We have already saw the devices subsystem the others in the example are:

cpu : this subsystem uses the scheduler to provide cgroup processes access to the CPU.
cpuacct : this subsystem generates automatic reports on CPU resources used by processes in a cgroup.
memory : this subsystem sets limits on memory use by processes in a cgroup, and generates automatic reports on memory resources used by those processes.

To list which types of Cgroups are known use the following command:

#cat /proc/cgroups 
 
#subsys_name	hierarchy	num_cgroups	enabled
cpuset	0	1	1
ns	0	1	1
cpu	1	1	1
cpuacct	2	1	1
memory	4	1	1
devices	3	1	1
freezer	0	1	1
net_cls	0	1	1
blkio	0	1	1

Example : Create 2 cgroups

So far we have saw a lot of information, it’s time to see a practical example.
We’ll create 2 control groups called Browsers and Multimedia and we’ll set cgropus so that Browsers can use at max half the shares of CPU used by Multimedia. This is done because we want to ensure that always the multimedia respond within a certain time, to do not miss a frame or hear the annoying “jumps” even when the rest of the system is overloaded.

To do it we have 2 ways, working on the filesystem and using some special commands from the command line.

First method it’s to use the meta-filesystem of cgroups.

The first thing to do it’s create the 2 cgroups, that are nothing more than a directory on our filesystem.
We want to work with Cpu subsystem, so we’ll create the 2 directory in that path:

># cd sys/fs/cgroup/cpu
># mkdir Browsers
># mkdir Multimedia

At this point we have to tell the system that the Cgroups Multimedia should have twice the weight than the Browsers, we do this by creatings within the group a file named cpu.shares . that contains only a value, and giving a value to Multimedia that is twice that of Browsers:

$ echo 2048 >  /sys/fs/cgroup/cpu/Multimedia/cpu.shares
$ echo 1024 >  /sys/fs/cgroup/cpu/Browsers/cpu.shares

We are done, well almost.
Now we must say which processes belong to the cgroup Multimedia and which one to the cgroup Browsers.
To move a processes into a cgroup his PID must be listed in the file /path_to_cgroup/tasks, so we could simply do something like this:

$ firefox &
$ echo $! >  /sys/fs/cgroup/cpu/Browsers/tasks
 
$ mplayer music.mp3 &
$ echo $! >  /sys/fs/cgroup/cpu/Multimedia/tasks

And now we are really done.

Second method it’s to use a set of commands

Creations of the 2 cgroups is done with the command cgcreate
Syntax is

cgcreate -t uid:gid -a uid:gid -g subsystems:path

So to create the same 2 cgroups we have to give the commands:

cgcreate -g cpu:/Browsers
cgcreate -g cpu:/Multimedia

Now we must give a weight to MultiMedia that is the double of the value of Browsers, to do it we use the command cgget
The syntax for cgset is: cgset -r parameter=value path_to_cgroup, so in our example we’ll give:

cgset -r cpu.shares=1024 Browsers
cgset -r cpu.shares=2048 Multimedia

To move a process into a cgroup you can use the command cgclassify
The syntax for cgclassify is: cgclassify -g subsystems:path_to_cgroup pidlist

So if our browser has pid 1234 and our multimedia application has pid 9876 we should use these commands:

cgclassify -g cpu:Browsers 1234
cgclassify -g cpu:Multimedia 9876

And that’s all, we have successfully created 2 cgroups where, for the cpu, one cgroups use the double of shares of the other.
We could use some similar rules also for memory (so the web server don’t eat all the memory of our database), network or disk I/O.

Naturally you can also mix the various cgroup rules so a process has limitations on Cpu and Memory.

The cgred Daemon

Cgred is a daemon that moves tasks into cgroups according to parameters set in the /etc/cgrules.conf file. Entries in the /etc/cgrules.conf file can take one of the two forms:

maria			devices		/usergroup/staff

maria:ftp		devices		/usergroup/staff/ftp

Conclusions

This was just a small example of the many uses that you can have for cgroups.
It’s easiest and most powerful way to limit resources on linux systems.

References

Effective Linux Resource Management, cgroups on Suse
Resource Management on Red Hta Enterprise 6
Cgroups in Debian Squeeze
Linux kernel hacking: contenitori di processi/1

Popular Posts:

None Found