Understanding Linux Containers: A Deep Dive into Modern Virtualization

Containerization has revolutionized the way we deploy and manage applications, and in the world of Linux, LXC (Linux Containers) offers a powerful yet lightweight solution. While LXC provides an excellent foundation for running isolated Linux environments, LXD, a system container manager built on top of LXC, makes managing containers much simpler and more user-friendly. LXD enhances LXC’s capabilities by offering a higher-level API, better management tools, and additional features that make it more suitable for production-scale container deployments.

In this article, we’ll explore both LXC and LXD, explain how they relate to each other, and look at how to use them effectively for modern Linux containerization.

What Are LXC (Linux Containers) ?

LXC (Linux Containers) is an operating-system-level virtualization method for running multiple isolated Linux systems (containers) on a single host. Unlike traditional virtual machines (VMs), which simulate hardware and require a full guest OS, LXC uses the host kernel directly. This results in less overhead, faster performance, and more efficient resource utilization.

In essence, LXC containers share the same kernel but are isolated from one another, providing a lightweight alternative to full virtualization.

Key Features of LXC:

• Lightweight: LXC containers use fewer resources than virtual machines since they do not require a full operating system, only libraries and binaries.
• Isolation: Each container is isolated from others, with its own filesystem, processes, and network stack.
• Compatibility: LXC uses standard Linux tools and works with any Linux distribution.
• Scalable: LXC is designed to run hundreds or even thousands of containers on a single host, making it ideal for large-scale deployments.

How LXC Works

LXC leverages a number of Linux kernel features to provide containerization, including:

Namespaces

Namespaces provide isolation between containers. Each namespace ensures that the container sees only its own processes, network interfaces, user IDs, and filesystems. There are different types of namespaces:

• PID namespace: Isolates process IDs, so processes in different containers can have the same PID.
• Network namespace: Provides network isolation, allowing each container to have its own network interfaces, routing tables, and firewall settings.
• Mount namespace: Ensures containers see their own file system, independent of the host system.
• UTS namespace: Provides isolation for hostname and domain name.
• IPC namespace: Isolates inter-process communication (IPC) resources, such as semaphores and shared memory.
• User namespace: Enables mapping of user and group IDs between the host and the container, providing a higher level of security.

Control Groups (cgroups)

groups are used for resource management. They allow administrators to limit, prioritize, and monitor the resources (CPU, memory, disk, etc.) that each container can use. This ensures fair resource distribution and prevents any single container from consuming excessive resources.

AppArmor / SELinux

LXC containers can also use Linux security modules like AppArmor or SELinux to define security policies that control how containers interact with the system. These tools provide an additional layer of security by restricting what containers can do based on predefined rules.

UnionFS Filesystems

LXC containers use union file systems like OverlayFS or AUFS to share the same base image between containers while allowing each container to have its own read-write layer. This reduces disk usage by enabling multiple containers to share common layers.

Use Cases of LXC

LXC is ideal for a variety of use cases, from development to production. Some of the most common scenarios include:

1. Development and Testing Environments
   Developers can use LXC to create lightweight, isolated environments for testing applications. This is particularly useful when testing different configurations or dependencies across different Linux distributions.
2. Microservices Architecture
   LXC can be used to run microservices, where each service is placed in its own container, ensuring better isolation and resource management. Containers can be scaled independently based on the load, enabling flexible microservices deployment.
3. Resource Isolation
   LXC containers can be used to isolate different applications or processes on the same machine, ensuring that they don’t interfere with one another. For example, you can run multiple databases or web servers on the same host but in isolated containers.
4. Prototyping and Lightweight Virtualization
   Since LXC containers are faster and use fewer resources than virtual machines, they are ideal for prototyping applications or quickly testing different configurations without the overhead of traditional virtualization.

What is LXD?

While LXC provides the underlying containerization technology, LXD is a system container manager that enhances LXC by adding an easy-to-use interface and a set of management tools for dealing with containers in production environments.

LXD was designed to simplify container management, add more robust networking features, and enable better orchestration of containers on a single host or across multiple hosts. LXD is often considered the next step up from LXC, making the process of managing Linux containers more streamlined and user-friendly.

Key Features of LXD:

• REST API: LXD exposes a powerful REST API that allows developers and administrators to programmatically interact with containers, manage resources, and monitor container health.
• LXC Integration: LXD uses LXC as its underlying container technology but adds a more convenient and feature-rich management layer.
• Enhanced Networking: LXD includes better support for container networking, including bridge networking, macvlan, and VLANs, to ensure containers can communicate in a flexible and scalable way.
• Container Clustering: LXD allows users to create container clusters, making it easy to manage containers across multiple physical machines.
• Storage Pools: LXD supports storage backends like ZFS, LVM, and Btrfs, making it easy to set up persistent storage for containers and control disk usage.
• Snap Support: LXD can be installed via Snap packages, providing a quick and easy installation process that works across many Linux distributions.

In essence, LXD makes LXC easier to use by providing high-level management features, better networking, improved resource control, and the ability to orchestrate containers at scale.

How LXC and LXD Relate to Each Other

LXC and LXD are closely related, but they serve different roles:

• LXC is the core Linux container technology that provides isolation, resource management, and virtualization at the kernel level.
• LXD is a container manager that builds on LXC’s capabilities to make container management easier and more scalable, with added features like networking, storage management, clustering, and a REST API.

While you can use LXC directly to create and manage containers, LXD simplifies this process by offering a higher-level abstraction and a rich set of tools for managing containers at scale.

Linux Containers vs Virtual Machines

A common question is whether to use containers or virtual machines (VMs). While both serve the purpose of isolating and running applications, they have significant differences:

For lightweight, ephemeral workloads and microservices architectures, containers are typically preferred. On the other hand, if you need full hardware isolation or are running legacy applications that require their own operating system, VMs may still be the better option.

LXC vs. Docker

While both LXC and Docker provide containerization, they differ in scope and complexity:

• LXC is more focused on providing a full system container that replicates a virtual machine. It’s a more generalized container solution, providing isolation at the operating system level.
• Docker focuses more on packaging applications and their dependencies into containers, providing a more streamlined approach to deploying and scaling applications. Docker is often preferred for microservices and CI/CD pipelines.

Installing and Using LXC and LXD on Linux

Prerequisites
LXC requires a Linux kernel that supports containerization features such as namespaces and cgroups. Most modern distributions already support these features, so you’ll need a relatively up-to-date kernel (3.8 or higher).

Before installing, ensure that the required LXC packages are available in your Linux distribution’s package manager.

Installation

# For ubuntu 
sudo snap install lxd 
sudo apt install lxc 

# For arch linux 
sudo pacman -S lxd lxc

Setting up LXD for First Use

Once LXD is installed, you’ll need to initialize it:

sudo lxd init

This command will guide you through configuring storage pools, networks, and whether you want to create a cluster of nodes. After initialization, you can use LXD to manage containers with commands like lxc launch, lxc exec, and more.

Here’s a straightforward initialization configuration to set up LXD usng Btrfs

'Would you like to use LXD clustering? (yes/no) [default=no]: no
Do you want to configure a new storage pool? (yes/no) [default=yes]: no
Name of the new storage pool [default=default]: pinitial
Name of the storage backend to use (btrfs, dir, lvm) [default=btrfs]: btrfs
Create a new BTRFS pool? (yes/no) [default=yes]: yes
Would you like to use an existing empty block device (e.g. a disk or partition)? (yes/no) [default=no]: yes
Path to the existing block device: /home/bt/mylxc.img
Invalid input: "/home/bt/mylxc.img" is not a block device'

You can create btrfs pool (optional) by

# create 20GB image file 

fallocate -l 20G lxbtr.img

# mount the image file 
sudo kpartx -av lxbtr.img

## find the where [at what /dev/loop?] your image is mounted by :

lsblk

# Create , format and automount /dev/loop1 using lxc command
## from now on automount will be handeled by lxd

lxc storage create mybtrfspool btrfs source=/dev/loop1

# Print all the storage pools 

lxc storage list

# Print all the info about specific pool

lxc storage show mybtrfspool
lxc storage info mybtrfspool

# Set this default storgae pool for default profile

lxc profile device set default root pool=mybtrfspool

Archlinux specific configuration

On arch linux you will also need to set uid and gid in default.conf file by:

# Append the following to /etc/lxc/default.conf
lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536

# your subuid and subgid files should look like this

cat /etc/subuid

# bt:100000:65536
# root:100000:65536

cat /etc/subgid

# bt:100000:65536
# root:100000:65536

# To test the lxc on Archlinux 
lxc launch ubuntu:22.04 ubuntu
lxc exec ubuntu bash

Listing LXD Containers

You can list all the prebuilt containers that are available for download using the command

sudo lxc image list images:

#  List and fillter alpine images 
lxc image list images: alpine
lxc image list images: |rg alpine
lxc image list images: |grep alpine

Launching , Starting , Stopping and deleting containers

The commands lxc launch and lxc start in LXC may seem similar, but they serve different purposes when managing containers. Here’s a detailed breakdown of the difference between launching and starting containers in LXC:

• lxc start : command is used to start an already existing container that has been created previously (either manually or with lxc launch)
• lxc launch : command is used to create and start a container in a single step. It is essentially a combination of two actions: creating the container (using a specified image) and starting it immediately.

# run image with custom container-name
lxc launch <image-name> <container-name>
# run image with random container-name
lxc launch <image-name> 

lxc start <container-name>
lxc stop <container-name>
lxc restart <container-name>
lxc delete <container-name>

# example 
lxc launch images:ubuntu/focal myUbuntu

Run command / bash in container

lxc exec <container-name> -- apt install htop

lxc exec bash <container-name>

Rename container

lxc move old-name new-name

Container configuration show / set

#show config
lxc config show <container-name>

# set config
lxc config set <container-name> boot.autostart 1
lxc config set <container-name> limits.memory 1GB
lxc config set <container-name> boot.autostart.delay 30
lxc config set <container-name> boot.autostart.order 8
lxc config set ubuntu limits.cpu 2

Snapshot — (create , restore and delete)

# create snapshot
lxc snapshot <container-name> <snapshot-name>

# print all the snapshot 
lxc info <container-name>

# delete snapshot
lxc delete <container-name>/<snapshot-name>

# restore from snapshot
lxc restore <container-name> <snapshot-name>

List/print the container , images and storage

# print all the containers
lxc list 

# print all the images
lxc image list

# print all the storage pool
lxc storage list

Create lxc image based on stopped container

lxc stop your-container-name
lxc publish your-container-name --alias your-image-alias

Export and import image to tar.gz

# using single command 
lxc publish your-container-name --alias your-image-alias --force --format=tarball -o your-exported-image-name.tar.gz

## Using 2 steps [First by creating image based on container]
## then by 
lxc publish your-container-name --alias <new-image>
lxc image export new-image-alias

# importing 
lxc image import your-exported-image-name.tar.gz --alias your-imported-image-alias

Bind mount directory

Here’s a rephrased and improved version of your statement:

By default, you can only read from the container but can write only from the host. To allow full read and write access from the container, you’ll need to modify the permissions of the host directory by running chmod 777 on the directory. This will grant both read and write permissions to the container.

# mount
lxc config device add <container-name> <mount-name> disk source=<dir/on/host> path=<dir/on/container>

# unmount
lxc config device remove <container-name> <mount-name>

Lxc — share tty from host to container

# in arch linux gid of wheel is 998
lxc config device add <container-name> ttyUSB0 unix-char mode=0666 gid=998 path=/dev/ttyUSB0

# in ubuntu gid of dialout is 20
lxc config device add <container-name> ttyACM0 unix-char mode=0666 gid=20 path=/dev/ttyACM0 

## Remove
lxc config device remove <container-name> ttyUSB0

Enable/disable autostart of container when system boots up

# enable
lxc config set ubuntu boot.autostart 1

# disable 
lxc config set ubuntu boot.autostart 0

## Here ubuntu is the name of the container

Copy dir/file to lxc container

# push file 
lxc file push  test ubuntu/root/

# push dir
lxc file push -r nvim ubuntu/root/.config

Copy dir/file from lxc container

# pull file
lxc file pull ubuntu/tmp/foo/myfile.txt /tmp/

# pull dir
lxc file pull -r my-container/tmp/foo/ /tmp/

Exporting snapshots to tars

# Create a snapshot of the LXC container:
lxc snapshot container_name snapshot_name

# Convert the snapshot to a temporary image:
lxc publish container_name/snapshot_name --alias exported_image

# Export the temporary image as a tarball:
lxc image export exported_image exported_image.tar.gz

# Delete the temporary image (optional):
lxc image delete exported_image

Run the python/bash script at start up using crontab

You can run your Bash or Python scripts either by using cron jobs or systemd units. Below is an example of how to schedule the script using crontab.

# using crontabs
crontab -e
@reboot /usr/bin/python3 /path/to/your/script.py

# using lxc config

lxc config edit <container_name>

### Add the following line to the lxc.hook.autodev section
sh -c "/usr/bin/python3 /path/to/your/script.py"

Lxc profiles

In LXC (Linux Containers), profiles are used to define sets of configuration options and settings that can be applied to containers. A profile is essentially a reusable collection of configurations that dictate how a container should behave, including settings for networking, mounts, security, and resource limits.

Profiles provide a convenient way to manage container settings, as they allow you to define configurations once and then apply them to multiple containers. You can think of a profile as a template or blueprint for container configuration, enabling easy management and consistency across containers.

lxc profile show default
lxc profile create pbProfile
lxc profile delete pbProfile

lxc profile list
lxc profile copy default pb
lxc profile edit pbProfile

lxc launch <container-name> --profile bt

lxc launch <image> <instance_name> --storage <storage_pool>
lxc move <instance_name> --storage <target_pool_name>

lxc profile add <instance_name> <profile_name>
lxc profile remove <instance_name> <profile_name>

lxc profile edit <profile_name> < profile.yaml

Conclusion

LXC is a powerful and lightweight containerization technology that allows you to run multiple isolated Linux systems on a single host. It offers many advantages over traditional virtual machines, including lower overhead, better performance, and improved scalability. While it may not have the same level of widespread adoption as Docker, it is an excellent tool for system-level virtualization, development, and testing scenarios where you need full control over the environment.

As containerization continues to shape the way we deploy applications and manage infrastructure, understanding LXC’s capabilities and use cases can help you leverage Linux’s powerful container features to enhance your workflow, optimize resources, and build more efficient systems.