LXC on VPS

How to (relatively) securely set up a Linux Container (LXC) on a hosted Virtual Private Server (VPS).

Intro

A hosted VPS, running Debian GNU/Linux, can be rented relatively cheaply (~$5/month) with a smallish SSD storage allocation.

To maximise the return on investment (RoI), we want to run multiple services on the VPS, such as a DNS nameserver, some webservers, a mailserver, some game servers etc.

To maximise security of the services, we want to run each in a separate area, which can be relatively easily achieved using Linux Containers (LXC).

Root filesystems

To maximise containment, each container is given its own root filesystem. As we generally cannot use Logical Volume Management or btrfs facilities on the single filesystem the VPS has been allocated, we use a sparse-file for each containers root filesystem and loop mount that as the containers root.

The advantage of using a sparse-file for the file-system is that we can over-allocate the available disk space on the VPS across the containers but each will only grow to use as much as it needs.

The disadvantage is that as a container frees up space in it's own root, the underlying file is not shrunk and the freed space is hard to recover.

$ sudo dd if=/dev/zero of=/var/lib/lxc/container1.raw bs=1024k count=0 seek=4096

We use an ext4 filesystem, but without a journal, as the underlying filesystem already has a journal:

$ sudo mkfs.ext4 -O ^has_journal /var/lib/lxc/container1.raw

Create a mount entry:

$ sudo mkdir /var/lib/lxc/container1/rootfs
$ sudo cat "/var/lib/lxc/container1.raw /var/lib/lxc/container1/rootfs ext4 loop 0 2" >> /etc/fstab
$ sudo mount /var/lib/lxc/container1/rootfs

Create Container

$ sudo lxc-create -n container1 -t debian

Fix networking etc.:

$ sudo vi /var/lib/lxc/container1/rootfs/etc/network/interfaces

change dhcp to manual for iface eth0

$ sudo vi /var/lib/lxc/container1/config

comment out (or delete) line with "lxc.network.type = empty" at end of file, add:

lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.ipv4 = 192.168.1.20/24
lxc.network.ipv4.gateway = 192.168.1.1
lxc.network.hwaddr = 00:ab:cd:ef:01:23

lxc.start.auto = 1

Networking

Each container lives on a private, un-routed network within the VPS. The VPS typically has a single IPv4 address, and different TCP and UDP ports on this external IP can be port-forwarded to ports on the different containers on the private network.

Typically, the containers will want to be able to download packages from the package repository. This is accomplished using a squid instance on the VPS, which is listening for requests from the containers bridge interface(s). Containers are then configured to proxy their package requests through this squid instance, by adding a file to /etc/apt/apt.conf.d, such as 00proxy, with

Acquire::http::proxy "http://192.168.10.1:3128/";

A default route is also set on each container so that it knows how to return traffic from the external Internet that has been port-forward to it.

Set up an Ethernet Bridge:

$ sudo apt-get install bridge-utils
$ sudo vi /etc/network/interfaces

add an entry for the bridge, giving it a local IP address:

auto br0
iface br0 inet static
  pre-up brctl addbr br1
  bridge_fd 0
  address 192.168.10.1
  netmask 255.255.255.0
  up /etc/network/br0.fw

Create an executable /etc/network/br0.fw file with some iptables rules:

$ sudo vi /etc/network/br0.fw

add something like:

#!/bin/bash

EXT_IP=10.20.30.40
BR=br0

IPT=/sbin/iptables

${IPT} -t nat -F
${IPT} -t nat -X

${IPT} -t nat -A PREROUTING -p udp --dport 53 -d $EXT_IP -j DNAT --to-destination 192.168.10.10

echo 1 > /proc/sys/net/ipv4/ip_forward