List of steps to create Debian based server to host contaners, with private internal subnet with NAT, static IPs and ZFS for storage Steps are for Debian 10 Buster and LXC 3.0.3.

  1. VM with at least 16G of memory and 2 disks
    • 32GB for system and swap
    • some GB for LXC containers
  2. Install debian use disk1 (/dev/sda) as system. Do not touch /dev/sdb
  3. Add ZFS support
    1. apt install zfs-dkms zfs-initramfs (it may take a while)
    2. /sbin/modporbe zfs
    3. zpool status

      should give you empty list.

  4. Create zpool
       zpool create -o ashift=13 data /dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:1:0

    I prefer to use by-path to specify disk for VMware. It is not going to change unless VM is reconfigured, for physicals hardware by-id probably works better.

  5. Configure dataset
    create dataset for containers
       zfs create data/lxc

    disable auto mounting

       zfs set canmount=off data
       zfs set canmount=off data/lxc

    Enable compression

       zfs set compression=lz4 data

    this property will be inherited by all child datasets Unfortunately ZFS v 0.7 shipped with Buster does not support zstd compression. zstd is vastly superior

  6. reboot
  7. check that zpool was imported during boot
       zpool status
  8. Install LXC
    1. apt install lxc
    2. create /etc/lxc/lxc.conf with following content
      lxc.bdev.zfs.root = data/lxc
  9. Configure network
    1. create interface file /etc/network/interfaces.d/lxcbr0.conf with content
      auto lxcbr0
      iface lxcbr0 inet static
          address 10.x.y.254
          bridge_ports none
          bridge_fd 0
          bridge_maxwait 0
          up iptables -t nat -A POSTROUTING -o ens192 -j MASQUERADE
    2. Bring it up
        ifup lxcbr0
         ip address show dev lxcbr0
         3: lxcbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
             link/ether 12:5f:c6:6f:43:89 brd ff:ff:ff:ff:ff:ff
             inet 10.x.y.254/8 brd scope global lxcbr0
                valid_lft forever preferred_lft forever
             inet6 fe80::105f:c6ff:fe6f:4389/64 scope link
                valid_lft forever preferred_lft forever

      (TODO: disable IPV6 here)

    3. Enable forwarding. Uncomment the following in /etc/sysctl.conf:
       # Uncomment the next line to enable packet forwarding for IPv4

      Enable forwarding runtime

       echo 1 > /proc/sys/net/ipv4/ip_forward
  10. Update default container config
    1. Remove line from /etc/lxc/default.conf = empty
    2. ADD to /etc/lxc/default.conf = lxcbr0 = up = 00:16:3e:xx:xx:xx = 10.x.y.254 = 10.x.y.???/24
     = 1
       lxc.start.delay = 1
       lxc.start.order = 100

      It is not possible to create contaner with this config as is incorrect (but it is intentional)

  11. Finaly test the thing
    1. Create Test contaner config
       cp /etc/lxc/default.conf /etc/lxc/Test.conf
    2. Edit /etc/lxc/Test.conf and change value to 10.x.y.1/24
    3. Create container
       lxc-create -B zfs -t debian -n Test -f /etc/lxc/Test.conf
    4. Check that is it created correctly and on ZFS backend:
       # lxc-ls -f
       Test STOPPED 1         -      -    -    false
       # zfs list
       data            270M   123G   192K  /data
       data/lxc        269M   123G   192K  /data/lxc
       data/lxc/Test   269M   123G   269M  /var/lib/lxc/Test/rootfs
    5. Run it
       lxc-start -n Test
  12. Enjoy


Debian WiKi: LXC Simple Bridge local copy