List of steps to create Debian based server to host contaners, with private internal subnet with NAT, static IPs and ZFS for storage Steps are for Debian 10 Buster and LXC 3.0.3.

  1. VM with at least 16G of memory and 2 disks
    • 32GB for system and swap
    • some GB for LXC containers
  2. Install debian use disk1 (/dev/sda) as system. Do not touch /dev/sdb
  3. Add ZFS support
    1. apt install zfs-dkms zfs-initramfs (it may take a while)
    2. /sbin/modporbe zfs

    3. zpool status

      should give you empty list.

  4. Create zpool 
       zpool create -o ashift=13 data /dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:1:0

    I prefer to use by-path to specify disk for VMware. It is not going to change unless VM is reconfigured, for physicals hardware by-id probably works better.

  5. Configure dataset
    create dataset for containers 
       zfs create data/lxc

    disable auto mounting

       zfs set canmount=off data
   zfs set canmount=off data/lxc

    Enable compression

       zfs set compression=lz4 data

    this property will be inherited by all child datasets Unfortunately ZFS v 0.7 shipped with Buster does not support zstd compression. zstd is vastly superior

  6. reboot
  7. check that zpool was imported during boot
       zpool status
  8. Install LXC
    1. apt install lxc
    2. create /etc/lxc/lxc.conf with following content 
      lxc.bdev.zfs.root = data/lxc
  9. Configure network
    1. create interface file /etc/network/interfaces.d/lxcbr0.conf with content 
      auto lxcbr0
iface lxcbr0 inet static
    address 10.x.y.254
    netmask 255.255.255.0
    bridge_ports none
    bridge_fd 0
    bridge_maxwait 0
    up iptables -t nat -A POSTROUTING -o ens192 -j MASQUERADE
    2. Bring it up
        ifup lxcbr0

         ip address show dev lxcbr0
   3: lxcbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
       link/ether 12:5f:c6:6f:43:89 brd ff:ff:ff:ff:ff:ff
       inet 10.x.y.254/8 brd 10.255.255.255 scope global lxcbr0
          valid_lft forever preferred_lft forever
       inet6 fe80::105f:c6ff:fe6f:4389/64 scope link
          valid_lft forever preferred_lft forever

      (TODO: disable IPV6 here)

    3. Enable forwarding. Uncomment the following in /etc/sysctl.conf: 
       # Uncomment the next line to enable packet forwarding for IPv4
 net.ipv4.ip_forward=1

      Enable forwarding runtime

       echo 1 > /proc/sys/net/ipv4/ip_forward
  10. Update default container config
    1. Remove line from /etc/lxc/default.conf 
       lxc.net.0.type = empty
    2. ADD to /etc/lxc/default.conf 
       lxc.net.0.link = lxcbr0
 lxc.net.0.flags = up
 lxc.net.0.hwaddr = 00:16:3e:xx:xx:xx
 lxc.net.0.ipv4.gateway = 10.x.y.254
 lxc.net.0.ipv4.address = 10.x.y.???/24

 lxc.start.auto = 1
 lxc.start.delay = 1
 lxc.start.order = 100

      It is not possible to create contaner with this config as lxc.net.0.ipv4.address is incorrect (but it is intentional)

  11. Finaly test the thing
    1. Create Test contaner config 
       cp /etc/lxc/default.conf /etc/lxc/Test.conf
    2. Edit /etc/lxc/Test.conf and change lxc.net.0.ipv4.address value to 10.x.y.1/24
    3. Create container 
       lxc-create -B zfs -t debian -n Test -f /etc/lxc/Test.conf
    4. Check that is it created correctly and on ZFS backend: 
       # lxc-ls -f
 NAME STATE   AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED
 Test STOPPED 1         -      -    -    false

       # zfs list
 NAME            USED  AVAIL  REFER  MOUNTPOINT
 data            270M   123G   192K  /data
 data/lxc        269M   123G   192K  /data/lxc
 data/lxc/Test   269M   123G   269M  /var/lib/lxc/Test/rootfs
    5. Run it 
       lxc-start -n Test
  12. Enjoy

References

Debian WiKi: LXC Simple Bridge local copy

Updated: