Network setup

This page should document some bits of the network infrastructure of kanthaus. Please maintain so people with a bit of network knowledge can understand and maintain the network.

Overview

TRUNK network

the backbone of our network, connecting all Access Points and the central firewall, carries all the different networks via VLAN:

¹ client isolation does not work between LAN <-> LAN clients

Wifi's

802.11r: Fast Roaming 802.11w: Management Frame Protection

¹ would be cool to have it on, but eventually led to issues. let's reevaluate in a few months

Device list

Firewall / Router

central point of all subnets, routes between them

• runs OPNSense
• due to lack of ports on firewall, a VLAN switch added, bound to the firewall, also connecting to the fritz!box
• Features:
  • VPN client
  • DNS Server
  • DHCP Server

Access Points

• see Device list
• so far all Archer C5 v1.2
• centrally configured by OpenWISP: https://openwisp.im.kanthaus.online (only reachable inside the ADMIN net)
• starting point for other ethernet cables on that floor
• in K20 all powered by PoE (802.3af) from the switch in the basement

Printer (Canon C2025i)

• Printer is in its own subnet together with the kanthaus-server
• On the kanthaus-server, there is CUPS running with printer sharing and auto discovery in the networks via avahi-daemon
• In RESTRICTED and UNSECURE the printer is also reachable directly via 192.168.4.153:9100

kanthaus-server

• Most of the services are running inside a docker-compose setup
  • Repo: https://git.kanthaus.online/kanthaus/kanthaus-server-services
  • in /opt/kh-services
  • checking state: docker-compose ps
  • starting everything: docker-compose up -d
  • logs: docker-compose logs -f --tail=20
• BTRFS raid for HDD's under /data

Random notes

Building customized OpenWRT for Archer C5

ssh kanthaus-server
sudo -iu openwrt-builder
cd openwrt
make
ls -l bin/targets/ath79/generic/

included changes

• root password
• default IP 192.168.178.200 in ADMIN range
• switch: all ports in ADMIN net
• disabled DNS rebind protection
• wireless interface names
• additional packages
  • openwisp-config
  • prometheus-node-exporter-lu
  • luci-ssl (for https)
  • ebtables (for effective client isolation)
  • tcpdump (for easier debugging)

Upgrading customized OpenWRT

Documentation: https://openwrt.org/docs/guide-developer/toolchain/use-buildsystem

ssh kanthaus-server
sudo -iu openwrt-builder
cd openwrt
git pull
git checkout v23.05.2

Edit feeds.conf to change the branch for each feed to the new version, e.g. ...;openwrt-23.05

./scripts/feeds update -a
./scripts/feeds install -a
make -j4 defconfig download clean world
ls -l bin/targets/ath79/generic/

Errors during make world are to be expected, usually the set of default packages changed and dependency conflicts are happening. Investigate generated the .config file and compare with the official ones at https://downloads.openwrt.org/releases/23.05.2/targets/ath79/generic/config.buildinfo

How to update OPNsense

There is a short downtime involved during the two reboots, usually 2 minutes each.

1. Join the kanthaus-admin wifi network and log in to 192.168.178.1 (user root, password in keepass). Export the configuration to make a backup.
2. Check if updates are available (System->Firmware->Updates)
3. In case of minor updates, just use the button to perform
4. If there is a major update (2x per year), download the newest version from https://opnsense.org/download/, unzip and flash to any USB stick
5. Take a VGA monitor, a USB keyboard and the USB stick to the K20 basement big room, open the rack box and connect the devices to the small computer (HP)
6. Log in with the USB keyboard (same credentials as above) and trigger a reboot. It should now boot from the USB stick.
7. Press any key when it prompts to run the "Importer". Now it will boot up a live environment and read the configuration file. If successful, network services should run as before.
8. If all looks good, it's time to install to disk. Run opnsense-installer from the shell (either on local keyboard or via ssh), choose ZFS and let the installer do its job.
9. Remove the USB stick and reboot again.
10. Check if the system came up good.
11. Perform any last updates from the web UI.
12. Delete the configuration backup again, it can contain secrets.

Pitfalls

• We use 802.1q VLAN tagging. All switches everywhere need to have at least passive passthrough support, otherwise the Vlans disappear at that switch. I don't know of any gigabit switch that does not support this.
• Again Vlan: Managed switches normally need to have all VLANs that should be forwarded (also tagged -> tagged forward) defined in them.
• Again Vlan: Some Access Points like WDR841 v7/v8 cannot handle tagged and untagged vlan on the same port at the same time. So far, we don't have any equipment like that and likely we will never have, but just to know...
• Again Vlan: fritz boxes have their switch in managed vlan mode and don't forward any tagged vlan