Network Upgrades

A network overhaul was long overdue, and I've upgraded to a setup that is modular, upgradeable, and more performant than what I had previously.

The old

For the last seven years, home network has been run entirely through an ASUS AC3200. I'd wanted an upgrade for a while, but was holding off because I wanted a system that was more powerful and offered more control. When I moved into my house three years ago, I added that additional "problem" of having more area to cover. In the last month or so, the AC3200 became increasingly unstable, with various features failing to work reliably, requiring multiple restarts and workarounds. I'm not sure I can entirely fault the device, given the age, continuous use, and an increasing number of clients. I did get seven years of use out of it.

The new

I wanted more control over the network topology, as well as having VLAN support to segregate my IoT devices, the lab, and the rest of my machines. Also, despite most of my machines being hardwired, I wanted support for multiple, non-meshed access points so I could get good wifi coverage throughout the house. I had originally considered going all-in with Ubiquiti, but was ultimately dissuaded by the cost/availability of hardware. Instead, I opted to roll the dice with a C3758 appliance from AliExpress. The C3758 and 16 GB of RAM is overkill, but allows for more headroom to run other services on the device. I did swap the fans with Noctua A4x20 PWM fans for noise reasons, and will likely have to do something about the 55 degree idle temps I'm seeing, but other than that the system has been great so far.

To that, I added a set of Omada access points, switches, and a controller. I went the Omada route mainly for the flexibility of the controller (hardware or software) and the prices of the access points.

The new network backbone consists of:

• The firewall, running pfSense+
• An OC200 Omada hardware controller
• A TL-SG3428 24-port (plus 4x SFP) smart switch as the primary switch
• A TL-SG2008P 8-port (4 PoE+) switch for powering the controller and access points
• A TL-SG1024 24-port unmanaged switch (my old switch, relocated out of the rack)
• A TL-SG108 8-port unmanaged switch (for the office machines).
• 2 EAP660 HD access points

Here's a diagram of how everything is wired together.

The reason I opted for the combination of the TL-SG3428 and the TL-SG2008P instead of a 24-port PoE+ switch was price and noise considerations. The rack most things are mounted in is in my office, and the fans in the rack-mounted PoE switches TP-Link offers are way too loud in that environment. Fan swaps, so I've read, are also difficult because TP-Link glues the connectors to the motherboard and uses a non-traditional pin ordering, making it unlikely to do the swap without having to re-pin some fan connectors and likely voiding the warranty on the switches cutting through the glue. The smaller, passively cooled TG-SG2008P solved the problem of powering the controller and access points while allowing for a higher-capacity, fanless TG-SG3428 to be used at the root switch.

In the future, this should allow me to upgrade some components to 2.5G or higher without those also needing to be PoE (to power the access points). Obviously, this currently limits the access points to 1G, but, given most of the wireless devices are IoT devices, phones, tablets, and the occasional guest device, I care much more about wireless coverage and stability over >1Gbps wireless networking. The firewall has two SPF+ ports, so I have that as an option when upgrading my plan from the ISP.

As for the choice of pfSense vs. OPNsense or some other software, I went with the one that was easiest to find documentation for. At some point I may change this, but for now, pfSense works fine.

I went with the EAP660 HDs for price reasons. At the time there was a sale that made them significantly cheaper than the other models. I don't have any wifi 6E devices, so I was unwilling to spend the extra money for 6E-capable access points.

Here's an image of the subset of components mounted in the rack.

Network layout

One of the motivating factors for doing this upgrade was VLAN support to better control traffic between the various areas of the network. I've gone with the pretty traditional separation of IoT and guest devices here, with the addition of a SERVER VLAN where the machines in the homelab live.

Here's an overview of how I have the network divided up:

(V)LAN	Subnet	Purpose
LAN	192.168.10.1/24	Management, switches, access points
TRUSTED	192.168.30.1/24	Desktop machines, laptops, cell phones, etc.
SERVER	192.168.40.1/24	Servers in the lab
IOT	10.22.50.1/24	Isolated network for cameras, smart switches, etc.
GUEST	10.22.60.1/24	Isolated network for guest devices

The choice of the 10.22.x.x VLANs for IoT and guest was just to aid in visually distinguishing that traffic in logs and such. My old network was entirely on 192.168.20.1/24, hence the gap in the current layout (which allowed me to run both networks at the same time without them interfering with each other).

Because I don't have the "complete" Omada ecosystem (no Omada router), I had to make Omada aware of the VLAN tags defined by pfSense, but that was easy enough and only a minor inconvenience.

The non-racked machines in the office are connected to the TL-SG108, and all traffic from that unmanaged switch is considered to be on on the TRUSTED VLAN. This was mainly done to simplify the wiring from the switches in the rack to the four machines on/under the desk.

There is a specific firewall rule that permits traffic from the IOT and GUEST VLANs to my Jellyfin server and port, but, other than that, those two VLANs cannot communicate with the LAN, TRUSTED, or SERVER networks or with each other.

Wireless

The old all-in-one router/gateway/access point I was using could only broadcast on one frequency band per SSID, where the EAP660s (and I assume most modern access points) do not have this limitation. Not having that limitation is particularly useful when dealing with IoT devices, many of which still only operate on the 2.4 GHz band.

The current wifi networks are as follows (A Marathon reference, if you were curious):

SSID	VLAN
Vylae FTL Network	TRUSTED
Vylae FTL IOT	IOT
Vylae Subnet	GUEST

The EAP660 HDs would allow for eight SSIDs, but I only needed three. Clients on their respective networks have their traffic tagged with the associated VLAN, which is then communicated back via the trunk ports on the connected switches to the firewall.

Replacing services

The old setup utilized two cheap beelink mini PCs, one acting as a tailscale subnet router, exit node and a separate wireguard server, with the other running pihole for adblocking and local DNS overrides. With the pfSense box, I've replaced the separate pihole by running pfblocker. I've also configured the tailscale and wireguard VPNs on the firewall itself. I haven't decided what I'm going use the two beelink devices for, but being able to reclaim those resources was a nice bonus.

Unlike with the AC3200, DHCP lease/static mapping to DNS resolution actually works on the pfSense box, so I was also able to replace that pihole functionality with the built-in functionality in pfSense.

The good

• My network is significantly more performant and stable.
• I have more control of traffic flow and better insights into what clients are doing/accessing.
• I can run the adblocking and other security applications (i.e. tailscale) on the firewall box itself, freeing those previously dedicated resources in my cluster.
• I can power-cycle certain components without interrupting the entire network. Previously if any service on the router needed to reboot, the entire network went down.
• DHCP lease to DNS mappings actually work.
• No wifi deadzones in the house.
• Simpler wifi network layout, better segregation between wifi clients.

The "meh"

• Boot time one the OC200 is a lot slower than I thought. Almost four minutes. It's not a big deal, but a bit annoying.
• Changing the Nest Hub wifi configurations was a huge pain, which basically required that I reset the devices to their factory defaults because of some bug where google thinks another user had registered them.
• Elgato's Key Lights, of which I have three, all required manual SSID configurations via connecting directly to a web interface at a specific port. This was because they refused to see the IOT network via the control center app.
• I am running out of room in the 25U rack. I will likely set up a dedicated network rack somewhere once I figure out where I'm going to terminate all of my cable runs.