Recently I've been reading a lot about the topic of mesh VPNs (tinc, Nebula, Tailscale, ZeroTier, Netmaker, Netbird, etc) and find them pretty interesting. Is anyone here using these in some capacity at home or maybe at work?
My problem so far is that many of the options seem to be aimed at corporate use, understandably, so the developers can earn enough to keep doing it. This means the focus is on a centralized control plane, one server which knows everything about the entire network and manages firewall rules for all of it.
This is why I'm leaning towards Nebula, since I think the decentralized design just makes more sense. There is some centralization for issuing certs though. How do I go about setting up PKI? Is there some open source solution for managing certificates and automatically renewing them?
There's also the option of using vanilla WireGuard. This is my current setup, but I really like the idea of meshing, since it means I don't need to care if my devices are physically on the same network or not, the best connection will be used. Basically the layer of abstraction is a nice convenience that lets me think about hosts or services independently of the physical network topology.
I'm interested to hear your thoughts on this topic! What's your setup like and what do you use it for?
I generally steer clear of anything like it because it makes your network less secure. Only one device or server, even remote ones, needs to be compromised and suddenly the attacker can see your internal network. The only exception is that I occasionally enable wireguard for travel so that I can access my network from a single external device.
What's your use case for wanting a mesh network?
If you're firewalling you can reasonably secure things but it becomes more complicated quick because you need one in front of each end point or else it's all being filtered centrally which defeats the decentralization
Good feedback, thanks. Making sure all my devices are properly firewalled is another concern of mine, since all this NAT traversal stuff is basically a way to bypass firewall rules which are there for a reason. I think most of these solutions have firewalls/ACLs built in. But it does mean that I would rely on their firewall implementation and can't switch it out for something else.
My use case is I'm hosting a number of services on machines inside my home LAN, and also a few services on a VPS. Two family members use a few of the services hosted in the LAN and on the VPS. For things inside the LAN, for now I've given them WireGuard credentials to access them, but this gives them full access to my LAN. On the VPS they can access a http server from the public internet.
I also travel a bit, so I connect via WireGuard to encrypt my traffic and have access to my LAN services at the same time. Tailscale offers "exit node" functionality which seemed to work when I tried it.
So my main goals are:
- Encrypt my mobile traffic through a secure tunnel back home, no need to pay for or trust a VPN provider
- Ability to give users role-based access to my privately hosted services from anywhere
- Close almost all ports on all machines including the VPS, no publicly hosted services at all (maybe not even ssh)
- Simplify access to my services in general. Generally just want to be able to e.g. type the same URL and access a service, regardless of whether I'm at home or on a public network. But there are probably other solutions for this point alone.
I have the exact same use case and I use headscale on a VPS + tailscale on my devices.
I haven't figured out how to give lay-friends access though. I can make them a user in headscale but it seems cumbersome to make them use tailscale to connect. Wireguard directly seems even more cumbersome.
During my research I came across ngrok, maybe this could be useful in your situation. I also came across zrok, which seems like an open source version of the same thing based on OpenZiti. Both of them seem like ways to give public portals to your private services. So you could give your friends access that way without them needing to use a VPN.
Update: I found a guide to use traefik to tunnel into the VPN. Idea is:
- Setup headscale behind traefik on a VPS.
- Connect all my local devices to the tailnet.
- Connect to the VPS to the tailnet. So its running both headscale and tailscale.
- Traefik can now proxy to any device on the tailnet.
Step 1 however is a gigantic pain in the ass. Traefik is overkill for anything non-enterprise. It was just three lines of Caddyfile to make it work with Caddy.
http://hs.example.com { reverse_proxy * http://localhost:8282 }Step 4 is almost exactly the same
http://app.example.com { reverse_proxy * http://node:8080 }
You can also use
tailscale funnelinstead any reverse proxy but then your exposing ports not sub domains. And whatever service you're funneling to is responsible for SSL.
What's your use case? I can't really think of what I'd do with a mesh VPN at home
For PKI I think you can automate it using certbot and Let's Encrypt
Just wrote up my use case here
These certificates are custom certificates generated for Nebula clients, I don't think Let's Encrypt can issue them. In this case I have a trusted machine at home which acts as a CA and signs certificates for all other hosts on the network. The certificate is used to authenticate the host, and also can include custom attributes to be used in firewall rules. So the problem I'd need to solve is keeping track of certificate expiry and renewing the certificates, or issuing new certificates when I add new attributes to a set of hosts.
Their docs don't mention anything unique for their pki certs so it seems like Let's Encrypt should work. They also mention rotating certs in this guide so you can definitely automate some part of that
Ah, yeah I recognize the cert rotation page. That docs page doesn't say it, but they do use a custom certificate, described a little bit here:
Nebula have implemented their own certificate structure. It’s similar to an x509 “TLS Certificate” (like you’d use to access an HTTPS website, or to establish an OpenVPN connection), but has a few custom fields.
Show
I think Let's Encrypt issues certs for validating that you own a (public) domain name, but for my use, these certs aren't associated to a domain name, just a machine not accessible to the public internet. I'll do some research to see if I can self host something that would allow other hosts to request a renewed cert automatically.
Ahh that's a bummer, not sure why they wouldn't put that in their doc page
That sounds cool, I will definitely do some reading. Thanks!
I use yggdrasil which, I believe, is wireguard under the hood, but the key generation and routing are all done automatically using LAN discovery or by connecting to peers on the network. I've tested out other overlay netw, rks like tinc and cjdns, but yggdasil has been the most reliable of the bunch. Its nice because the host key is used to derive an ipv6 address that can be physically relocated without doing any manual route changes.
I used to run a bunch of services over yggdrasil, but I had some economic struggles for a while and I had to delete most of it. I'll bring it back at some point though, this time with even more yggdrasil.
Cool! I was really intrigued by yggdrasil, do you use it for a private network only, or do you connect with the public peers? And do you recall any connectivity issues related to NAT or firewalls (if mobile clients are part of your network)?
Mostly private, but I've used public peers in the past to give myself more flexibility. I usually have a cheap VPS setup as a gateway. The auto routing is really powerful, I just have a couple of hosts on my LAN that peer directly with the gateway and the rest is handled by local discovery. My mobile clients will usually have the same gateway setup for roaming, but if I'm hanging out at a library I might use a public peer over TLS if my gateway's port is blocked.
