A VPN Out of Parts: PPP, Tap Devices and Netcat

A VPN is not a product. It is two jobs bolted together, and your machine already has a program for each of them.

The first job is making a virtual link — an interface that looks like a network card, takes IP packets and hands them elsewhere. The second is carrying that link’s bytes end to end. Buy a VPN appliance and you buy both jobs with a licence stapled on; do it yourself and each is one program already installed. There are two ways to do the first job on Linux, and this post builds both: pppd, which brings a whole link protocol — negotiated addresses, a liveness check, framing — and a tap device, which brings nothing and is a hole in the kernel you push frames into. Different trades, not competitors, and the second half sets them side by side.

That protocol is PPP, and the reason this works is written into its first paragraph: PPP is a link protocol for point-to-point links¹, and it does not specify what the link is made of. A modem and a phone line were the original answer. They were never the only one. PPTP put PPP inside GRE². L2TP put it inside UDP³. PPPoE put it inside Ethernet frames. Every one of those is the same protocol with a different courier, and none of them needed PPP changing to do it.

So the question is not whether you can run PPP over a TCP or UDP socket, but which, and what it costs. The short answer, working shown: use UDP. Running a stream protocol inside a reliable stream is the one mistake here that looks fine on your desk and falls apart on a real link.

A VPN Is Two Jobs. You Already Have Both.

Strip the marketing off any tunnel and three things happen: something presents a virtual interface and turns packets into a byte stream, something carries that stream across a network that already works, and something encrypts it, or nothing does.

PPP does the link half properly — address negotiation, a liveness check, header compression, several network-layer protocols over one link, an authentication step if you want one — a lot of finished work sitting in /usr/sbin doing nothing. What it does not do is care about the carrier: give it a file descriptor that moves bytes both ways and it runs over it. Netcat is a program whose entire purpose is being that file descriptor.

The encryption is the part nobody hands you, and it is the part most of this post is spent earning, because netcat has no answer for it and pretending otherwise is how people build things they should not.

The Stages, And Why They Go In This Order

Everything below is built in stages, each adding one thing to the one before. That is not a teaching device. It is how you should build it, because when stage six breaks you need to know whether stage two still works, and you only know that if stage two was ever a thing you ran on its own.

The link itself is built the same way — the reasoning behind the one-second ramp further down: a tunnel comes up raw, proves it can carry a frame, and only then gets clever. A build that turns everything on at once fails as one lump, and you spend the evening guessing which layer did it.

What pppd Actually Wants Is A File Descriptor

pppd was written for serial ports, so the naive reading is that it needs one. It does not. It needs a terminal device, and it will make one for itself:

pty script — Specifies that the command script is to be used to communicate rather than a specific terminal device. Pppd will allocate itself a pseudo-tty master/slave pair and use the slave as its terminal device. The script will be run in a child process with the pseudo-tty master as its standard input and output.⁴

Read that again — the whole build is in it. pppd makes a pseudo-terminal, keeps the slave, and hands the master to a command as stdin and stdout. What that command does with the bytes is not pppd’s business. Run nc there and they go down a socket.

There is a second route, notty, which uses pppd’s own stdin and stdout instead. But it spawns a character-shunt process that every byte passes through, so it “increases the latency and CPU overhead”⁴. Use pty unless you have a reason not to.

Three practical facts before the first command, all checked on the box this was written on — Fedora 44, ppp-2.5.1-7.fc44:

$ pppd --version
pppd version 2.5.1

$ ls -l /usr/bin/pppd /dev/ppp
-rwxr-xr-x. 1 root root 393784 Jan 17  2026 /usr/bin/pppd
crw-------. 1 root root 108, 0 Sep 14 08:34 /dev/ppp

$ pppd noauth nodetach pty 'true'
pppd: using the noauth option requires root privilege

It needs root. No way round that. The binary is not setuid on Fedora, /dev/ppp is mode 0600 root-owned, and the options you need are privileged anyway. This is a thing you run as root or under a unit file, not something a user does casually. That matters later, when we get to what it means for your egress policy.

The kernel side is modular. ppp_generic does the interface, ppp_async the byte-stuffed framing a pty needs, and ppp_deflate/bsd_comp/ppp_mppe compression and encryption; they load on demand. If ppp_async is missing from a stripped container image the link comes up and carries nothing — a miserable half hour if you do not know to look.

Modem control lines do not exist on a pty. With no carrier-detect pin, pppd waits for a carrier that never comes. The fix is one word, local, which tells it to ignore the modem control lines⁴. Leave it out and nothing happens, with no error worth reading.

Netcat Is Not One Program

Before the build, the trap that eats the most time: “netcat” is at least four programs with incompatible flags, and which one you get depends on your distribution. On this machine /usr/bin/nc is a symlink to /usr/bin/ncat, Nmap’s rewrite. So a command copied off a fifteen-year-old wiki page fails for reasons that have nothing to do with PPP.

As such, check which one you have before you blame pppd:

readlink -f "$(command -v nc)"
nc --version 2>&1 | head -1 || nc -h 2>&1 | head -1

I will use ncat explicitly below. It is the one with TLS built in, which matters later, and being explicit means the commands do not silently mean something else on your box.

Stage One: The TCP Build, Because That Is What Everyone Tries First

One end listens, one end connects. Addresses here are from the documentation range, so paste them straight into a lab and nothing routable is harmed. The port is 443 throughout, and that is deliberate: outbound 443 is open on almost every network by default. Wrap the carrier in TLS a few stages down and the traffic on it is indistinguishable from any HTTPS session. Binding a listener there needs root, which the far end — the attacker’s own box, or a relay — has. That is the whole egress argument in a port number, and I will come back to it.

On the listening end:

pppd nodetach noauth local passive \
     nodefaultroute noipdefault \
     192.0.2.1:192.0.2.2 \
     lcp-echo-interval 10 lcp-echo-failure 3 \
     mtu 1400 mru 1400 \
     pty 'ncat --listen --keep-open 443'

On the connecting end:

pppd nodetach noauth local \
     nodefaultroute noipdefault \
     192.0.2.2:192.0.2.1 \
     lcp-echo-interval 10 lcp-echo-failure 3 \
     mtu 1400 mru 1400 \
     pty 'ncat 198.51.100.10 443'

Every word there is doing a job, and one of them, noauth, is doing quiet damage. The tunnel has no access control, a section of its own later.

Bring both ends up and you get a ppp0 on each, a point-to-point route to the far address, and an interface you can ping, route through and tcpdump. That is a VPN — no package installed, no daemon configured, no key exchanged, which is the problem, and I will come back to it.

What It Negotiated, And How To Watch It

Add debug and pppd logs the control protocol exchange, which is worth reading once even if you never read it again. The link comes up in stages, and each stage can fail on its own.

The two useful debugging tools are already installed and nobody uses them:

# log every frame in both directions to a file
pppd ... debug record /tmp/ppp-trace

# then read it back in a human-readable form
pppdump -h /tmp/ppp-trace | less

record writes a timestamped capture of every byte, pppdump turns it into something readable⁴, and with tcpdump -ni ppp0 you see both sides — the framing underneath and the packets on top.

The Frame On The Wire, And Why 0x7E Is Everywhere

PPP over a serial line — and a pty is one as far as pppd is concerned — uses async HDLC framing, and understanding it is the difference between tuning this and guessing. Every frame begins and ends with the same byte: “Each frame begins and ends with a Flag Sequence, which is the binary sequence 01111110 (hexadecimal 0x7e)”⁵. If 0x7e marks a boundary it cannot appear inside a frame, so it is escaped. So is the escape byte 0x7d, and anything else either end asked for.

The escaping rule is exact: “Each Flag Sequence, Control Escape octet, and any octet which is flagged in the sending Async-Control-Character-Map (ACCM), is replaced by a two octet sequence consisting of the Control Escape octet followed by the original octet exclusive-or’d with hexadecimal 0x20”⁵.

That last part is the tuning knob. The ACCM is 32 bits, one per control character, and a 1 means “escape this”. Ask for all of them and, on encrypted payloads where the bytes are effectively random, roughly one byte in eight is under 0x20 — about 12% overhead for nothing.

Modern pppd already does the right thing here. Read the man page rather than the folklore:

If no asyncmap option is given, the default is zero, so pppd will ask the peer not to escape any control characters.⁴

So asyncmap 0 is the default, not a trick, and a socket is a clean 8-bit path needing no escaping beyond the two mandatory bytes. Leave it alone; set bits only when something in the middle genuinely eats control characters — a terminal server, a serial concentrator, a bad console proxy. The frame check sequence at the end is a 16-bit CRC, and it is the thing that makes the UDP build work — which is next.

TCP Over TCP Is The Wrong Carrier

The build above works. On a lab bench, over loopback or a quiet LAN, it works beautifully, which is exactly why people ship it.

Then it meets a real path with real loss. It falls over in a way that looks like everything except what it is.

The problem is two independent retransmission timers stacked on each other, the outer one hiding the loss from the inner. Olaf Titz wrote the definitive explanation, opening on precisely this build:

A frequently occurring idea for IP tunneling applications is to run a protocol like PPP, which encapsulates IP packets in a format suited for a stream transport (like a modem line), over a TCP-based connection. […] Unfortunately, it doesn’t work well. Long delays and frequent connection aborts are to be expected.⁶

Here is the shape. Both TCPs set a retransmission timer from their round-trip time. When the carrier loses a segment it retransmits, so the inner connection’s data arrives late. And the inner TCP, which cannot see the carrier, reads late as congestion and retransmits too. Now the carrier has the original and a duplicate to deliver over a link already losing packets. The inner timer doubles, the outer queue grows, and throughput collapses long before the link does. Nothing in the logs says why.

This is not a subtle efficiency point. It is the difference between a tunnel that degrades gracefully and one that stops passing traffic at maybe 2% loss while ping across the same path still looks fine — the reason to use UDP, and to keep TCP in reserve only for a path that will pass nothing else.

So use UDP — as the default, not a preference.

Stage Two: The UDP Build, Which Is The Foundation

Same pppd, different carrier. The only change is in the pty command.

Listening end:

pppd nodetach noauth local passive \
     nodefaultroute noipdefault \
     192.0.2.1:192.0.2.2 \
     lcp-echo-interval 10 lcp-echo-failure 3 \
     mtu 1400 mru 1400 \
     pty 'ncat --udp --listen 198.51.100.10 443'

Connecting end:

pppd nodetach noauth local \
     nodefaultroute noipdefault \
     192.0.2.2:192.0.2.1 \
     lcp-echo-interval 10 lcp-echo-failure 3 \
     mtu 1400 mru 1400 \
     pty 'ncat --udp 198.51.100.10 443'

Two things about a UDP listener that will catch you out, and neither is a PPP problem.

The listener cannot answer until it is spoken to. A UDP socket has no connection to accept, so netcat cannot know where to send replies until a datagram arrives. It locks onto the first source address and port it hears from and talks to that. This means the connecting end must send first — which pppd does by itself, because the non-passive end starts firing LCP configure-requests immediately. It also means that if the client’s source port changes, the carrier is silently talking to the wrong place. Behind NAT with a short UDP timeout, that is a tunnel that dies every few minutes for no visible reason.

--keep-open does not do what you want on UDP. Ncat’s -k keeps a TCP listener accepting after a peer goes away. On UDP there is nothing to accept, so recovery means restarting the carrier — which is what persist and holdoff are for, further down.

Both are arguments for socat, which handles UDP peers more carefully, and for a supervisor rather than trusting it to stay up.

Why PPP Survives A Lost Datagram

The obvious objection to a UDP carrier is that PPP expects a byte stream and UDP is not one: datagrams arrive whole or not at all, and can arrive out of order. It works anyway, because of two bytes the framing has carried all along.

PPP over async HDLC was designed for a line that corrupts bytes. Every frame carries a 16-bit frame check sequence; a frame that fails it is discarded, and the next flag byte resynchronises the receiver. Reordering is rarer than loss and produces the same outcome: a bad FCS, a discarded frame, a resync.

So a lost datagram costs one PPP frame — one IP packet, the normal condition of every network ever built. The packet’s owner retransmits, and congestion control sees a real loss and responds correctly. That is the whole argument for UDP: it lets the traffic inside the tunnel find out the truth about the path.

Do not let frames get big enough to need fragmenting by the carrier, though. One lost fragment then kills the whole datagram and your effective loss rate multiplies. Keep the PPP MTU well under the path MTU.

Addresses, Routes, And Doing IPv6 Properly

ppp0 is a point-to-point interface — no subnet, no ARP — so local:remote is the whole of the addressing and you route across it explicitly. For a single host reaching one network behind the far end:

# on the client, after the link is up
ip route add 203.0.113.0/24 via 192.0.2.1 dev ppp0

For the far end to forward on behalf of the client, the usual two steps:

sysctl -w net.ipv4.ip_forward=1
nft add rule inet nat postrouting oifname "eth0" ip saddr 192.0.2.2/32 masquerade

proxyarp will make the server answer ARP for the client on its own segment⁴, which is neat until a second client wants the same. Then do the part most people skip. Run IPv6 over it — a point-to-point link with no NAT, no broadcast domain and no address scarcity is the easiest place in your network to do IPv6 properly:

pppd nodetach noauth local \
     +ipv6 ipv6 ::1,::2 \
     nodefaultroute noipdefault \
     192.0.2.2:192.0.2.1 \
     pty 'ncat --udp 198.51.100.10 443'

+ipv6 enables IPV6CP; ipv6 <local>,<remote> sets the two 64-bit interface identifiers⁴, the link comes up link-local, and you add a global /64 on top⁷. IPV6CP is independent of IPCP, so noip carries nothing but IPv6 — a reasonable thing to build in 2026, in one word.

Keeping It Up When The Carrier Dies Quietly

This is the failure that wastes an afternoon, so it gets its own section.

A TCP carrier that dies badly — the far end powered off, a NAT table entry expired, a middlebox that stopped forwarding — does not close. There is no FIN, no RST, nothing. Netcat sits there holding a socket that will never deliver another byte, pppd sits there holding a pty that will never see another frame, and ip link cheerfully reports ppp0 as UP. A UDP carrier has no connection state at all, so it never notices anything.

PPP has the answer built in, and it is off by default:

lcp-echo-interval 10 lcp-echo-failure 3

That sends an LCP echo every ten seconds and tears the link down after three go unanswered⁴ — thirty seconds to catch a dead carrier, on exactly the case the man page names, “no hardware modem control lines”⁴, which is every pty ever made. Then decide what happens after:

persist maxfail 0 holdoff 5

It Has No Encryption And No Authentication

Everything above is a working tunnel. It is not a secure one, and the gap is not a detail.

There is no encryption. Not weak encryption. None. Every packet you put through this is on the wire in clear, wrapped in an HDLC frame that any capture tool decodes on sight. tcpdump will show you the contents of somebody else’s tunnel as readily as your own.

There is no authentication of the peer. noauth says so. A netcat listener accepts whatever arrives on the port: the first connection or datagram from anywhere. Whoever gets there first gets a routed link into your network. PPP does have authentication (PAP in the clear, CHAP as a challenge-response⁸), and both prove who the peer is while encrypting not one byte of what follows: CHAP here gets you a tunnel that knows who it is talking to and still publishes the contents to anyone on the path.

There is an encryption option in the PPP family — MPPE⁹, the module sitting in ppp_mppe.ko. Do not reach for it: it is RC4 keyed off the MS-CHAPv2 exchange, broken in public for over a decade, and the reason PPTP is dead. Building a new tunnel on it in 2026 chooses a known-broken cipher over a working one that costs nothing.

So the honest summary so far: a routed link with no confidentiality and no access control. Fine for a lab, fine inside a link that is already encrypted, not fine anywhere else. The fix is to encrypt the carrier — the next two sections, and the reason to use ncat rather than whichever netcat your distribution shipped.

Stage Three: Wrapping The Carrier In TLS

The clean answer leaves pppd exactly as it is and replaces the carrier with one that does TLS. pppd never learns anything changed.

First the certificate. A self-signed one is enough as long as the client verifies it. An unverified TLS session is an encrypted conversation with somebody you have not identified, which stops nobody:

openssl req -x509 -newkey rsa:4096 -days 825 -nodes \
  -keyout tunnel.key -out tunnel.crt \
  -subj "/CN=tunnel.example.net" \
  -addext "subjectAltName=DNS:tunnel.example.net"

Ncat

Ncat has TLS built in. It also chains through a proxy, --proxy host:port --proxy-type http|socks4|socks5¹⁰, so the carrier can terminate at a relay rather than the tunnel’s far end. That is the point the egress section turns on. Listening end:

pppd nodetach noauth local passive \
     nodefaultroute noipdefault 192.0.2.1:192.0.2.2 \
     lcp-echo-interval 10 lcp-echo-failure 3 mtu 1400 mru 1400 \
     pty 'ncat --listen --keep-open --ssl --ssl-cert tunnel.crt --ssl-key tunnel.key 443'

Connecting end:

pppd nodetach noauth local \
     nodefaultroute noipdefault 192.0.2.2:192.0.2.1 \
     lcp-echo-interval 10 lcp-echo-failure 3 mtu 1400 mru 1400 \
     pty 'ncat --ssl --ssl-verify --ssl-trustfile tunnel.crt tunnel.example.net 443'

--ssl-verify is the word that matters. Without it, --ssl gets you encryption against a passive listener and nothing against whoever answers the port first; with it, Ncat verifies trust and domain name against the trust file¹¹. Ncat has no server-side client-cert check, though, so the server cannot identify the client. Pair it with CHAP, or use one of the next two.

Stunnel

The traditional wrapper, and the one that does mutual authentication properly:

[ppp]
accept  = 443
connect = 127.0.0.1:6001
cert    = /etc/stunnel/tunnel.crt
key     = /etc/stunnel/tunnel.key
CAfile  = /etc/stunnel/clients.crt
verify  = 2

verify = 2 requires a client certificate signed by a CA in CAfile — the access control the netcat build never had. The client runs stunnel in client mode, and pppd’s pty command connects to the local plaintext side.

Socat

socat does the whole thing in one process per end, with verification on by default:

# listening end
pppd ... pty 'socat - OPENSSL-LISTEN:443,reuseaddr,cert=tunnel.pem,cafile=clients.crt,verify=1'

# connecting end
pppd ... pty 'socat - OPENSSL:tunnel.example.net:443,cafile=tunnel.crt,verify=1'

socat is not installed on the machine this was written on, so that is from its documentation — check your own flags. It is the one I would reach for, because it is the only one of the four that also speaks DTLS.

Openssl, when there is nothing else

openssl is on every box that has TLS at all, and s_server/s_client will carry a pipe:

# listening end
pppd ... pty 'openssl s_server -quiet -accept 443 -cert tunnel.crt -key tunnel.key'

# connecting end
pppd ... pty 'openssl s_client -quiet -verify_return_error -CAfile tunnel.crt -connect tunnel.example.net:443'

-quiet suppresses the banner that would otherwise land in your PPP stream, and -verify_return_error makes a verification failure close the connection instead of warning and carrying on. These are debugging tools behaving like it, but on a box where you cannot install anything they get a link up.

Stage Four: DTLS, Because The Carrier Should Still Be UDP

Here is the awkward part, and the reason this section is separate. Every TLS option above runs over TCP, so wrapping the carrier in TLS undoes the argument for UDP and hands you the meltdown back. Encryption and the right transport should not be a trade. DTLS is TLS over datagrams, and it is what you want: it keeps the record-layer protection, drops the ordering and retransmission guarantees, and leaves lost packets lost, which is exactly what PPP’s frame check sequence is built to absorb.

Ncat cannot do it. socat and openssl can:

# listening end
pppd ... pty 'socat - OPENSSL-DTLS-LISTEN:443,cert=tunnel.pem,cafile=clients.crt,verify=1'

# connecting end
pppd ... pty 'socat - OPENSSL-DTLS-CLIENT:tunnel.example.net:443,cafile=tunnel.crt,verify=1'

And with openssl alone, where -dtls selects any DTLS version:

# listening end
pppd ... pty 'openssl s_server -quiet -dtls -accept 443 -cert tunnel.crt -key tunnel.key'

# connecting end
pppd ... pty 'openssl s_client -quiet -dtls -verify_return_error -CAfile tunnel.crt -connect tunnel.example.net:443'

Watch the frame sizing. A DTLS record cannot be fragmented the way a TLS record spreads across a TCP stream, so everything has to fit the path MTU at once.

Tuning The PPP Build: MTU, Compression, And What Actually Helps

Four knobs, all pppd options — the tap build in the next section has none of them, because it has none of the machinery they configure.

None of it makes the tunnel faster, and the reason matters because PPP gets blamed for it and should not. PPP is not the bottleneck. PPPoE carries 2 Gbit on the same daemon, because its data path never leaves the kernel — ppp_generic and pppoe do the framing and forwarding, and pppd only handles the control plane. What costs you here is the pty: every byte crosses into userspace, through netcat, into a socket and back, a round trip PPPoE never makes. That belongs to the pseudo-terminal, not to PPP.

Which is a good moment to look at the other way of doing this, where there is no pty at all.

The Other Way: A Tap Device, And No PPP At All

Everything so far has used PPP for the link half. There is a second way to make a virtual interface, and it needs no protocol at all.

The kernel’s TUN/TAP driver hands you an interface and a file descriptor tied together: write a packet into the descriptor and it appears on the interface as if off a wire; read and you get a packet the kernel wanted to send. That is the entire interface¹², and it has been in Linux since 1999.

ip tuntap add dev tap0 mode tap user damien group damien
ip link set tap0 mtu 1400 up
ip addr add 192.0.2.1/30 dev tap0

Two details in that first command are worth more than they look.

user damien makes the device persistent and unprivileged. Created this way it survives the process that uses it, and a named user can open it without being root. Root creates the device once; the thing that shovels frames does not need root at all. Hold that thought for the egress section.

mode tun is the other half of the same driver, and it is the one most people actually want. Tun carries IP packets. Tap carries Ethernet frames. The difference matters enough to get its own section below.

What you give up against PPP is everything PPP negotiates: no LCP so no agreed frame size and no liveness check, no IPCP/IPV6CP so both ends are configured by hand, no authentication, no header compression. A tap device is a hole in the kernel, and the protocol through it is whatever you put there. What you gain is no framing overhead at all, no escaping, no control channel, and an interface that can be bridged.

Tap Over UDP, Where One Datagram Is One Frame

This is the cleanest mapping in the whole post, and it falls out of the design. A tap device is a datagram device — one read() returns exactly one frame — and a UDP socket is a datagram socket, one sendto() per datagram. So a frame goes into a datagram, arrives as a frame, and there is nothing to delimit, buffer or resynchronise. Lose a datagram and you have lost one frame, which is what a dropped packet looks like anyway.

With socat, both ends are one command each:

# listening end
socat TUN:192.0.2.1/30,tun-type=tap,tun-name=tap0,iff-up UDP-LISTEN:443

# connecting end
socat TUN:192.0.2.2/30,tun-type=tap,tun-name=tap0,iff-up UDP:198.51.100.10:443

socat is not installed on the machine this was written on, so those two are from its documentation rather than a run here — check your own build’s address names before trusting them. It is worth having: it is the only tool in this post that does tun, tap, TLS and DTLS in one process.

Where you cannot install anything, the whole job is about fifty lines with no dependencies beyond the standard library. The heart of it, IFF_NO_PI turning off the four-byte header the driver would otherwise prepend:

TUNSETIFF, IFF_TAP, IFF_NO_PI = 0x400454CA, 0x0002, 0x1000   # IFF_TUN is 0x0001

def open_tap(name):
    fd = os.open("/dev/net/tun", os.O_RDWR)
    fcntl.ioctl(fd, TUNSETIFF, struct.pack("16sH", name.encode(), IFF_TAP | IFF_NO_PI))
    return fd

# ... learn the peer from the first datagram (UDP has no accept), then shovel:
while True:
    ready, _, _ = select.select([tap, sock], [], [])
    if tap in ready and peer:
        sock.sendto(os.read(tap, MTU), peer)     # one read = one frame = one datagram
    if sock in ready:
        frame, src = sock.recvfrom(MTU)
        peer = src                               # last speaker wins — see the note below
        if frame:
            os.write(tap, frame)

The full file — argument parsing, IPv6 via getaddrinfo, the zero-length datagram that tells a UDP listener where to answer, and the compression the next section adds — is in the bundle:

peer = src on every datagram is the part to read twice. Whoever last sent a frame becomes the peer. Convenient behind NAT whose source port keeps moving, and an open door on an untrusted network, where anyone who can send one datagram to the port takes over the tunnel. Fine inside a DTLS session, which is where this is going; not fine on its own. The socket half of the script was exercised here over IPv6 loopback: the opener arrives, the listener learns the peer, a frame crosses and the reply comes back; the tap half needs root, the one part I could not run.

Over TCP You Have To Invent The Framing Yourself

Now swap the carrier for TCP and watch a whole problem appear that PPP quietly solved in 1994.

TCP is a byte stream with no record boundaries and no promise about how bytes are grouped on arrival: two frames written back to back can arrive in one read, one frame in three. The receiver holds a pile of bytes with no idea where one frame ends, and a tap device accepts only whole frames.

So over TCP you write your own framing. Two bytes of big-endian length in front of every frame is the usual answer:

# sending
sock.sendall(struct.pack("!H", len(frame)) + frame)

# receiving
def recv_exactly(sock, n):
    buf = b""
    while len(buf) < n:
        chunk = sock.recv(n - len(buf))
        if not chunk:
            raise ConnectionError("carrier closed")
        buf += chunk
    return buf

length = struct.unpack("!H", recv_exactly(sock, 2))[0]
frame  = recv_exactly(sock, length)

That works, and it is strictly worse than the UDP version. It is the TCP-over-TCP problem again; it adds two bytes and a reassembly loop per frame; and it has no way back from an error, because a length-prefixed stream that loses its place reads every subsequent length out of the middle of a frame. HDLC resynchronises on the next 0x7E; this cannot, short of reinventing HDLC worse. Third argument for UDP, and the strongest: over a datagram carrier there is no framing problem, because the carrier already has the only feature you needed.

Tun Or Tap: Layer 3 Unless You Genuinely Need Layer 2

The same driver gives you two devices, and people pick the wrong one constantly because a tutorial said tap.

Tun is a routed link — like the PPP interface from the first half: two addresses, a route, packets in and out. Tap is a virtual Ethernet cable, so every broadcast, ARP query and bit of multicast noise on the segment now crosses your tunnel and burns bandwidth.

Change one line in the script to switch:

IFF_TUN = 0x0001            # instead of IFF_TAP

Use tap when you genuinely need layer 2. There are real reasons: a protocol that is not IP, a cluster heartbeat that expects to see broadcasts, a DHCP server that must reach clients across the tunnel, or bridging two segments into one.

That last one is the common case and the one to be careful with:

ip link add br0 type bridge
ip link set tap0 master br0
ip link set eth1 master br0
ip link set br0 up

Now the remote segment is part of your local one — its broadcasts, its spanning tree, its MAC churn and, if someone was careless, its DHCP server. Bridging two sites that both run 192.168.1.0/24 is a bad afternoon; one that loops back to the same segment is a bad week. Default to tun, reach for tap when you can name the layer 2 thing you need, and bridge only after checking what is broadcasting on both sides.

The Same Wrappers, One Process Per End

The tap build has the same hole as the PPP one: the carrier is in clear and the listener accepts whoever gets there first. The fix is the same, and with socat it collapses into one command per end, because it makes the device and terminates the DTLS session in a single process:

# listening end
socat TUN:192.0.2.1/30,tun-type=tap,tun-name=tap0,iff-up \
      OPENSSL-DTLS-LISTEN:443,cert=tunnel.pem,cafile=clients.crt,verify=1

# connecting end
socat TUN:192.0.2.2/30,tun-type=tap,tun-name=tap0,iff-up \
      OPENSSL-DTLS-CLIENT:tunnel.example.net:443,cafile=tunnel.crt,verify=1

That is the shortest correct build in this post. A virtual interface, a datagram carrier, mutual certificate authentication and encryption. Two commands. No daemon, no protocol negotiation.

verify=1 is not optional — the same point as --ssl-verify, and with the peer = src behaviour above, an unidentified party is one datagram from owning your tunnel. If you are stuck with the Python script, do not bolt TLS into it: point it at a loopback port and put the wrapper in front, or use socat. A hand-rolled TLS wrapper around a hand-rolled tunnel is two chances to get the interesting part wrong.

Stage Five: Compressing The Stream With zstd

There is one more thing worth putting in the backend, and unlike the compression options on pppd it can genuinely pay: compress the frames with zstd before they go into the carrier.

The important word there is frames, plural. Compress the stream, not each packet on its own. It is the single biggest measured effect in this post.

Network traffic is repetitive in a way that only shows up across packets — the same headers, hostnames and JSON keys, over and over. A compressor that starts fresh on every 1,400-byte frame sees none of it; one that keeps its window across frames sees all of it.

On a current Fedora this needs nothing installed — Python 3.14 brought zstd into the standard library¹³; this box has 3.14.7 against zstd 1.5.7:

from compression.zstd import ZstdCompressor, ZstdDecompressor   # Python 3.14+

_c, _d = ZstdCompressor(level=1), ZstdDecompressor()

# FLUSH_BLOCK: emit everything so far, keep the window for the next frame.
out   = _c.compress(frame, mode=ZstdCompressor.FLUSH_BLOCK)
frame = _d.decompress(out)

One compressor per direction, alive for the life of the link. One FLUSH_BLOCK per frame, so a frame goes out the moment it arrives with nothing buffered, and the far end hands back one frame per block.

What the window is worth, measured

Numbers from this machine. Same 1,400-byte frames, same level 1, the only difference being whether the compressor keeps its history:

Throughput at level 1 ran 205 MB/s on text and over 1 GB/s on the repetitive traffic — the more compressible, the faster, because there is less to encode.

Log traffic goes to 7.5% of its original size, against 24.2% per packet — three-fold, same data, same level, from one flag. And level 1 is the level: level 3 bought about one percent, level 9 another while dropping throughput from 211 MB/s to 61.

The catch, and it is the same catch as everything else here

A shared window means every frame depends on the ones before it: lose one and the decompressor’s history no longer matches, and nothing after it decodes. So stream compression needs a carrier that delivers everything in order — TCP, or TLS over TCP, not UDP or DTLS. That is the one honest argument for the TCP carrier in this whole post. If what crosses your tunnel is genuinely compressible — syslog, database replication in the clear, telemetry, a chatty API — a stream-compressed TLS session moves a third of the bytes a datagram carrier would, which on a decent path can beat the retransmission penalty. Measure it on your own traffic.

Over UDP, Where A Dictionary Does The Window’s Job

Where the carrier is UDP or DTLS, and it should be by default, you cannot keep a window: each datagram stands on its own, which means FLUSH_FRAME and the weaker column above. A trained dictionary gives the compressor the cross-packet context a window would have, without any dependency between datagrams:

# capture a few thousand real frames off the link first, one per file
zstd --train frames/* -o tunnel.dict --maxdict=110000
```[^zstddict]

```python
from compression.zstd import ZstdCompressor, ZstdDecompressor, ZstdDict
d = ZstdDict(open("tunnel.dict", "rb").read())
_c = ZstdCompressor(level=1, zstd_dict=d)      # load it ONCE, not per frame
out = _c.compress(frame, mode=ZstdCompressor.FLUSH_FRAME)

On the repetitive traffic that took per-packet compression from 14.8% to 5.3%, recovering 96% of the gap to full stream compression while staying loss-tolerant; on log lines 55% of the gap, on general text 44%.

Three rules come with it. Both ends must load the same dictionary or nothing decodes — verified here, a dictionary-compressed frame raises ZstdError without it. Train on a capture of the real traffic, because a dictionary is a prior and a wrong one costs you: the text dictionary made different text slightly worse. And load it once into a long-lived compressor; passing it per call measured 5 MB/s, which is not a typo.

The header byte, and the one-second ramp

Two small things that stop this being fragile.

Every datagram carries a one-byte header, and it names the mode rather than just saying “compressed”: 0x00 the frame as it is, 0x01 a self-contained zstd frame, 0x02 a block from a continuing stream. A receiver can then decode whatever the far end chose without being configured to match it, which is worth the byte on its own.

In frame mode the sender only sends the compressed form when it is actually smaller, because compression is not always a win: random bytes came out at 100.7%, and a 64-byte TCP ACK compresses to 73 — the ten-byte zstd header on a packet with nothing to squeeze. Packet counts on a real link are dominated by small packets, so without that check you would inflate the majority of your traffic to shrink the minority. In stream mode it always sends the compressed form, because skipping one would put the two windows out of step.

And the link starts raw: for the first second every frame goes out uncompressed, whatever the settings say, and each direction ramps on its own:

RAMP = 1.0     # seconds of raw frames before compression starts

def pack(self, frame):
    if self.c is None or time.monotonic() - self.started < RAMP:
        return RAW + frame
    out = self.c.compress(frame, mode=ZstdCompressor.FLUSH_BLOCK)
    return ZSTD + out if len(out) < len(frame) else RAW + frame

That is the staging idea from the top of the post applied to a single link. The tunnel comes up on the simplest path it has, proves it can carry a frame, and only then starts doing anything clever. When it breaks, you know which second it broke in.

Stage Six: Compressed And Encrypted, And What The Order Costs

Order matters, and only one works: compress, then encrypt. Ciphertext does not compress, as the 100.7% row shows — which is also why TLS 1.3 removed its own compression, leaving your layer as the only place to do it. That order has a known problem, the same one I raised against deflate: compressing before encrypting leaks plaintext through ciphertext length, and where an attacker can inject chosen data alongside a secret and watch the sizes, that leak has been a working attack more than once — CRIME and BREACH against TLS¹⁴, and VORACLE against exactly this shape¹⁵.

That is not a reason never to compress. It is a reason to know which case you are in:

• A link carrying your own traffic between two boxes you own — replication, backups, logs, telemetry — has no attacker-chosen plaintext travelling with secrets. Compress it, and compress the stream.
• A link carrying arbitrary user browsing, where somebody else’s web content and your credentials go together, is the case VORACLE was written about. Leave it off.

The reason I am comfortable putting this in the tap build and not the PPP one is not principle: here you choose a modern algorithm deliberately, for traffic you have looked at, while pppd’s deflate compresses everything by default with a 1996 one whether the case suits it or not.

What The Tap Build Gains, And What It Gives Up

Put the two halves side by side, because they are not competing. They are different trades.

PPP gives you a negotiated, self-monitoring link and charges a protocol for it. A tap device gives you a raw hole for nothing, and you supply the missing parts or do without them. For a tunnel left running, the missing liveness check is the one that bites: pppd notices a dead carrier in thirty seconds and rebuilds it, the tap build notices nothing because nothing in it is watching. Add a keepalive, run it under something that restarts it, or use the build that already has one.

When This Is The Right Tool, And When It Is Not

It is never really the right tool, and I am not going to pretend otherwise. Everything above works, and none of it is what you should run in production. The honest version of a how-to includes the part where you put the tool down, and this is that part.

What it is genuinely good for is showing you how a thing works. A VPN pulled apart into its parts, and, in the next section, how egress actually behaves once somebody is inside your network with root. Those are the reasons to have read this. The narrow cases below are real, but they are not why the post exists.

Reach for the PPP build when:

• The carrier is not IP at all — a serial console, a USB gadget, a radio link, a named pipe, an SSH channel. pppd does not care what the bytes travel on, and a tap device cannot help you here.
• You are rescuing something: a machine with a serial console, no network, and a job that needs finishing tonight. PPP over that console is a routed link, installed at both ends with nothing to copy across.
• You want the link to look after itself. LCP echo, address negotiation and restart are free; writing them yourself is how the tap build becomes a small unreliable product.

Reach for the tap build when:

• You need layer 2 — a non-IP protocol, a cluster heartbeat that wants broadcasts, DHCP across the tunnel, or two segments that must be one.
• You want the fewest moving parts: over UDP with DTLS it is two commands, no daemon, no negotiation, nothing to escape.
• Root is scarce. Create the device once with user, and the process moving frames never needs privilege again.

Both are the right tool when you are learning. Every layer is visible and separately swappable, and there is no better way to understand what a VPN product does than to build one out of the parts and watch each one come up.

Neither is the right tool when you want a VPN. For that, use WireGuard. It is in the kernel, a fraction of the code, it does the crypto properly with nothing to negotiate and nothing to get wrong, and it is a datagram protocol by design. ssh -w gives you a tun device over an existing SSH session in one command, and OpenVPN is the mature, audited version of the DTLS-over-tap shape above. All three are better at this than anything built here.

Build these because you want to know what is inside the thing you buy. Not because it was clever.

What This Really Shows: Egress From The Attacker’s Side

This is the reason to read a build post you were just told not to use. Turn it around and look from inside your own network, as the person who has just landed there with root. Every real breach ends up there, by a stolen key, a container escape, an unpatched service, an insider. The question that then decides how bad the day gets is not “what can they run”, because they can run anything. It is “what can leave, and had you decided that before they arrived.”

If outbound is open by default, the answer is everything, and there is little you can now do. Nothing here was exotic. pppd, ncat, socat and ip are signed distribution packages already on the box; the tap shim is fifty lines of standard library. One allowed outbound port — and it is 443, the one every network opens by default — and there is a routed link from your network to somebody else’s, encrypted, authenticated, surviving restarts, carrying IPv4 and IPv6, and indistinguishable at your border from any HTTPS session your users make ten thousand times a day. Make it tap and bridge it and what left the building is not a route. It is the segment.

There is nothing for a scanner to catch: no malware signature because there is no malware, no odd protocol because it is a normal TLS handshake to 443, no unusual binary because your own package manager installed every one. The proxy logs a connection and a byte count, and both look like work.

And the destination is not even fixed. A socket does not have to terminate where the packets end up, because the carrier can be pointed through a proxy — a relay that is nothing more than two connections and a pipe, which the next section builds in a line of shell. ncat takes --proxy with --proxy-type http, socks4 or socks5, so the TLS session your border sees ends at whatever the attacker told it to connect through — an internal jump host, a permitted SaaS endpoint that happens to forward CONNECT, a cloud relay — and the tunnel rides on from there to somewhere you never see. HTTP CONNECT and SOCKS both do this by design, because that is what a proxy is for. So an allow-list entry for a destination you trust is only ever trust in the destination and in everything it will forward to, which you do not control and cannot enumerate. The endpoint on your firewall log is the proxy. It was never the far end.

So the uncomfortable truth: once someone is inside with root and outbound is permissive, the tunnel is not the thing you get to prevent. The parts are installed, the exit is open, and it does not even lead where it appears to. Your one chance to make this hard was before the attacker arrived, at the border, by deciding what may leave.

That is default-deny egress, and it is the whole lesson. Outbound blocked by default; a short, named allow-list where a person justified each destination and port; everything else refused, logged and alerted. Not because it stops a determined attacker cold — a permitted destination is a permitted tunnel — but because the alternative is having no decision to enforce at all. An egress policy written as a list of permitted ports is a policy about port numbers. It was never a policy about what leaves, and once someone has root, port numbers are all it protects.

This is the same finding as the ping post, which builds the tunnel out of ICMP echo, and the protocol helpers post, where your firewall opens the holes itself. Three ways in, one conclusion: the control you thought you had was over protocols, and not one of these protocols is what it says. The border, decided in advance and default-deny, is the only control that was ever real.

A Proxy Is Two Connections And A Pipe

It is worth seeing how little a relay is, because it explains why you cannot reason about the far end from the near one. A proxy is not special software. It is one connection joined to another by a pipe. The oldest form uses a named pipe, a FIFO, to carry the return direction: one ncat listens, another connects onward, and the FIFO wires the reply path between them.

mkfifo backpipe
ncat -l 7000 0<backpipe | ncat farend.example.net 7100 1>backpipe

Read it as plumbing: the listener’s output runs into the second ncat and on to the far end, and the replies come back through the FIFO to the client. Two sockets, one pipe, both directions, and the client’s connection terminates here, at the relay, while the packets carry on to farend and back. I ran exactly this on loopback with a third ncat echoing at the far end, and a line sent in came back having made the whole trip.

Ncat will do the same thing in one process, execing the onward connection for each client that arrives:

ncat -l 7000 --keep-open --sh-exec 'ncat farend.example.net 7100'

Same shape, fewer parts: the listener’s socket is joined to the exec’d ncat’s by the pipe the shell sets between them. Chain three and the tunnel crosses three networks, terminating and re-originating at each, every hop’s firewall logging a tidy local connection to the next and nothing beyond.

That is the whole trick, and it is why the border log is not evidence of a destination. Each relay is the far end as far as the box before it can tell, and the real other end is however many pipes away nobody was watching.

Now put those relays on machines that are not the attacker’s.

Each owner along that chain sees only a connection from the hop before to the hop after: no origin, no destination, just middle. This is not a novel idea I am handing anyone. It is how pivot chains and C2 networks have worked for twenty years, and why a compromised host’s outbound traffic so often leads to another victim rather than the attacker. Your logs show you talked to a box in a data centre somewhere. Whose, and what it forwards to, was never in them.

The defensive weight is one line: you cannot attribute or trust a destination you did not restrict in advance. By the time the traffic is leaving, the address it leaves to tells you almost nothing, because it is a relay on someone else’s machine and the real endpoint is laundered behind it.

The Link Was Never The Product

What strikes me, having pulled this apart, is how little of it is new and how much of it is sold.

RFC 1661 is from 1994. pppd has been in every Linux distribution for thirty years, the kernel modules are eight files in one directory, and the whole of what makes a VPN a VPN — a virtual interface, a negotiated link, an encrypted carrier, a route — is four programs and a certificate. None of it is hard or secret. They documented every byte and gave it away, and an industry grew up between you and it selling the same four parts in a box with a per-seat licence and a support contract that expires. The parts did not get better. They got wrapped.

That is not an argument for running this in production. I have just told you not to. It is an argument for knowing what is in the box you buy, because the day the vendor changes the licensing, gets acquired, or ends your model, the difference between a bad quarter and a bad year is whether anyone at your place knows what the thing was made of.

Take an evening and build the tunnel out of the parts. Watch LCP negotiate, break the link and watch it come back, pull the certificate and see what stops. Then read your VPN vendor’s datasheet again, and see how much of it you recognise.

The same evening buys the other half. If a routed link out of your network is four installed programs and a certificate, the person who just got root on one of your boxes is not held back by how hard the tunnel is to build. It is not hard. They are held back only by what you decided, before they arrived, was allowed to leave. Build it once and you stop thinking of egress as something a product enforces, and start thinking of it as a decision you either made or did not.

Not much of it is magic. Most of it is 1994 with a coat of paint, and there is nowt wrong with 1994. It worked, it was documented, and it still runs.