Accessing (Docker/OCI) Containers over a WireGuard VPN (pt 2)

...
Raphaël ParréePublished on

In part 1 we've set-up a VPN using WireGuard. In this follow-up, we'll be looking into various ways to connect with docker containers running on the Server.

We've mentioned in part 1 why this can be helpful. You might be running on a public cloud and need access to certain internal services such as a nexus server, kibana, grafana or internal admin applications. We also said there are a different solutions to this problem: an allow list on your reverse-proxy, using vpn-docker containers, or just use authentication. However, this two-part blog series, assumes you want to use a configured VPN connection on the host.

Our container will be an off-the-shelf instance of (nginx

In this post we'll look at different ways of making a docker container accessible over the VPN connection.

This part also gives some insight in how docker works with iptables.

So where are we? We have a VPN setup between two peers. As a reminder here's the diagram we used at the beginning of the previous entry.

Make sure both peers are running the VPN (you could check using systemctl status wg-quick@wg0)

$ sudo systemctl start wg-quick@wg0

And then, test you can ping your server from the client-peer using the WireGuard address:

client:/ $ ping 192.168.2.1

Cool, all set to follow me along and explore ways of exposing docker containers on the VPN.

Bind to your server's VPN Address

The first technique we'll look into is a simple one. Simply bind a container port to the VPN address of the server. The VPN IP address of our server is 192.168.2.1. We don't do anything more than publish the container's port 80 to 192.168.2.1 using the --publish CLI option (or use the more commonly used abbreviation: -p):

server:/ $ docker run -p --rm --name nginx 192.168.2.1:8080:80 nginx

(For those unfamiliar with all the docker CLI options. The --rm option above ensures our container is removed after we stop it)

With the container running we can now test if we can access it. Below we use cURL together with the -I/--head option to just fetch the headers. If you feel more confident You could also type the url (http://192.168.2.1:8080) in your browser.

client:/ $ curl -I http://192.168.2.1:8080
HTTP/1.1 200 OK
Server: nginx/1.17.10
…

That was easy!

What happened here is that docker added a NAT to the DOCKER chain, mapping traffic on port 8080 to your container's port 80 (the destination ip address shown below, 172.17.0.2:80, might differ from yours. See further below)

server:/ $ sudo iptables -L DOCKER -n -t nat
…
target  prot opt   source       destination
DNAT    tcp  --    0.0.0.0/0    192.168.2.1   tcp dpt:8080 to:172.17.0.2:80

The ip address 172.17.0.2 (or whatever you're seeing) is the address of your docker container. You can check the IP address of your container using the following command (if you don't have jq installed, then replace the jq command with grep IPAddress):

server:/ $ docker inspect $(docker ps -ql) \
   | jq ".[].NetworkSettings.Networks[].IPAddress"

In most cases this approach will be sufficient. Image instead of using nginx, you expose a reverse proxy such as Traefik. THis approach would then allow you to access a plethora of services behind your internal reverse proxy.

Next we look at setting up a separate network for your internal services and access the container ports directly.approach

Before you continue, assure your nginx container is stopped and removed (if you followed along, it is running in the foreground, so you can just use ctrl+c)

Using a docker network with custom iptables rules

Let's create a docker (bridge) network with subnet 192.168.3.0/24 (Notice the 3, instead of 2)

server:/ $ docker network create docker-vpn  --subnet 192.168.3.0/24

And this time we launch the nginx container using this network using the --network option:

server:/ $ docker run  --network docker-vpn -d --name nginx  nginx

For those asking what are the other options: we launched our container in the background/detached (-d) and gave it a name "nginx" (--name)

So now what? Well lets figure out the IP address of our container (if you don't have jq, then strip that part, and find the docker-vpn network).

server:/ $ docker inspect nginx | jq '.[].NetworkSettings.Networks."docker-vpn"'
{
  …
  "Gateway": "192.168.3.1",
  "IPAddress": "192.168.3.2",
  "IPPrefixLen": 24,
  …
---

We've removed some details from the output. Very likely you have the same values. This means our container has an IP address of 192.168.3.2/24.

We can actually connect to that IP address from our client. However, recall from part 1, we need to route traffic to 192.168.3.2/24 over the wg0 WireGuard network. We need to have a match for 192.168.3.2/24 in our AllowedIPs

Let's open the wg0 configuration on the client:

client:/ $ sudo vi /etc/wireguard/wg1.conf

We'll route all traffic to 192.168.3.0/24 over the tunnel. Add this IP to your AllowedIPs values:

AllowedIPs = 192.168.2.0/24, 192.168.3.0/24

The restart your interface:

client:/ $ systemctl restart wg-quick@wg0

Access container directly

Before we can access our container's port 80, we need to add an ALLOW rule to the FORWARD chain in the filter table:

server:/ $  sudo iptables -A FORWARD  -p tcp -i wg0  \
       --dst 192.168.3.0/24  --dport 80 -j ACCEPT

We'll allow all traffic to our docker network (hence the 192.168.3.0/24 ). If you only wanted to allow access to our container, you would have to replace the it with 192.168.3.2 (192.168.3.2/32 in CIDR notation)

We are now able to access nginx's port 80 directly on its ip 192.168.3.2:

client:/ $ curl -I 192.168.3.2:80
HTTP/1.1 200 OK
…

Cool! We are in fact able to access any container directory on our docker-vpn network.

But wait a minute, How would we know the IP address of a container? They can assigned by docker's DHCP. Well, you can just assign a static IP to it. Let's set it to 192.168.3.10:

server:/ $ docker run -d --network docker-vpn \
     --name nginx2 --ip 192.168.3.10    nginx
client:/ $ curl -I 192.168.3.10:80
HTTP/1.1 200 OK
…

We won't need this second nginx container anymore, so you may remove it:

server:/ $ docker rm --force nginx2

Using NAT to our container

We still have our first nginx container running. And we were able to access it directly using http://192.168.3.2:80.

If you wanted to use NAT and expose your container using a different port on the wg0 network, you would need to a nat rule within the PREROUTING chain:

server:/ $ iptables -A PREROUTING -t nat -i wg0 -p tcp \
   --dport 8080 -j DNAT --to 192.168.3.2:80

You are now able to access your nginx instance using your Server's WireGuard IP address:

client:/ $ curl -I 192.168.2.1:8080
HTTP/1.1 200 OK
…

Before you continue, delete the iptables rules you've created:

server:/ $ iptables -D PREROUTING -t nat -i wg0 -p tcp \
   --dport 8080 -j DNAT --to 192.168.3.2:80

And the allow rule:

server:/ $  sudo iptables -A FORWARD  -p tcp -i wg0  \
       --dst 192.168.3.0/24  --dport 80 -j ACCEPT

Hold on..

But wait a minute, hold on! This is the same as using the -p option on the docker run command. Let's find out:

server:/ $  docker run -p 192.168.2.1:8080:80 --network docker-vpn \
  -d  --name nginx    nginx

Obviously we can access our container (we did the same earlier on in the article):

client:/ $ curl -I 192.168.2.1:8080
HTTP/1.1 200 OK
…

You can even use the container port directly on our docker-vpn network:

client:/ $ curl -I 192.168.3.10:80
HTTP/1.1 200 OK
…

In fact the iptables rules we created are nearly identical as those created by docker. You can check these using the following commands:

  • To find the allow rule:

    • Check the FORWARD chain:

      server:/ $ sudo iptables -n -L FORWARD
      
    • notice it jumps to the DOCKER chain, let's check that one:

      server:/ $ sudo iptables -n -L DOCKER
      target     prot opt source      destination
      ACCEPT     tcp  --  0.0.0.0/0   192.168.3.2  tcp dpt:80
      

      Voilà :)

  • To find the NAT rule, we'll take a shortcut. As above, the PREROUTING rules in the nat table refer to the DOCKER chain. So let's have a look at that one directly:

    server:/ $ sudo iptables -n -t nat -L DOCKER
    target     prot opt source      destination
    DNAT       tcp  --  0.0.0.0/0   192.168.2.1  tcp dpt:8080 to:192.168.3.2:80
    

Conclusion

The easiest way to connect to your container over a vpn is to use docker's built-in mechanism for exposing ports. This way docker is responsible for adding/removing the iptables rules.

  • Use -p, when you want to access your container over your server's WireGuard IP:

    server:/ $  docker run -p 192.168.2.1:8080:80  nginx
    

    This makes your container available on http://192.168.2.1:8080

  • Use custom iptables if you only want to access your containers on a custom network directly:

    • Add an allow rule

      server:/ $  sudo iptables -A FORWARD  -p tcp -i wg0  \
             --dst 192.168.3.0/24  --dport 80 -j ACCEPT
      
    • Run your container without -p and with a custom network:

      server:/ $ docker run  --network docker-vpn nginx
      

    This makes your container available on http://192.168.3.2:80