Today I installed my brand new docker host system, started some images and as usual trying to setup iptables to restrict the ports to specific source servers and ports.
Hours later I learned that this was kinda not so easy.
- You cannot use the INPUT chain as Docker works with the standard bridge in the FORWARD chain.
- The DOCKER-USER chain (which is a part of the FORWARD chain) see packets after they have been redirected to the destination port, so if we have multiple containers running at port 8000 internally it get’s very complex to find the correct container, not to mention new containers starting up.
I have a docker image that runs a web-server listening inside the container on port 8000 which is mapped to the host port 80 as in this config:
This means my host is listening on port 80 and that get’s redirected to the Docker container on port 8000.
Now the DOCKER-USER chain sees only port 8000 where I could drop some other source IP.
But I want to selectively open ports and let not configured ones dropped by default, even if somebody spin’s up new containers.
The docker-documentation and some other sources disable the iptables feature of docker and manually configure the firewall, which is okay but not that what I wanted as the sources allow general inter-container communication or have a VERY complex rulesystem that I don’t want to explain the my Administrator colleagues of that system.
The solution to this is hidden within the iptables flow diagram:
The PREROUTING chain of the nat table is the first one that get touched and where the source port of 80 is redirected to the container on the destination port 8000.
So we tag the SYN packets we want to drop in the PREROUTING chain of the mangle table with the MARK target and later at the DOCKER-USER chain of the filter table we will drop them accordingly.
My final ruleset looks like that:
#Flush and setup:
/sbin/iptables -F DOCKER-USER
/sbin/iptables -t mangle -F PREROUTING
# Docker conf
#packets in NAT table are only traversed with the first packet!
#MARK1: allowed packet
#MARK2: drop this in FORWARDING (DOCKER-USER)Chain
#Mark everything with 0x2, so block everything
/sbin/iptables -t mangle -A PREROUTING -i ens3 -m state –state NEW -j MARK –set-mark 2
# Mark the packets we want to allow with 0x1 (which are previously also mark’ed with 0x2)
/sbin/iptables -t mangle -A PREROUTING -i ens3 -m tcp -p tcp –dport 80 -s xxx.xxx.xxx.xxx -j MARK –set-mark 1
/sbin/iptables -t mangle -A PREROUTING -i ens3 -m tcp -p tcp –dport 443 -j MARK –set-mark 1
# Drop the packets marked with 0x2
/sbin/iptables -A DOCKER-USER -m mark –mark 2 -j DROP
/sbin/iptables -A DOCKER-USER -j RETURN
This ruleset will allow port 80 from a specific IP address and port 443 from every source IP. Everything else exposed by docker containers is not reachable for anybody – or at least they cannot open a TCP connection to it because the initial SYN packets are dropped away.
And the best is I can start as much containers as I want on port 8000 and don’t have to fiddle around with the internal docker ports.
Hope that helps somebody ;)