docker firewall blocking at the source port

Today I installed my brand new docker host system, started some images and as usual trying to setup iptables to restrict the ports to specific source servers and ports.

Hours later I learned that this was kinda not so easy.

  • You cannot use the INPUT chain as Docker works with the standard bridge in the FORWARD chain.
  • The DOCKER-USER chain (which is a part of the FORWARD chain) see packets after they have been redirected to the destination port, so if we have multiple containers running at port 8000 internally it get’s very complex to find the correct container, not to mention new containers starting up.

One example:

I have a docker image that runs a web-server listening inside the container on port 8000 which is mapped to the host port 80 as in this config:

ports:
– “80:8000”

This means my host is listening on port 80 and that get’s redirected to the Docker container on port 8000.

Now the DOCKER-USER chain sees only port 8000 where I could drop some other source IP.

But I want to selectively open ports and let not configured ones dropped by default, even if somebody spin’s up new containers.

The docker-documentation and some other sources disable the iptables feature of docker and manually configure the firewall, which is okay but not that what I wanted as the sources allow general inter-container communication or have a VERY complex rulesystem that I don’t want to explain the my Administrator colleagues of that system.

 

The solution to this is hidden within the iptables flow diagram:

The PREROUTING chain of the nat table is the first one that get touched and where the source port of 80 is redirected to the container on the destination port 8000.

So we tag the SYN packets we want to drop in the PREROUTING chain of the mangle table with the MARK target and later at the DOCKER-USER chain of the filter table we will drop them accordingly.

My final ruleset looks like that:

 

#!/bin/bash

#Flush and setup:
/sbin/iptables -F DOCKER-USER
/sbin/iptables -t mangle -F PREROUTING

##############
# Docker conf
###############
#packets in NAT table are only traversed with the first packet!
#MARK1: allowed packet
#MARK2: drop this in FORWARDING (DOCKER-USER)Chain
#Mark everything with 0x2, so block everything
/sbin/iptables -t mangle -A PREROUTING -i ens3 -m state –state NEW -j MARK –set-mark 2
###
# Mark the packets we want to allow with 0x1 (which are previously also mark’ed with 0x2)
###
/sbin/iptables -t mangle -A PREROUTING -i ens3 -m tcp -p tcp –dport 80 -s xxx.xxx.xxx.xxx -j MARK –set-mark 1
/sbin/iptables -t mangle -A PREROUTING -i ens3 -m tcp -p tcp –dport 443 -j MARK –set-mark 1

###
# Drop the packets marked with 0x2
###

/sbin/iptables -A DOCKER-USER -m mark –mark 2 -j DROP
/sbin/iptables -A DOCKER-USER -j RETURN

This ruleset will allow port 80 from a specific IP address and port 443 from every source IP. Everything else exposed by docker containers is not reachable for anybody – or at least they cannot open a TCP connection to it because the initial SYN packets are dropped away.

And the best is I can start as much containers as I want on port 8000 and don’t have to fiddle around with the internal docker ports.

Hope that helps somebody ;)

Advertisements

Gitlab “file changed as we read it”

Months after the OSCE and my shiny new OSCP certification, a refactor of the infrastructure and a current build-up of a “Security Operation Center”, I’m fully back at work.

My desk is fullblown with management documents and other shit I don’t want to read, instead I’m looking at my screen after I builded a Docker Infrastructure including a GitLab server:

root@7845ae79a54c:/# gitlab-rake gitlab:backup:create
Dumping database …
Dumping PostgreSQL database gitlabhq_production … [DONE]
done
Dumping repositories …
done
Dumping uploads …
tar: ./-/….. file changed as we read it
Backup failed

Hmm, great :) Not only that the gitlab server is no more than a day old, the backup won’t work. Grr.

Long Story short, after a lot of googling my problem was a GlusterFS mount on my docker host which is bound into the Docker container.

The fix is to issue i issued the following commands on my docker host:

gluster volume info
gluster volume set dockergfs cluster.consistent-metadata on

And after a host reboot (or remount just as you like), surprise surprise the backup works now!

 

If I find more time between my family, my private projects and my daily business I’ll write a review of the OSCP course if there is interest :)

Transparently sniffing and modify traffic with scapy in-path

I decided to play a little bit with scapy and tried to find something in the internet that helped me to sniff and modify traffic without ARP-Storming the whole network.

I want to connect a box to my computer eth0 and the network to eth1, so that I’m between the box and the network.

Note: This one does not defeat SSL right now, but that could be easily done with all the tools out there.

The idea is to transparently forward all traffic and only pick out specific packets for modification with scapy.

I did do this with an combination of bridged interface, iptables NFQueue and scapy.

First we gonna create a bridge between eth0 and eth1 to transparently forward all traffic:

ifconfig eth0 0.0.0.0
ifconfig eth1 0.0.0.0
brctl addbr br0
brctl addif br0 eth0 eth1
ifconfig br0 up

Second, we need to setup iptables to send the traffic we want to change to an NFQueue.
Important is that this line is executed within the startup of the scapy script, so the actual modding only happens if the script is running.
It would be bad if the script is not running and the packets don’t traverse..

The line we need to set in the scapy script is:

iptables -A FORWARD -m physdev –physdev-in eth1  -s 192.168.1.10 -j NFQUEUE

Third, our scapy program catches those packets, modify them if they are of interest for us and forwards them accordingly.
Important is to command scapy that it should also forward traffic we do not want to be modified (e.g. other SNMP calls in this example):

https://github.com/devBioS/scapy-mitm

What I actually do is to modify a response of our target system to the monitoring system.
I choose the SNMP response of an HP ProLiant power-supply check that checks if the power supplies are OK or not (or disconnected):

The “original” says that both power-supplies are degraded or in other words they have a problem:

# snmpwalk -v 1 -c public -O eq 192.168.1.10 1.3.6.1.4.1.232.6.2.9.3.1.4
iso.3.6.1.4.1.232.6.2.9.3.1.4.1.1 3
iso.3.6.1.4.1.232.6.2.9.3.1.4.1.2 3

# ./check_psu -H 192.168.1.10
PSU 1 DEGRADED, PSU 2 DEGRADED

after starting the scapy script, it will happily transform to something much better:

#./scapymod.py
Adding iptable rules :
iptables -A FORWARD -m physdev –physdev-in eth1 -s 192.168.1.10 -p udp -j NFQUEUE
[+] Running Main loop
Got a packet ! source ip : 192.168.1.10 dest: 192.168.1.60
Modding…
Old status: 2 (<ASN1_OID[‘.1.3.6.1.4.1.232.6.2.9.3.1.4.1.1’]>)
New status: 2 (OK)
Got a packet ! source ip : 192.168.1.10 dest: 192.168.1.60
Modding…
Old status: 2 (<ASN1_OID[‘.1.3.6.1.4.1.232.6.2.9.3.1.4.1.2’]>)
New status: 2 (OK)

and in the console:

# snmpwalk -v 1 -c public -O eq 192.168.1.10 1.3.6.1.4.1.232.6.2.9.3.1.4
iso.3.6.1.4.1.232.6.2.9.3.1.4.1.1 2
iso.3.6.1.4.1.232.6.2.9.3.1.4.1.2 2

# ./check_psu -H 192.168.1.10
PSU 1 OK, PSU 2 OK

And this all by just carefully crafting and modifying packets

Have Phun :)

Weekend Project: Connect a letterbox to Jabber with XBee

As i promised this is my first XBee Project. I just needed a more or less useful application i can “test” the XBee’s in a real environment.

It is in my nature to do crazy things, so i thought it would be really cool to have a notification Jabber Message to my Phone when someone put some letters for me in my letterbox. Here it is ;)

01-08-2010 Update:
The FTDI Chip gives me A fscking LOT PAIN more to come in the next Post. DO NOT USE IT :)

This is my Setup:

  • XBee “Coordinator” API Mode connected through a FTDI USB Chip to a linux box
  • XBee “End Device” Interfaced with an Atmel ATTiny13v power by two 1.5v AA Batteries
  • Perl XBee Module API.pm from Thomas Jager
  • Jabber Perl Modules to enable sending messages
  • Siemens S685IP DECT Phone that can recieve Jabber messages

Before you read further you should note that i flashed the ZIGBEE firmware (XB24-ZB) API on my XBee’s because i don’t want to miss the mesh feature.

This Setup now runs with 2x Alkaline Batterys in the End-Device for 4 weeks now, and is still running!

Continue reading “Weekend Project: Connect a letterbox to Jabber with XBee”

Multi-Boot USB Thumb Drive

Ever thought it might be cool to only have an USB-Stick where all your individual security / pentest / recovery / hack-a-tack bootdiscs can be booted?

I thought so!

Crawling the Internet looks promising and shows two different ways how to get an bootdisc on your USB thumbdrive:

  • Booting a bootdisc as ISO stored on the drive (which is not compatible to most bootdisc’s)
  • Booting abootdisc ISO extracted to a extra Partition on the USB-Drive (which is more compatible)

Remember: both ways are possible on a single Stick, so you can have ISO’s there AND extra partitions with the contents of the original ISO.

Continue reading “Multi-Boot USB Thumb Drive”

SipToSis with Asterisk

I was little busy these days, had a lot of work to do like re-waterproofing my bathtub..

2010-02-17 Edit: Please Read the references i shown on the end of this Post to have an HowTo how to exactly install SipToSis! If i find the Time i can write a detailed Howto with Display environment variables etc, but only if i get some comments to do so :)

Nevertheless i finally managed to Get a working Skype <-> Asterisk connection via SipToSis. Hurray!
I didn’t get Skypeiax to work..

This is how i did it – with Asterisk 1.4.x branch on the same machine skype should also do – running Debian 5.0

Continue reading “SipToSis with Asterisk”