OpenVPN bridge and VMWare ESX 3.5

July 12, 2009 at 13:53 (VMWare) (, , , , )

These days i wanted to add an OpenVPN Server in bridged mode to my Network Infrastructure.

But there was one big problem: the OpenVPN System can’t live on a special Subnet and it cannot be routed through because the Network is maintained by other People which do not want to give me another Subnet and i don’t want to set the OpenVPN Gateway as default gateway in the Network.

 
So the only solution to this was a bridged OpenVPN Server and i set it up in my VMWare ESX 3.5 the following way:

VMWare ESX & OpenVPN

I can connect to the Server from Outside and get an IP-Address from the OpenVPN Server as specified in “server-bridge …..”

 

However i can’t connect to the other system’s in the Subnet, with tcpdump i only see a lot of “who-has” requests but it seems that the ARP replies didn’t go back to the Connected VPN Client.
I tried virtually anything and after a day of research i finally got it working. Why? Simple after all :)

  • First you need to enable Promisc mode for the Virtual Network, this is a setting of the vSwitch under the “Security”  Tab
  • Second, ensure you DO NOT connect more than one Network card to the Virtual Switch providing the vSwitch your OpenVPN Server is connected. That was my main problem.
  • Third, ensure you enable ip_forwarding -> “echo 1 > /proc/sys/net/ipv4/ip_forward”
  • Forth, if you do ping tests and your VPN client is Windows, double, triple-check the firewall is off :)

 VMWare Single

After removing the other Network cards from the Virtual Switch i could reach my complete Network instantly!

Advertisements

20 Comments

  1. Tom said,

    im so glad i found this post, thx a bunch
    i was experiencing the same problem but couldnt figure it out.

    • devbios said,

      Hi Tom,
      thanks for the response, i’m happy that you find it useful!

  2. K-O said,

    Thanks for your helpful information!
    Do you think connecting more than one NIC to the Virtual Switch causes some arp related problems?
    Now I have ARP problem in that situation…

    • devbios said,

      Hi K-O,

      i think the problem is that you have in the guest a bridged interface whichself is bridged in the VMWare host to different physical NIC’s.

      So the possible situation is that the packets leave interface #1 and come in on another interface, but the system expects an other MAC-Address.

      So yes, i think it will cause ARP Problems when you do some advanced networking in the guest when you have more than one NIC’s connected to the host.

      • K-O said,

        Thanks,

        What was the teaming mode?
        I tried the “originating port ID” only, but if we choose “ip hash” it might solve the problem too.
        The “ip hash” (Link Aggregation) cares about each link every time, I think.

      • devbios said,

        Great Idea!
        I will keep up with it and give it a try the next time i get my hands on the OpenVPN Server..

  3. OpenVPN Bridging - Forum Fachinformatiker.de said,

    […] […]

  4. Jiri Horky said,

    Hi there,

    thx for sharing your experience, I faced exactky the same problem and found the solution here!

    Jiri Horky

    • devbios said,

      You’re welcome, it’s always a pleasure if I can help with my blog entries!

  5. Diego said,

    Graaaciaaaaaaassss!!!
    TRANKS YOU!!!!!
    “enable Promisc mode for the Virtual Network, this is a setting of the vSwitch under the “Security” Tab”
    i have been 4 days searching possible solutions!!!
    THANKS again.
    Diego (from Argentina)

  6. Vincent MARECHAL said,

    (Can put ESXi bridge to work, client connects but no bridge.)

    Please,

    I can’t put to work my Openvpn ESXi bridge to work.

    Could someone show me it’s ifconfig output, so I could figure what is wrong in my config ?

    Or could someone help me ?
    I’m developer public school teacher, not network specialist. But I need this bridge to make some tests.

    I installed Openvpn Access Serveur Virtual Appliance in my ESXi box.
    And I put all vswitches to promiscuous mode !
    Then my /etc/network/interfaces :
    auto lo
    iface lo inet loopback

    # The primary network interface
    allow-hotplug eth0
    iface eth0 inet static
    address 200.53.114.43
    netmask 255.255.255.240
    gateway 200.53.114.33

    #Private Network
    allow-hotplug eth1
    iface eth1 inet static
    address 172.1.50.9
    netmask 255.255.255.240
    # gateway x.x.x.x

    My ESXi Openvpn : public IP : 200.53.114.43, and LAN : 172.1.50.9.
    I put VPN mode to : Layer 2, but no Bridge Name to Join (it says it’s optional).

    I need to bridge from home to LAN 172.1.50.0 network, so I put in Advanced VPN -> Server Config Directive the line :
    ifconfig-pool 172.1.50.12 172.1.50.12 255.255.255.240

    When I connect from home, I get the ip : 172.1.50.12, and the right route print entry, but I can’t ping Openvpn ESXi bridge LAN ip (eth1) 172.1.50.9, neither ips from LAN computers.

    In my Openvpn ESXi bridge, the ifconfig give me :
    as0t0 Link encap:Ethernet HWaddr fe:ff:ff:65:1f:00
    inet6 addr: fe80::fcff:ffff:fe65:1f00/64 Scope:Link
    UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
    RX packets:1223 errors:0 dropped:0 overruns:0 frame:0
    TX packets:1138 errors:0 dropped:1 overruns:0 carrier:0
    collisions:0 txqueuelen:200
    RX bytes:118971 (116.1 KiB) TX bytes:122625 (119.7 KiB)

    asbr0 Link encap:Ethernet HWaddr 00:0c:29:ae:ba:74
    inet addr:200.53.114.43 Bcast:200.53.114.47 Mask:255.255.255.240
    inet6 addr: fe80::20c:29ff:feae:ba74/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:4861 errors:0 dropped:0 overruns:0 frame:0
    TX packets:2774 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:541219 (528.5 KiB) TX bytes:498187 (486.5 KiB)

    eth0 Link encap:Ethernet HWaddr 00:0c:29:ae:ba:74
    inet6 addr: fe80::20c:29ff:feae:ba74/64 Scope:Link
    UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
    RX packets:41472 errors:0 dropped:0 overruns:0 frame:0
    TX packets:4003 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:22860963 (21.8 MiB) TX bytes:617626 (603.1 KiB)

    eth1 Link encap:Ethernet HWaddr 00:0c:29:ae:ba:7e
    inet addr:172.1.50.9 Bcast:172.1.50.15 Mask:255.255.255.240
    inet6 addr: fe80::20c:29ff:feae:ba7e/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:17364 errors:0 dropped:0 overruns:0 frame:0
    TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:1438884 (1.3 MiB) TX bytes:468 (468.0 B)

    lo Link encap:Local Loopback
    inet addr:127.0.0.1 Mask:255.0.0.0
    inet6 addr: ::1/128 Scope:Host
    UP LOOPBACK RUNNING MTU:16436 Metric:1
    RX packets:31 errors:0 dropped:0 overruns:0 frame:0
    TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:13074 (12.7 KiB) TX bytes:13074 (12.7 KiB)

    Is it normal that asbr0 get ip 200.53.114.43 ?

    If I keep mode to : Layer 2, but but put Bridge Name asbr0, ifconfig give me :
    eth0 Link encap:Ethernet HWaddr 00:0c:29:ae:ba:74
    inet addr:200.53.114.43 Bcast:200.53.114.47 Mask:255.255.255.240
    inet6 addr: fe80::20c:29ff:feae:ba74/64 Scope:Link
    UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
    RX packets:42295 errors:0 dropped:0 overruns:0 frame:0
    TX packets:5015 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:23005868 (21.9 MiB) TX bytes:820754 (801.5 KiB)

    eth1 Link encap:Ethernet HWaddr 00:0c:29:ae:ba:7e
    inet addr:172.1.50.9 Bcast:172.1.50.15 Mask:255.255.255.240
    inet6 addr: fe80::20c:29ff:feae:ba7e/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:17631 errors:0 dropped:0 overruns:0 frame:0
    TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:1460972 (1.3 MiB) TX bytes:468 (468.0 B)

    lo Link encap:Local Loopback
    inet addr:127.0.0.1 Mask:255.0.0.0
    inet6 addr: ::1/128 Scope:Host
    UP LOOPBACK RUNNING MTU:16436 Metric:1
    RX packets:31 errors:0 dropped:0 overruns:0 frame:0
    TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:13074 (12.7 KiB) TX bytes:13074 (12.7 KiB)

    But it’s the same, from home I get 172.1.50.12 but can’t ping 172.1.50.x nodes.

    I spent some days without success. I suppose I don’t have all the knowledge to fully understand the problem.
    I’m teacher for public school and need this bridge for my students to learn some programing…

    Please if someone can help.

    Thanks a lot.

    Vincent MARECHAL

  7. devbios said,

    Hi Vincent,

    i think you have just messed up with eth0 and eth1, the bridge is getting your external IP-Address so i think the bridge is just configured to use eth0 instead of eth1 to connect the tap adapter.

    It would be helpful if you post or mail your current openvpn-server.conf so we could have a look what’s wrong there.

    Normally – if you use bridging – you do not need to use the ifconfig-pool directive as the connected client should get an IP-Address from your DHCP Server.

    What is even more important is that you say if your OpenVPN-Server is your default gateway device, hence if it is establishing the connection to the internet or if it is just a “computer” that is in your network and you have some other device that establishes your internet connection.

  8. Vincent MARECHAL said,

    Hi devbios,

    Thanks a lot for your help !
    There is no server.conf in openvpn_as, all the stuff is coded in conf.db…

    So I searched and learn about bridges, before asking more help. And I made tests…

    Yes, it’s a pity openvpn_as does this mistake…
    In the virtual appliance, it’s a debian, interfaces comes this way (with my addresses) :
    # This file describes the network interfaces available on your system
    # and how to activate them. For more information, see interfaces(5).

    # The loopback network interface
    auto lo
    iface lo inet loopback

    # The primary network interface
    allow-hotplug eth0
    iface eth0 inet static
    address 200.53.114.43
    netmask 255.255.255.240
    gateway 200.53.114.33

    #Private Network
    allow-hotplug eth1
    iface eth1 inet static
    address 172.1.50.9
    netmask 255.255.255.240
    # gateway x.x.x.x

    But in the internet page of Admin UI, they says :
    “Specify a pre-existing bridge to join. If blank, bridge with the primary network adapter on the machine. In most cases, this field may be left blank.”

    This means that the normal behavior of openvpn_as is to make the bridge (the famous asbr0 you saw).
    But if they say eth0 is the public one, then their bridge connect to the wrong one : eth0 = the primary network adapter !!!

    I exchanged eth0 and eth1 IPs to make eth1 the public one… (I can exchange then in ESXi which interface is connected to each virtual one).
    BUT… then openvpn_as consider eth1 as the primary network adapter. WHY ???

    Then no way… if eth0 as the public IP, openvpn_as take eth0 in the bridge. And if I put the public IP in eth1, openvpn_as take eth1 in the bridge.
    I’m going mad…

    I searched the whole Admin UI of openvpn_as, but no clue.
    It’s in https://200.53.114.43:943/admin
    user : openvpn
    pw : tteesstt0

    Well, there is no risk, I just use it to make tests…
    I think openvpn_as is bad scripted, because it should offed to choose the eth to bridge or take the one called “Private Network” in interfaces.

    I searched hours information about this problem but found nothing.

    Thanks a lot again for your help.

    Vincent MARECHAL

  9. Vincent MARECHAL said,

    I forgot, in https://200.53.114.43:943/admin, the VPN config panel is named “VPN Mode”.

    • devbios said,

      Hi Vincent,
      configuration looks half-way good, i need access to the the console to see the ifconfig’s, can we chat in a jabber or IRC sessions or something?

    • devbios said,

      just modified some bits in your config, now it’s seems to work :)

      Pinging 172.1.50.1 with 32 bytes of data:
      Reply from 172.1.50.1: bytes=32 time=173ms TTL=127
      Reply from 172.1.50.1: bytes=32 time=173ms TTL=127
      Reply from 172.1.50.1: bytes=32 time=173ms TTL=127

      I changed from Layer2 to Layer 3 mode and gave the Server your 172.1.50.0/24 network to route.

  10. Vincent MARECHAL said,

    Hello devbios,

    Yes ! I can connect and ping 172.1.50.0 nodes !
    Thanks you very much.

    I wanted to do it with bridged openvpn_as instead of using a NAT.
    I think you used a NAT because 172.1.50.0 nodes have another gateway and could not answer with a just routed openvpn_as.

    So with a NATed one, that’s perfectly ok !

    Just one thing, I wanted to use a bridged openvpn_as to not have NAT limitations.
    But I stayed stuck by this asbr0 which use the primary network adapter, and consider it’s the one with 200.53.114.43, no matter that I use eth0 or eth1 for it…
    I don’t know why it always consider as primary network adapter the one with 200.53.114.43 ? Then I suppose should build the bridge manualy and then give it to openvpn_as instead of letting it doing the job ?
    But at this point, I’m not at ease.

    My skype account is vincentvije (vincent at vije.net). I’m connected.
    I’d like to get the bridge working.
    For now I’m backing the NATed openvpn_as.

    I can give you root account by skype or email.
    Sorry I just connect for I was qith my two babies… and they don’t like computing…

    Thanks a lot for your time.

    Vincent

  11. Vincent MARECHAL said,

    Hi devbios,

    I tried the NATed config with my programs and labs jobs for my young students, it’s perfectly ok, so I will not need more or bridge, etc…
    And with your config, I can learn more of openvpn_as.

    Thank you so much !

    Vincent

  12. OpenVPN Access Server – An Exercise in Frustration | dfreer.org said,

    […] some creative googling, I discovered this blog post discussing the issues specific to running a layer 2 VPN on ESXi. I’m not sure as to the […]

  13. Shalon said,

    Working perfect !!!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: