...an imaginary Ethernet switch or a WiFi network with unlimited range. Actual traffic is carried by your local LAN or the Internet, but the network virtualization engine takes care of encrypting, authenticating, and routing it.
It's (I think anyways) an ad-hoc virtual private network, and apparently it uses its own VXLAN-like encapsulation and includes encryption. The two use cases they espouse on their website are 1) hybrid cloud connections and 2) peer to peer VPNs.
I decided to take a look into what this is, or at least try it out, because I have recently been doing some work with OpenVPN and other VPN solutions, of which there are few, and none of them make me very happy. This zerotier essentially sounds like a simple to use VPN with a "joiner" concept.
I tested this out on two Ubuntu Trusty boxes. I downloaded the deb from zerotier, installed it and started the service.
zclient1:~$ wget https://download.zerotier.com/dist/zerotier-one_1.0.5_amd64.deb --2015-10-03 14:47:15-- https://download.zerotier.com/dist/zerotier-one_1.0.5_amd64.deb Resolving download.zerotier.com (download.zerotier.com)... 220.127.116.11, 2001:19f0:6000:91c7:5400:ff:fe10:2421 Connecting to download.zerotier.com (download.zerotier.com)|18.104.22.168|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 326330 (319K) [application/octet-stream] Saving to: ‘zerotier-one_1.0.5_amd64.deb’ 100%[=====================================================================================================================>] 326,330 1.96MB/s in 0.2s 2015-10-03 14:47:15 (1.96 MB/s) - ‘zerotier-one_1.0.5_amd64.deb’ saved [326330/326330] zclient1:~$ sudo dpkg -i zerotier-one_1.0.5_amd64.deb sudo: unable to resolve host openvpn1 Selecting previously unselected package zerotier-one. (Reading database ... 51354 files and directories currently installed.) Preparing to unpack zerotier-one_1.0.5_amd64.deb ... Unpacking zerotier-one (1.0.5) ... Setting up zerotier-one (1.0.5) ... *** ZeroTier One install/update *** Getting version of existing install... NONE Extracting files... tmp/ tmp/systemd_zerotier-one.service tmp/init.d_zerotier-one var/ var/lib/ var/lib/zerotier-one/ var/lib/zerotier-one/zerotier-one var/lib/zerotier-one/ui/ var/lib/zerotier-one/ui/main.js var/lib/zerotier-one/ui/ZeroTierNetwork.jsx var/lib/zerotier-one/ui/ztui.min.js var/lib/zerotier-one/ui/index.html var/lib/zerotier-one/ui/ZeroTierNode.jsx var/lib/zerotier-one/ui/react.min.js var/lib/zerotier-one/ui/simpleajax.min.js var/lib/zerotier-one/ui/zerotier.css var/lib/zerotier-one/uninstall.sh Getting version of new install... 1.0.5 Creating symlinks... Installing zerotier-one service... Done! Installed and service configured to start at system boot. To start now or restart the service if it's already running: sudo service zerotier-one restart
Now to start it up...
zclient1# service zerotier-one start Starting ZeroTier One...
Nice, that was super easy.
Then I joined the "earth" network, a test network provided by zerotier.
zclient2# zerotier-cli join e5cd7a9e1ce71cff 200 join OK zclient2# ip ad sh SNIP! 6: zt0:
mtu 2800 qdisc pfifo_fast state UNKNOWN group default qlen 500 link/ether 02:63:6c:ab:02:48 brd ff:ff:ff:ff:ff:ff inet 22.214.171.124/7 brd 126.96.36.199 scope global zt0 valid_lft forever preferred_lft forever inet6 fd80:56c2:e21c:0:199:9363:6cb7:e08a/88 scope global valid_lft forever preferred_lft forever inet6 fe80::63:6cff:feab:248/64 scope link valid_lft forever preferred_lft forever SNIP!
After doing so I have a zt0 network interface with an IP from the 188.8.131.52/7 network.
zclient2# zerotier-cli listnetworks 200 listnetworks
200 listnetworks 8056c2e21c000001 earth.zerotier.net 02:63:6c:ab:02:48 OK PUBLIC zt0 184.108.40.206/7,fd80:56c2:e21c:0000:0199:9363:6cb7:e08a/88
Unfortunately it seems that you have to create an account, login to the zerotier web interface (ie. it's a saas service), and create private networks from there. There are a few options, and the interface is a litle unpolished but it's straight forward enough. Once you've created a private network (which is default, and can only have a max of 10 devices or you are required to pay $4 a month which is fine) you can have systems/devices join that private network. One of the choices you'll have to make is what network IP space to give it, or to manage it yourself. By default zerotier will manage who gets what IP.
zclient1# zerotier-cli join
200 join OK
And then in the interface set the device to be allowed and it'll get an IP.
zclient1# zerotier-cli listnetworks 200 listnetworks
200 listnetworks testnet fe:42:1f:5d:f2:7d OK PRIVATE zt0 10.147.17.23/24
I ran a quick iperf test against the two nodes. So first I should note that I'm running these tests on virtual machines inside an OpenStack system and the network between them is also "virtualized", ie. is based on a VXLAN overlay, which is then running on VLANS underneath. So there is a lot of encapsulation going on here. :)
Performance over the zerotier VPN:
zclient2# iperf -c 10.147.17.23 ------------------------------------------------------------ Client connecting to 10.147.17.23, TCP port 5001 TCP window size: 85.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.147.17.61 port 59802 connected with 10.147.17.23 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 509 MBytes 426 Mbits/sec
Performance over the regular network:
zclient2# iperf -c 192.168.44.35 ------------------------------------------------------------ Client connecting to 192.168.44.35, TCP port 5001 TCP window size: 45.0 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.44.2 port 39575 connected with 192.168.44.35 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.06 GBytes 911 Mbits/sec
So you can see we get almost wirespeed without zerotier and then about half that with zerotier. So there is a performance hit. And, as I mentioned, the "normal network" is based on VXLAN, so it's already encapsulated. Perhaps the performance hit comes from the encryption. That is most likely. I haven't looked into the encryption provided by zerotier at all yet.
I'm a big fan of software defined networking, whatever it actually means. I am not a big fan of the current state of virtual private networking. It seems OpenVPN is the best we have right now, and it's quite complicated and unwieldy. I think it's fairly obvious that one of the best things about SDN is going to be easier creation of secure virtual private networks, and I think they will either look something like zerotier, or even easier.
While I like how easy zerotier is, I'm not sure I like that it is essentially software as a service (saas), which means centralized. They say their system hasn't gone down, but it will. Then again they mention that existing devices will be able to communicate if their central system is down, but new devices can't be added and new networks can't be created. However, I'm a proponent of the public cloud (at least IaaS) so it'd be hypocritical of me to say that a centralized "cloud" network controller is not a good idea when I think using compute and storage in a public cloud is. That said, I'm not sure what performance issues there might be in having a cloud based controller. I would imagine that zerotier will eventually provide a controller that can be installed on premise. That'd be interesting, and if they made encryption optional then maybe performance would come up. Then again, it would be interesting to encrypt all network communication on premise too.
I quite like the idea of servers joining networks. Recently I have been installing a lot of Linux boxes, sure, using an automated process based on Cobbler, but I find the networking component to be extremely time consuming and frustrating, even just in terms of setting up templates. I'd really like to have the networks on servers setup by pulling from some kind of data center level source of truth (CMDB?) and just automatically being allowed to join specific networks based on their function. Can I hack that kind of process up, yes, but I don't want to do that work myself all the time. I want to just "happen" based on some tags pulled from the SST.