IPv6 with macvtap and libvirt

From the "I wish I'd have known that earlier" files:

I'm posting this here in the hopes of saving some folk the trouble of running this down.

If you are trying to use macvtap onto an existing adapter for a libvirt guest and you're having odd problems with dropped IPv6 traffic, you'll need to add trustGuestRxFilters='yes' to the <interface> stanza in your XML.

An example:

    <interface type='direct' trustGuestRxFilters='yes'>
      <mac address='52:54:00:0d:42:ce'/>
      <source dev='eth0' mode='bridge'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/>
    </interface> 

The problem is caused by the macvtap not updating its multicast tables correctly. This is considered correct by design for security reasons. I hope with increased deployment of IPv6 this decision will be revisited.

Some of the reading that led me here:

• linux - How to configure macvtap to let it pass multicast packet correctly?
• Bug 1035253 - IPv4/IPv6 multicasts are not forwarded via macvtap/macvlan bridge to VM (between VMs)
• [SOLVED]Re: ip link macvtap interface not processing specific ipv6 multicast traffic without promiscuous mode on

Getting DHCPv6 delegation working

...or "How I learned to make friends with duct tape"

Last week I wrote about not being able to route delegated prefixes as none of the servers seemed to have functionality to update the routing table. Shortly after tweeting this, a colleague replied with a way forward. In case Twitter's memory goes a bit hazy, I'll recap here:

FAIL: Serving prefixes with DHCPv6-PD

I've been wanting to update the lab for some time so people can bring in their routers to simulate what they'll be seeing at home. This would also allow us to mock things up for ourselves without tearing apart our home networks. For now, this means DHCPv6-PD (Prefix Delegation) as that is what many ISPs are deploying. I finally got some time last weekend to start updating the lab. Sadly, I couldn't get it to work.

To delegate a prefix to downstream, a DHCPv6 server must allocate a free prefix from its pool and then update the router configuration with the new target for that prefix. In our case, the DHCPv6 server and the router are the same machine, so no crazy remote updating is needed. We're running ISC DHCPd at the moment. It turns out, there is no way to update the routing table with this software. Someone has already tried to do this! In our case the IPv6 prefix is static, so there was no need to get a dynamic prefix from upstream. However, the crux of the problem is that while ISC makes hook scripts available for the events, they don't provide the target address to create the route. The linked article was written two years ago. I can find no evidence that anything has changed in that time. If someone out there has gotten this to work with existing software that I haven't found, please let me know.

All Things Open 2016 - IPv6 enabled sponsors

An interesting exercise for a technical event is to see how many of its sponsors have IPv6 enabled. I have attended All Things Open (who itself enabled IPv6 on their site via CloudFlare) since its inception three years ago and will be doing so again in two weeks, so I decided to perform this exercise. The parameters are simple: Is there an AAAA record on the webserver for the sponsor URL provided? Rather than list the resulting table for all of the sponsors, I will only list those who are enabled.

Sponsor	Host
RedHat	Akamai
CenturyLink Business	Self
CoreOS	CloudFlare
OpenNMS	DigitalOcean
Pendo	Google
Mozilla	CloudFlare
elastic	Amazon (legacy)

Of 46 total sponsors, only 6 have IPv6 enabled on their website, or approximately 13%. It should also be noted that this does not indicate whether these companies have IPv6 support in their product, only that they've cleared the simple hurdle of enabling it for their website.

Particularly interesting here is the one site that is hosted on Amazon. Their site is hosted on a legacy load balancer, which new accounts (VPC) cannot use. I am somewhat hopeful that more Amazon hosted sites will begin to leverage the recently announced IPv6 on CloudFront in the near future.

I plan to continue to meet with folk at these sorts of events and explain the benefits of IPv6 for their projects and the future of the Internet. Hopefully next year's numbers will be much improved.

IPv6 wildcard DNS

The use of the colon as an address separator in IPv6 has caused some mild annoyances over the years. Microsoft created unique solution, essentially mapping colons to dashes and putting this in a top-level domain. They chose ipv6-literal.net and implemented this internal to their software.

In the IPv4 world, the utility of having addresses resolvable as hostnames in DNS had sufficient demand that someone set up xip.io. Unfortunately I've never seen such an equivalent for IPv6 addresses, leaving folk to use the bracket notation and hope for the best.

Retrospective: IPv6 Utility

When I first started my journey to learn IPv6, I could not get native IPv6 at home from my residential ISP. Undeterred, I went to Hurricane Electric; HE offers both an IPv6 certification program that's a great place to start to learning how IPv6 works, and free IPv6 tunnels. I still recommend their certification as a first step to learn IPv6. (If you can't get native IPv6, you should start by asking your ISP for their timeline to offer access to the entire Internet before getting a tunnel. I would like to think that the excuse "no one is asking for IPv6" is no longer used, but the only way to be sure of that is to ask for it.)

I had my IPv6 tunnel, and I made sure I knew the security stance of anything on my home network with an IPv6 address, but I didn't use it much when I wasn't at home. During this time, I've gotten more confident with my ability to secure my home network as I intend.

At the same time, I've started to use my home network as a testbed for ideas; it's my "proof of concept lab" before I try it at scale at work.

As my ideas have gotten more complicated, I realized I needed a better system for my notes. At work, we use PmWiki, so I decided to use it for my project notes.

I created a new VM for PmWiki, and started to install it. However, I couldn't download PmWiki! Eeek! A quick check showed me that pmwiki.org only had an IPv4 address, and for some reason this VM didn't get a DHCPv4 lease (behind NAT, of course). I really wanted to finish my task, so I used a host that I knew was properly dual-stacked to download, then transferred the tarball from there. I did have to use an intermediate host for its dual-stack visibility into both IPv4 and IPv6 networks, but it didn't take much longer to do that two-step download, and I quickly had PmWiki installed, configured, and running as I wanted.

In fact, it was running so well that I never made time to go back to troubleshoot why this VM host had not gotten an IPv4 address. I could use it when I was at home because I now have native IPv6 (and NAT'd IPv4) at home. I could use it at work because I have dual-stack (native IPv6 and IPv4) there. I was curious to know why it didn't get an IPv4 DHCP lease as expected, but I don't always want to troubleshoot networks when I get home ... that's what I do at work! Several rounds of patches and reboots later, many months later, I noticed that this VM had picked up a DHCP lease, so whatever was broken before wasn't badly broken. My point, though, is not that I was a lazy troubleshooter at home.

My point is that I barely noticed the lack of IPv4 on one of my hosts at home; all I needed to do everything I wanted to do was IPv6. Notably, the reverse would not have been true! Without an IPv6 address on this host at home, I would not have been able to access it while at work without doing a fair amount of port mapping on my home router. With a global IPv6 address, I was able to firewall it to talk only to my home network and to my work network. IPv6 made this easy; I did not miss IPv4 enough to bother figuring out why it didn't work when this VM was created. I would have freaked out without IPv6, and I didn't care about IPv4.

My, how far IPv6 has come! It used to be shiny, and now it's the thing that makes my network easier to use.

Mission: Possible

You've probably seen our mission statement: "Promoting the adoption of IPv6 for the preservation of an open Internet." Mission statements seem to be a dime a dozen these days, so let's look at just how important this work really is.

At the All Things Open conference last year, one of the speakers talked of "dorm room ready software". It's wonderful that this plethora of software has become available, increasing the speed of prototyping. However, the only reason this is still possible is because those dorm rooms still have global IP addresses. The computer sitting on the desk can still serve content to the world. As we run out of IPv4 addresses, more and more of us will be closed off behind NATs beyond our control. Poor college students can ill afford hosting services that still have those scarce IPv4 addresses. This creates another barrier to innovation unless we do something.