Make IPv6 Work For You, Not Against You [Archived]

By Adam Kennedy

Watch Communications IPv6 Case Study

Watch Communications started as a telephone company back in 1902; I worked at Omnicity, where we had built a small IPv6 network prior to being acquired by Watch Communications in 2010. When we came on board, Watch Communications said they wanted to look into doing IPv6 too. Today we have every office IPv6-enabled and push IPv6 to our hosting customers. On the Internet service side, our biggest hurdle has been trying to justify the costs in upgrades for equipment and investment in time to train networking staff and support staff on IPv6 and how to troubleshoot it with downstream customers.

Every work station can get to the Internet over IPv6

The way we started our initial IPv6 project was to make every office fully IPv6-enabled. Some offices have native connectivity to an IPv6 leg of our network, and others we provide IPv6 through a VPN tunnel back to our corporate headquarters. One hundred percent of our data infrastructure is IPv6-enabled. Every server has at least one IPv6 address as well. For office locations that don’t have native IPv6 connectivity out to the Internet, we’re pushing the IPv6 default route as well.

Taking IPv6 to hosting customers

In 2013, we started pushing our hosting customers to have IPv6 addresses for all of their websites and email hosting. Today 90% of all our hosting customers are actively using IPv6 in their communications, whether they realize it or not. We do full IPv6 connectivity outbound through email.  As soon as customers sign up for any kind of website hosting, even if they’re on a shared IP with the hosting solution, they get IPv6 as part of that automatically.

The first thing we needed to do was get our server infrastructure connected to IPv6 so we could log in and manage the servers. Once the servers could get out to the Internet on IPv6, we made sure our DNS servers were accessible and could serve AAAA records. We added IPv6 addresses to our glue records with our registrar, so from the furthest point out that we could touch to the Internet, it would be able to browse IPv6 from that point. Once that was operational, we started pushing IPv6 to a select group of customer domains. We called a few customers asking if they wanted to be part of an experiment. A few said yes, so we enabled their domain on IPv6 and added some proper DNS records. What we noticed off the bat was that any email traffic headed to big name carriers, like Office 365 or Google, automatically went through IPv6. To us, it felt like those connections were faster.  I don’t know if they were, but I feel like there is less latency that carries through the entire connection with IPv6 due to less analysis of every individual packet like in IPv4. Everything seemed to go really well in the sample group we started with. So, we decided to go all in and enable it for every hosting domain that we had, unless someone requested otherwise.

IPv6 on the Internet service side

In the last decade, Watch has added terrestrial wireless Internet connectivity to customers, website and email hosting, fiber to home and business initiatives. We haven’t been able to push IPv6 out to those customers on a large scale yet. Our routers along our edge and along our access point all support IPv6 and we can push IPv6 out without any problem. We use PPPoE which makes pushing IPv6 out to customers very simple, but it seems like every individual vendor for home routers has their own way of handling IPv6.  In many cases,  customers would need to reboot all their home devices to get a new IPv6 address, and it is bad business to force a situation to create an inconvenience for a customer.  This is not the case with IPv4, since it’s all handled through NAT. We don’t want to force our customers to buy new gear, and until their router breaks, they are probably not going to go buy a new one. I suspect the solution is that ISPs like us are going to have to work with router vendors to sell or lease a router to every customer to get IPv6 pushed out to the customer edge.

We do have IPv6 rolled out to the customer edge in some limited areas. We made phone calls to a couple tech savvy customers who have purchased a new router and asked if they wanted to be guinea pigs. For those who said yes, we enabled IPv6 on those Points of Presence on the network similar to the hosting customers. They got their IPv6 prefix delegation. Those customers picked up. Everything worked.  More tech savvy users did notice some websites gave them some trouble. They could browse there yesterday, but today with IPv6, they couldn’t. We discovered a website our customer was trying to access had enabled IPv6, but it did not handle firewalls appropriately. IPv4 requests were coming in fine, but IPv6 requests were getting mangled which caused them to disconnect. We happened to know the operator of the site, so we worked with them to solve the problem. Luckily, we had this experience with a tech savvy user, so we got through it ok, and it was a good learning experience to help walk less tech savvy users through the same issues.

Push IPv6 further into the network piece by piece

We were prepared moving in to our IPv6 deployment and knew what to expect. Some old pieces of equipment weren’t going to be able to handle static IP routing, so we upgraded them.  We began by rolling things out at our network core where BGP comes into the network. We asked, can IPv6 operate there? If yes, then we went to the next set of routers upstream from that. We worked our way down the line one step at a time. Every couple of days, we pushed IPv6 further into the network. Everything went fine.

As expected, privacy extensions gave us trouble in our office locations, especially for some operating systems. They caused connectivity problems when we did software deployments, made updates to new programs, or remotely accessed computers to offer assistance. Sometimes DNS wouldn’t keep up fast enough with the new IP that the privacy extension was handing out, so we’d try to connect, and it wouldn’t work.  The problem turned out to be an old IP that was getting ready to expire from the privacy set and wasn’t taking new connections inbound. Disabling privacy extensions across the board ended up fixing that issue.

Don’t be intimidated

IPv6 can be extremely intimidating, especially for people who are not IP infrastructure engineers. They look at an IPv4 address, and it’s something small and familiar. They look at IPv6, and all of the sudden, it’s not just numbers, it’s a hexadecimal address, and there are letters. The advice we gave our staff was to not let that intimidate you. It is really just the difference of calling someone by their first name or calling someone by their first, middle, and last name. Don’t let it scare you. Once we got into teaching folks how our IP structure is planned out and how our addressing is being done, a lot of the staff actually ended up liking IPv6 better.  From our infrastructure it was easier for them to figure out where the IP was coming from. With IPv6 we started using the 4th nibble so that it was our building ID.  It just so happened that our building IDs fit perfectly with hex addresses. Helpdesk staff can look at an IP address and easily figure out the building and device whether it’s a printer, desk phone, or work station.

IPv6 & NAT

Many people seem to be against NATing in IPv6, but I think it still has a function in IPv6 with prefix delegations. We consulted with a few other businesses who were looking at deploying IPv6. They had an IPv6 address from their carrier, but were having the same issue we were having with customers. They didn’t have a route out to the Internet. The solution we came up with for that partner was that they would implement internal IPv6 NAT. Their internal IPv6 address was a unique local prefix that was a purposeful prefix. So they generated a prefix out of that range that they used for their internal network, and then it’s NATed before it gets to the Internet. Then whatever address they get from their carrier can change every hour but internally their devices don’t know. Their firewall does the translation and then out to the Internet they go. That solved that problem for them immediately and worked fine for two years. Everyone wants to do native public IPs all the way to the work station, and if your environment will allow it, and your firewall and your carrier can handle that appropriately, great. If it can’t, don’t dismiss NAT.

Take the time to learn how IPv6 works

Organizations will want to take a look at how IPv6 fits into their environment and works for them, not against them. We didn’t want to push our deployment so fast that we didn’t spend the time to do it right. Slow down. Take the time to brainstorm how you can make it work for you.  We took a step outside of the box and decided to look at what identifiers we could use to put into subnets to help tell what building or device an IP is assigned to. Some organizations may have multiple stories in an office building and the floor number comes into play. Take some time and really think about how IPv6 is going to be best implemented for you.

Any views, positions, statements or opinions of a guest blog post are those of the author alone and do not represent those of ARIN. ARIN does not guarantee the accuracy, completeness or validity of any claims or statements, nor shall ARIN be liable for any representations, omissions or errors contained in a guest blog post.