Questions about Comcast Business Delegated Prefixes

Search for some of the presentations by John Brzozowski (e.g. on YouTube). He was the IPv6 architect for Comcast, and has given some talks about how Comcast implemented it.

Haha.  I'm familiar with JB and have met him before.  What's funny is that I've already been exchanging emails with him the last few days.  Haven't really corresponded with him since the IPv6 pilot days at CC residential.

According to my last email, the CPE just points routes at the requesting DHCP client when a delegation is made, as I've inferred from some of the posts I've read on here and elsewhere.

I guess the problem is, not every CPE device actually does this properly.  🙂  

I have static IPv4 with the cable modem in bridge mode. I have tried many combinations of DHCPv6 prefix lengths on the WAN side, static config/routes and prefix lengths on the WAN side, and static IPv6 and route advertisements on the LAN side. No combination works. In one or two combinations, I could get ping to work from the LAN to the modem's router address, but anything beyond the modem doesn't respond to ping or traceroute6. Rapid-commit enable/disable didn't seem to make any difference either. Enabling/disabling DHCPv6 on the WAN didn't seem to make a difference either for SLAAC/radvd/static configuration.

It's insanely frustrating that with a Tunnel Broker account and a UniFi USG (vyatta underneath), setting up a tun0, configuring static IPv6 /48 /56 and /64 networks... it works with no problem.

Every time I login to the account management site, I see a broken promise that says I have a static IPv6 /56 network that I can't use.

@MikeAce completely nailed it with his post.  That is exactly how it works.  I have it working that way with both OPNSense as well as an older cisco router. 

I wish we could either bgp peer with the cable modem to notify it of next hops for subnets within our /56 or simply type in ipv6 static routes (you can already do this with ipv4) in the modem.  Would make things much more configurable.

@MikeAce Yep.  I understand how DHCPv6 and SLAAC work.  That's not really what I'm asking though.

In order for any delegated prefix to be useful, the cable modem/router has to put a route in for the prefix it delegates, to the right gateway.  In the DHCPv6 PD RFC, the mechanism to do this is not defined, and is left for other standards or custom mechanisms to handle.

My question is how are these routes put into the cable modem's routing table?  How does Comcast implement this function.  My guess is that it simply enters a route for each prefix it delegates to the requesting router, pointing it through the link-local address of the requesting router.

AFAIK, there's no support for a routing protocol (like RIPng) to advertise the delegated prefixes to the cable modem, or anything like that.  That would provide a more flexible way set up your routing (for instance, if you for some reason had two edge routers, two could ask for prefixes for internal nets, and each could advertise their assignments, as well as alternate routes for the other routers assignment, etc).

So, getting back to my question, it appears that the acceptance of the delegated prefix automatically causes the cable modem to put a route in for the prefix in question?  Hard to check ones self, since AFAIK, the cusadmin login provides no way of looking at the IPv6 routing table.

@bitblaster2 I think bridged mode breaks certain things.  I have no idea how the IPv6 "statics" are set up with CB.  One would presume that they implement it using DHCPv6 PD from the upstream router.  E.g., the cable modem asks for a /56, assigns one /64 out of that to the inside interface, and makes the remainder available for PD from the CM.  If this is how they do it, then you should be able to configure your router behind the CM in bridged mode to do the same.  But my guess is that they just set it up w/ the CM configuration file in a semi-static manner, per-customer.

I vaguely remember reading some FAQ or article stating that IPv6 "statics" are only available through the CM in "pass-through" mode, rather than bridged mode.  "Pass-through" mode, if I've deduced it correctly, could just be called "router mode".  That is, in this mode, NAT and the FW are completely disabled, and the CM simply acts as router with a DOCSIS/cable interface on upstream side, and ethernet on the downstream side.  It just routes and passes packets like any other router.  The WAN interface of the CM is assigned an IPv4 and IPv6 address via one mechanism or another (DHCP or static downloaded config).  The downstream interface is where your /29 (or whatever) IPv4 assignment lives, and where it assigns the first /64 of your IPv6 /56 for a dual-stack LAN between your CM and CPE router.  Since the /29 is presumed to be NATed via your CPE router, there's no need for additional assignment or routing mechanisms.  The IPv6 /56 space is available via DHCPv6 PD for your CPE router to request and assign prefixes from it, and the CM just puts routes in to the CPE router requesting the PD.  Unfortunately I've never been able to get a Comcast tech to confirm this deduction, but it makes sense to me.

If I'm right about pass-through mode, this lends credence to my guess that the cable modem itself doesn't use DHCPv6 or PD to request its user IPv6 space.  It is statically configured via a downloaded CM config file, which sets up the IPv4 and IPv6 static ranges on the downstream interface, and the CM itself provides the inside /64 to your router via SLAAC RAs, and additional prefixes via DHCPv6 PD.

So, short answer is, if you want to use your IPv6 prefixes, you need to have CB reconfigure your router for "pass-through" mode, then your CPE router has to use DHCPv6 PD to ask for your internal /64s out of that assigned /56. 

CAVEAT:  I've found through experience and inet research (mostly on this forum) that the  "Comcast Business Router", e.g., the Technicolor CGA4131COM is broken for both DHCPv6 PD, and for 6in4 tunnels!  I had one after my old SMG died, and my 6in4 tunnel refused to work.  Even with all firewalls turned off, it appeared that the CGA4131COM would block IPv4 protocol 41 (6in4).  Posts I've encountered here and elsewhere have indicated that a firmware update of the CGA4131COM also broke the routing of the IPv6 delegated prefixes.  The CM will delegate a prefix to your CPE router, but will not put a route in for it, so it's useless.  For my HE 6in4 tunnel to work, I had to "downgrade" to a Netgear CG3000CR.  From posts I've read here, DHCPv6 PD also works with this, so I'm working to configure this on my Linux router so I can make use of the native IPv6.

It's a shame that all of these things need to be guessed and theororized, and aren't documented anywhere by CB.  Most of the techs don't seem to know how things actually work (or don't work) either.  It'd be nice if they fixed all this stuff, documented it, and educated their techs.

As near as I can tell, the modem does not add a route for delegated prefixes to the downstream router.  Instead, when the modem first receives a packet destined for a host in the delegated network, the modem sends an ICMPv6 neighbor-solicitation to find the host.  If it receives a valid response from the downstream router, then it sets up a route for the host and forwards the packet to the router.

For some unknown reason, this mechanism stops working after some period of time (days / hours) and the modem must be rebooted to restore downstream connectivity.

More information here: https://forums.businesshelp.comcast.com/conversations/ipv6/ipv6-prefix-delegation-workaround-and-lingering-reliability-issues/5fe0a64fc5375f08cd9aba6d

@jeffbrown Is this specific to certain cable modems?  From what I understand, the Netgear CG3000CR does proper routing for a delegated prefix.

What you describe isn't actually routing at all.  It appears that the device in question simply puts the entire /56 on the inside interface, just as it does w/ the /28, /29, etc IPv4 prefixes.  If this is what's happening, a DHCPv6 PD wouldn't even be needed, since as far as the CM is concerned, the entire /56 range lives on the inside interface.

That's why the CM is sending ICMPv6 neighbor-solicitations.  For this to work, the downstream router must do "proxy-ND" (like IPv4 proxy-arp) to answer for all the IPv6 in the /56 range, and the CM just sends it to the CPE router, which it thinks is the owner of the address.  The CPE router can then route it.

My guess for why it stops working is that it fails when the ND cache expires on the CM, it doesn't do another ND, or the CPE router isn't answering for it.  Or it's just yet another bug in the CM firmware.

I experimented with doing this while I still had the Technicolor CBR CM using a linux daemon called "ndppd" (https://github.com/DanielAdolfsson/ndppd), and it worked.  But is a suboptimal and not-very scalable way of making use of your delegated prefixes, since the CM would have ND entries for every address instead of a handful of routes for the delegated /56 space it doles out via PD.  I don't know what the limit for the neighbor table is in the CM, but it's probably not very big.  It's also inefficient to constantly have neighbor entries timing out and having to solicit for them over and over again.  With enough addresses in use, you'd wind up soaking up some of your bandwidth on that perimeter LAN with ND traffic!

IMHO, the way it should work is like this:

1. CM receives DHCPv6 PD request
2. CM assigns prefix
3. CM either puts a route in to the requesting router, or preferably relies on a routing protocol like RIPng, or an ICMPv6 RA prefix announcements with "AdvOnLink" turned off (which I think is a legal way of advertising a router's routable prefixes).
4. Profit :-)

I wonder if CB really expects us to do proxy-ND for their delegated static IPv6 addresses, as if we're doing NAT or something?  :p

Yup, that's right.  This is what happens on the DPC3941B.

Sending a prefix delegation request isn't even needed and it seems to have no effect on routing.  I tried a wide variety of configurations before settling on a neighbor discovery proxy as a viable workaround.

Pretty broken huh?

@jeffbrown Yep.  Apparently only a few of their cable modems "do it right".  Sigh.

Hmm.  So which of their modems "do it right"?

I was told that my modem was the only one that supported static IPs.

My previous modem was long in the tooth and didn't support IPv6 routing at all.  Replacing it with my current model was a saga in its own right.

@jeffbrown Ha.  Every modem supports static IPs.  Who told you that?

It depends on what you consider "doing it right".  I vaguely remember reading that some CMs supported RIPng.  I've read that the Netgear CG3000CR puts in a route when you get a PD.

That's what I have.  It's an older modem and only supports up to 150mbits/s.  For higher speeds, I believe the only available modem is the Technicolor CGA4131COM, which is super broken for both PD and 6in4.  The only way to make that one work with your static IPv6 ranges is using proxy ND on your CPE router.

@jb_geek Did you get a resolution, even having to do anything hacky, regarding PD? 

I am "lucky" to have one of the illustrious CGA4131COM and a static /29 IPv4 block which requires me to put it into bridge mode. 

I was forced to upgrade when going from 150Mbps to 300Mbps.  Which makes me pretty screwed I think.

I'm using a Juniper SRX300 as my CPE which worked flawlessly, with PD,  when I had the older Netgear: CG3000DCR. 

I would appreciate any suggestions that anyone has... I tried doing just basic static /64 subnet assignments to my LAN side interfaces with basic DHCPv6 on the WAN one from the SRX but I think as everyone has mentioned, the Technicolor when in bridge-mode (required for the static IPv4 block to work) is fundamentally broken with IPv6 PD.

My firmware on the modem is:

Thanks in advance!

I never got any sort of resolution.  I replaced my Technicolor with the older CG3000DCR and things like 6in4 started working again.

I haven't had time to mess with PD or setting up native IPv6 lately.

Are you sure your modem is in bridge mode, or is it in so-called "pass-through" mode?  I have a static /29 and mine is in pass-through, which is basically just router w/o NAT or firewall mode.  In bridge mode, your SRX would have to be doing all the talking with the upstream CC routers directly.  So in that mode, your DHCPv6 and PDs are coming from upstream, and now delegated through the cable modem.  Or at least that's how it should work.