February 7 Firmware Upgrade Broke IPv6

Thanks for your response. I already have a ticket [Edited: "Personal Information"] and if you can cause it to be expedited that would be great.  I do not want a tech, and do not want a new modem.  Getting a new modem would change my IPv6 delegation and would not fix the problem.  It would download the same broken firmware.  The only way to fix this problem is to revert the firmware (which the tech cannot do) or fix the current version.

@user_c7e85c Thank you! I'm happy to hear that the request was already created. Unfortunately the system will not allow me to access the details needed to work with our peers to have the request escalated. I have to verify the account first. If you could please send us a direct message. I would need your name, the serivce address and the associated phone number, MAC address or Account number to verify the account and access the pending request. 

Please send us a direct message with your full name, full address, and phone number. • Click "Sign In" if necessary
Click the "Direct Messaging" icon in the top right corner
Click the "New message" (pencil and paper) icon
The "To:" line prompts you to "Type the name of a person". Instead, type "Comcast Business" there
As you are typing a drop-down list appears. Select "Comcast Business" from that list
An "Comcast Business" graphic replaces the "To:" line
Type your message in the text area near the bottom of the window
Press Enter to send it

@idolum Please send us a direct message with your full name, business name, full address, and last four digits of your account number or full phone number.
 
• Click "Sign In" if necessary 
• Click the "Direct Messaging" icon in the top right corner
• Click the "New message" (pencil and paper) icon 
• The "To:" line prompts you to "Type the name of a person". Instead, type "Comcast Business" there 
• - As you are typing a drop-down list appears. Select "Comcast Business" from that list 
• - An "Comcast Business" graphic replaces the "To:" line 
• Type your message in the text area near the bottom of the window 
• Press Enter to send it

Hey Allen. I do see the ticket open on the account, but until the tier 2 team reviews it to contact you there wouldn't be anything more I can do to speed it up. At this time the ticket is open for review for the tier 2 team, and they should reach out as soon as they work through your ticket. 

Thank you Eric. Can you add this thread to the ticket? I hope it convinces Tier 2 this is a wide spread issue affecting multiple users. Maybe they can finally get some movement on this, and not just troubleshoot my circuit. To add, I also see "Firewall Blocked" messages on my firewall logs similar to the screenshot above.

Thanks @user_c7e85c.  The firewall just confirms the problem and I'm not going crazy. I have been trying to convince them it was an upstream issue, not thinking the modem is not honoring the "Disable firewall" setting. I added myself and my ticket to the DM as suggested earlier in this thread - every bit helps since all we can do is continue to complain.

I can notate it, but they normally wouldn't look at a Forums tread as they work on a case by case basis. Never hurts to try though. 

@Comcast_Eric​ Thank you. I'll mention it to them as well when they call.

They should have rolled back the update the day after we all reported this. It's ridiculous this is still broken, and Comcast doesn't appear to really care. The fix is simple. Yet they won't do it.

@gantzm_mi​ 

What makes this even more frustrating is that the firewall is doing a DROP instead of a DENY.  The applications have no idea what's going on and you have to basically wait for a TCP timeout to occur for the application to incur an error.

@Comcast_Eric​ 

Xfinity Support does not show up in the list.  Is that because I have a business account or something?

"Comcast Business" - in a previous message does show up, sent a message there.

That was the right move @user_c7e85c. I just replied to your message, and we'll be on standby.

gantzm_mi 'm glad everything is working out as it should please enjoy your weekend

@user_c7e85c​ 

Do you know if there is any way from the modem web interface to know when the latest firmware was installed?

I don't think so. You can check the version under Connection/Comcast Network/Cable Modem. If you haven't rebooted your modem since the problem started, you could check the uptime.  My boot version is S1TC-3.77.21.67 and is definitely broken. I don't know if the version varies with the model of modem.

@user_c7e85c​ 

Yup, I've got the same exact version.  Go figure.

I think the version you want to look at us under 'Software' on the left then 'Software Image Name'. On the CGA4332 modem, version 6.7p7s3 is the (current) broken version, 5.6p7s1 is the previous one which was working.

@allan​ 

Outbound only works on the IPv6 subnet local to the router.  If you actually try and use the /56 subnets behind a router they get blocked on the outbound side at the modem.

So yeah, you get a /56 but only the first /64 is actually usable.

This was in fact working perfectly find just a little while ago.

@gantzm_mi​ My setup and symptom are a bit different. Out of the 2603:3018:xxxx:xx00::/56 delegated to my account, the Comcast modem takes the first 2603:3018:xxxx:xx00::/59 and uses a /64 from that for its local switch DHCPv6. My firewall is then delegated the next one 2603:3018:xxxx:xx20::/59 and I use the first /64 out of that for my LAN. Whatismyip.com confirms it sees me on that delegated 2603:3018:xxxx:xx20::/59 subnet. This part continues to work; I am replying to you on it. I have no issues with outgoing connections before or after I factory reset the modem.

@allan​ I agree with you that it is going to be a long time, and the situation is absurd. However, I'm having a hard time empathizing with your 13 minutes.  I don't think I have ever gotten a human in less than 13 minutes.  By the time I negotiate their automated menu hell, get forced to reboot my modem at least once even though there's no chance it will make any difference, wait the required 10 minutes, and finally get a clueless person, it's usually at least 15 minutes, more like 20. Then it's at least 15 more before I give up, recognizing that the technical support agent doesn't have any idea what IPv6 is, and doesn't even understand the concept of packet delivery. If I look at the ticket later, it usually says something like "complaining that he can't get to an internet site."

@user_c7e85c​ That 13 minutes was actual conversation time with this one Tier 2 support person - trying to convince him of the problem. I already spent a couple of days on this, and this is my 4th ticket on it. I didn't count how much time it took to convince Tier 1 to escalate, but I'm sure we have similar experiences.

Edit: I looked at the comments on my tickets. The first one said "Sales Pitch" !!

@allan​ 

OK, this is getting more interesting.  So I configured IPv6 to Custom Security on the modem and selected "Disable entire firewall".

Now, my outbound connections are working same as yours.  But inbound connections do not work, same as yours.

Great news, and thanks! Without naming names, how did you get to somebody like that? I still haven't found anybody who understands what IPv6 is and the difference between inbound and outbound.

@gantzm_mi​ That's great news!

@user_c7e85c​ 

I basically do this for a living. So I asked nicely and provided plenty of details along with experiments I did showing how the modem works with the different settings.  I also detailed how the current settings differ from how they used to work.

I also have some good news to share! I just got off the phone with a Tier 2 support person. He said that engineering has identified some Comcast Business modems running on "version 2" are experiencing the symptoms I reported. He created a new ticket for them to look at my modem.

Astounding! I got all the way to a tier 4 tech and he still doesn't seem to understand that I'm not complaining that there are certain websites I can't get to over ipv6. Plus I think he spends his days sitting at his desk waiting for other people to do things. Every time I write him, he says, "I haven't heard back from the CPE team" and then doesn't do anything until I write him again.

Maybe we'll get it sooner than you think.  If Comcast did any firmware testing at all, they wouldn't have missed this HUGE issue.  I'm guessing they will skip the testing phase.

I would like to add that I am grateful to the moderators of this forum for believing us when we said there was a problem, and for pursuing it until it was fixed.  I do not think it would have happened without their intervention, and certainly not as quickly.  

Hello adamr thank you for taking the time to reach out to us on our Business forums. You should see this information when logging into the modem via the Admin page.  Our article below will show you how to do this. 

Set up and manage your Comcast Business Wireless Gateway - Under the final header "Log in and Secure the Admin Tool"

@Comcast_Alfonso​ This issue would not cause a slowdown.