Bridge Mode inbound/ingress traffic issues

[m3_del]

Hi, I recently received a speed upgrade to my service which came with a new Comcast supplied gateway. Previously I had been running on a modem I supplied. I was happy to see I could enable bridge mode for my service (I do NOT have static IP service). I set the gateway to bridge mode, plugged my laptop in directly first and viola! I had a public IP and blazing fast speeds. The tech left and I unplugged the laptop and plugged in my ASUS wifi router (3rd party). It too got a public IP form Comcast and all my outbound traffic worked great! Still blazing fast. We run a couple of simple web sites locally and have port forward rules setup on the ASUS router for forward port 80 and 443 to the appropriate server. This has worked great for the last 10 years and I had anticipated no problems with bridge mode enabled. Sadly that is not the case. While I can ping the IP my router receives, no web traffic is being passed through. I thought it could be something funky with the router connecting to the bridge mode. So I called tech support and unplugged my router from the gateway, and plugged my laptop (firewall disabled!!!!) into the gateway device. I spun up a simple hello world web service on port 80. Alas, still no traffic forwarded to my public ip that my laptop received! We disabled bridge mode, got a NAT'ed IP, setup a standard 80:80 port forward on the Comcast gateway device and of course that worked.

Long story short I want bridge mode to work so that all traffic is sent on to my ASUS router and to allow that to handle all the traffic/forwarding/etc. Double NAT and 1-to-1 NAT sound horrible and make my DDNS solutions less than elegant. HELP!

[Comcast_Robert]

Hi there! I'm so sorry for the delay. During this time we have a need for an increased support, but we are doing our very best to make sure we respond to you as soon as possible. Thanks so much for reaching out about your internet connection. You've absolutely reached the right place, and are in good hands. I will own this Issue for you and ensure that I provide the best help I can today. All I need is your full name, account number (follow link https://comca.st/3hI4SWc, account number is at the top right)and address including city, state, and zip code exactly how it reflects on the bill, and I can help you with whatever questions or concerns you may have. 

[m3_del]

I can not see how to get you this info privately... so here it is. I will spell out my street numbers to try and obfuscate it.

I see I can PM you now... PM sent!

[m3_del]

Just a note to say that I replied back to your DM to go ahead and reset/reboot the modem. Actually doo what you need to do when you need to as I really need this problem resolved. Thanks!

[m3_del]

Hi, not sure if you all work weekends but hopeful for some help today. Adding in here what I have tried from a user standpoint so far. Also sent in my DM.

Here is the list of items I have tried either on my own or with a support associate on the phone with me.

Resetting the modem to factory settings

30-30-30 reset (older router trick)

In NAT mode (non-bridged), disabled the firewall, wireless and then enabled port forwarding (no entries, just enabled) then turned on bridged mode (both advanced and basic)

Straight up bridge mode both advanced and basic modes.

When I use NAT with my laptop hosting a port 80 web service I can set up port forwarding and external traffic flows [ Internet-> gateway IP -> internal IP on my laptop ]

When I enable bridge mode, my laptop gets a public IP as expected but traffic does not flow as expected [ Internet -> Laptop Public IP ]

One thing that stands out to me is that my router seems to have maintained the IP it had before I upgraded the speed of my service. Not sure if that is cause any problems or if there is a way to invalidate the DHCP lock to allow my router to snag a new IP. Complete speculation at this point. 

[SBT]

I'm see the same issue at two client sites both sites have public IP's but I can't access any port forwards when I look at the logs I see the access I'm trying to make but from another IP on comcast's network. Do you have 4G backup from Comcast? I think its related to that because the two sites I'm seeing this at have that.

[m3_del]

No 4G here. Not sure if the gateways is 4G capable though... here are my details.

The gateway model is: CGA4131COM

HW revision: 2.3

eMTA & DOCSIS Software Version:CM DOCSIS Application - Prod_18.3_d31 & MTA Application - Prod_18.3

Software Image Name:CGA4131COM_4.2p7s2_PROD_sey

[SBT]

This sounds like the same issue.

https://forums.businesshelp.comcast.com/t5/Equipment-Modems-Gateways/Do-I-need-to-set-my-gateway-to-...

[SBT]

same hardware here.

Model:CGA4131COM
Hardware Revision:2.3

[seanglgr]

@m3_del I sent you a DM, but I wanted to post here for others to see as well.  I was able to resolve a similar issue (could not connect to VPN) by changing the modem from Advanced Bridge Mode to Basic Bridge Mode.  Thanks and good luck.

[m3_del]

Thanks for the message! I switched back to basic mode this morning just to see if it made a difference. So far no luck.

Does anyone know the difference between basic and advanced mode? I can't find any information.

[SBT]

Created a level 2 ticket, level 1 tech wanted me to turn of bridge mode to troubleshoot. Disconnected everyone from the internet and losing direct access to the internet for my VPN feature was not a step forward.

[Comcast_Anisa]

Hi there :)! Thanks for taking the time to reach out to the Digital Care team here through the forums and we are so sorry to see that you are having some issues with your internet service. We do understand how important having reliable internet is and you have reached an amazing team to help! Can you please send us a private message with your name, the full address, and the phone or account number? 

[m3_del]

Hi, are you referencing me? I have already messaged all my info. I was told Saturday I should hear from someone yesterday or today. I am hoping today! I believe I passed along my email and phone number...

Does anyone know the difference between basic and advanced bridge mode?

[Comcast_Gabe]

Thank you for your patience and for that great question, Craig. After further research, I have discovered that the difference with the Advance Bridge mode is it leaves the Xfinity WiFi hotspot active.

[m3_del]

So advanced Bridged mode leaves the wifi hotspot active?

[m3_del]

So it seems others are seeing similar issues with, perhaps this HW, not passing through traffic correctly in bridged mode. Support is telling me (via voicemail and then closing a ticket, BTW) that this is a de-mark issue so I need to contact my IT (which is me).

Where can I get help? I am having to have the poor support gal open yet another ticket for me regarding this.

[Comcast_Gabe]

Thank you for your time and patience. You are correct. The advanced bridge mode leaves the option to have the WiFi hotspots active. In reviewing the account, I do see that we have another ticket open with our Advanced Repair team. What I will do from here is monitor this ticket and follow up with you in 24 hours to make sure you are contacted. How does this sound?

[m3_del]

OK, So with the help of a friend I am starting to get to the bottom of this. Here are some items we ran through with Wireshark on my end.

Gateway in bridge mode for all of this. Laptop plugged directly in with public IP address.

ICMP (Ping from my buddy's house to my public IP on my laptop): Wireshark file: brad-icmp | Wireshark filter: icmp

• Success! pings happen. Wireshark see's them all. I see the request and the reply. It does not show my friends public IP as the source, rather it shows the IP address the gateway has received. I assume this is normal.

Traceroute from buddy to me: Wireshark file: brad-traceroute | Wireshark filter: tcp.port == 787

• Seemingly a failure. He targeted my port 787 (which I was hosting a web on on at the time). We give up after 100 plus hops most of which are unknown/unreachable addresses. Wireshark does see traffic during this time. A LOT OF RETRANSMITS START TO OCCUR. More on this later.

At this point I started my web server. I decided to listen on port 787 rather than 80 to try and quiet the noise in Wireshark. This worked well to isolate traffic!

TCP locally on my laptop to port 787 on my laptop Wireshark file: localhost-port787 | Wireshark filter: tcp.port == 787

• Success (big surprise!) Wireshark sees exactly 4 transmits. The GET followed by my ACK, Then the html payload being sent (in clear text since not using https) and the ACK.

TCP from external user (friend) to my laptop port 787 Wireshark file: brad-tcp-port787 | Wireshark filter: tcp.port == 787

• Failure. Wireshark sees the attempts and it just floods in with re-transmits. A lot of RST and TCP Retransmits.

I am not network expert but I believe there could be some packets being malformed as they pass through the gateway device.

I have captured all of the wireshark outputs and am more than happy to share and even walk through live over a zoom/webex/teams/whatever screen share tool of your choice. Please let me know if you are able to update the ticket with this information to help with the troubleshooting.

Here are the Wireshark captures. I have updated the BOLD above to correlate the captures with the traffic type.

https://gofile.io/d/hdiKbS

Because I am not going to leave my laptop plugged in to my gateway all night. If you need a place to test against you can use the IP my router has on port 80 73.97.101.85 -> when this works my web service should just return a 404 page not found error

[SBT]

My call back from Level 2 support said I need to factory reset the comcast router. I'm trying that now.

I see the traffic coming in my logs thru my port forwards but the source IP is not what I expect and nothing works. Sounds like we have found a firmware bug or config bug that's not large enough for them to see enough issues to take notice.

[m3_del]

Unless they have a special reset different from the one we can try thru the GUI, I have tried that more than one time 😉

In NAT mode using the gateway's port forward to my laptop plugged directly in, everything works great.

In bridged mode with the laptop getting a public IP... no dice. My laptop see's the traffic but it appears to be malformed packets (or something is getting in the way of response). I will have my friend run Wireshark captures from his end tomorrow to see what it looks like from his end.

All the traffic inbound looks like it comes from the gateway IP (even in bridge mode). I am not sure if that is to be expected or not though.

[SBT]

I held little hope that a reset was going to fix it, but I jump thru the hoop.

The onsite gateway is mangle the src address or something.

This is a log entry of me sending an RDP connection thru a router at customer with comcast that is working. (diffrent model modem) I had to change the source address because it was my IP

Sep 21 20:29:08 Board007 kernel: [WAN_IN-3003-A]IN=eth0 OUT=eth1 MAC=04:18:d6:a1:24:53:28:52:61:f0:34:22:08:00 src=XXX.XXX.XXX.11 DST=10.6.7.110 LEN=52 TOS=0x00 PREC=0x20 TTL=119 ID=30148 PROTO=TCP SPT=41266 DPT=3389 WINDOW=64240 RES=0x00 SYN URGP=0

This is a log entry doing the same to a router connected to the not properly working modem.

Sep 21 14:16:57 Board005 kernel: [WAN_IN-3003-A]IN=eth0 OUT=eth1 MAC=04:18:d6:a1:24:11:0c:11:67:02:48:22:08:00 src=73.11.255.94 DST=10.6.5.25 LEN=52 TOS=0x02 PREC=0x20 TTL=102 ID=20303 DF PROTO=TCP SPT=61896 DPT=3389 WINDOW=64240 RES=0x00 CWR ECE SYN URGP=0 MARK=0x64800000

I have no idea what's with this source address, its not my external IP address that I expect it to be.

I have come to the conclusion that the fastest resolution since I don't expect any help from comcast is to replace the modem with a store bought modem. I really like the Aris surfboard modems and have several deployed a clients and it saves them $15/month too.

[Comcast_Michelle]

Thank you so much for taking the time to supply all of the details.  A bridged modem will only assign dynamic public IPs to connected devices in the same manner as a basic cable modem. When you are setting up bridge mode are you using basic bridge mode? If so do you have your device set to allow WAN DHCP?

[Comcast_Robert]

That is actually a really good modem! If you are looking into getting a modem here are the ones that work best with our Business Services, here is the link below. I would absolutely like to troubleshoot your modem further. Could you respond using a private note, just in case we need to dig deeper into your concern and need to access your account information. I would like to take a few more steps with troubleshooting your service, but it might impact any work that you are currently in process of working on. Would now be an okay time to work on your service? 

https://comca.st/2FSzhnU

[m3_del]

While I am sure you are asking to work with SBT. I am still up and open to working on troubleshooting this issue.

If you are open to trying a few things, let me know.

[m3_del]

This is the problem! Thanks SBT for tag teaming this with me!

When I switch the gateway to NAT mode and port forward to my router which then port forwards to my server the SRC address shows my clients IP. (Double NAT's make me cringe)

When I have the gateway in bridged mode, my router gets a public IP but and maintains the same port forward settings as above. traffic shows up with injected packets where the SRC IP is actually the IP of the Gateway (yes it still gets an IP in bridged mode [Gateway->Connection->Comcast Network].

This is most certainly a bug. I am more than happy to document all of information that illustrates both scenario's

[m3_del]

Ok, heading to bed. I will document the two scenario's in the morning.

SBT I can only imagine getting a fix for this will take some time as it is most likely a software fix and will require first being put on someones radar, having some sort of priority then finally going through test/Q&A before being released. If you need a fix in a timely manner my guess is purchasing your own modem is going to be the way to go. Sad thing is Comcast asked me to not bring my own as bridge mode would work just fine... 

[Comcast_Robert]

I truly apologize that you are currently having issues with your service. I know issues with the consistency of your internet service can affect your business and I would like to get to the bottom of this before it becomes a bigger problem. Would it be okay to further troubleshoot your service. This may affect any work you have in progress. Would it be okay to try a few more steps at this time?  

[m3_del]

I can only perform disruptive activities before 8am pst or after 3pm pst

I was available around 11:30pm right after you offered help. But couldn’t keep my eyes open until 2am. I will document the bug in your firmware this morning if you are interested.

[m3_del]

I saved my post that was just marked as spam (who knows why) if anyone from Comcast would like it. In my post I detailed my setup in both bridge mode and NAT mode showing the Wireshark captures that outline the supposed bug we found with these gateways when set in bridge mode.

The long and the short of it is that in bridge mode the gateway is injecting its IP into the source field versus passing the actual source on.

[Comcast_Anisa]

Thank you so much for providing that information as this does give us a better understanding of what is going on. I do see that we still have our open ticket for advanced repair and we should be reaching out to you within 24-48 hours. I know how frustrating this has been for you and we really do appreciate your patience with us and I will reach out as soon as we have an update with the request. 

[m3_del]

Do you know why my post was marked as spam? Here it is with IP's edited out (assuming that is why this was marked as spam)

Simple testing reveals the bridge mode issue. in both examples I am hosting a simple web page on my laptop on port :787 (to keep the filters simple in Wireshark).

Comcast Gateway Model: CGA4131COM

HW Version: 2.3

Boot Version: S1TC-3.60.19.137

Download Version: CM DOCSIS Application - Prod_18.3_d31 & MTA Application - Prod_18.3

Double NAT Working

The above Wireshark capture is my laptop behind my ASUS router with (192.168.0.X). *IP's removed*

1. The Comcast Gateway (Public IP 76.104.XXX.XXX) is in NAT mode (10.1.10.X). 
2. Only the ASUS router is plugged in (10.1.10.XX)
3. My laptop is connected to my ASUS router via WIFI. (192.168.0.XXX)
4. Gateway port forward is 787->787 to my ASUS router
5. ASUS port forward is 787->787 to my laptop

Notice even through this mess of a double NAT the Source IP is showing the client inbound IP (73.83.X.X). We validated that was his IP. This is expected behavior. He gets to the web pages listening on my laptop on port :787.

Now for the problem.

Bridge Mode Direct not working

The mess above is now showing what happens when my same laptop, hosting the same web page on port :787 is plugged directly into the gateway while in bridge mode. The client is now reaching out to my laptop via the public IP my laptop gets via Comcast DHCP. Notice in this session the SRC, which should be my clients IP, has been replaced with the IP of the gateway device. Now my web server does not know where to send ACK's and the web page data. This setup is

1. The Comcast Gateway (Public IP 76.104.XXX.XXX) is in bridge mode (basic)
2. Only my laptop is plugged in and getting a public IP (76.104.XXX.XXX)
3. Website is the same and still hosted on port 787.

[Comcast_Gabe]

Thank you for providing these details and for sending over those pictures! We appreciate you for the time you have spent on this so far. We want nothing more than to get this resolved for you. Most of these settings are beyond our demarcation of support as internal networks vary so drastically from business to business. Thanks for that great question about the posts being marked as spam. We definitely would not want your IPs posted publicly for security purposes. What I would like to do from here is monitor the request that we have opened with our Advance Repair team who will be contacting you within the next 24-48 hours. Our Advance Repair team and will work hard with us with dedication and commitment to resolving the modem concern as quickly as possible. In the meantime, please feel free to reach out for any additional questions or concerns. Thank you for your business and patience.

[m3_del]

Thanks for the note Gabe.

The issue (second part) is in no way past your Demarcation. Hence I am directly connecting to the gateway in bridge mode. As user: SBT is also seeing the gateway is doing something with the packets as they pass through to the laptop public IP that replaces the SRC IP with the IP of the gateway. Without getting shell access into the gateway to run a TCP dump there is no way for me to prove this to you besides what I am showing here. The standard answer of demarcation is not acceptable. At this point if that would be the answer it would need to be proved to me that this is in fact an issue on my end. I have provided mountains of data that I believe show the issue is with the gateways bridge mode mangling packets.

[m3_del · Accepted solution]

Welp, this is now resolved. I was in a meeting and notice my connection blipped. Logged into the gateway, checked out the logs. Saw that them mso user logged in and changed my mode back to bridged advanced and then bridged basic. Had my client try to access my web pages and it now worked. I even grabbed a tcp dump filtered on port 787 to show the now bridge mode success!Bridge Mode success!

If all the techs did was switch back to bridge mode then I am at a loss as to what was happening. To future people with this issue. This is certainly a prove innocence model! I recommend creating tcp captures to illustrate the issue. Hopefully Comcast_ will be able to shed some light on what was occurring here! Thanks Comcast, I know I was a pain through this 😉 just needed resolution!

[SBT]

Just so I understand, Comcast did something on their end to fix your issue.

[m3_del]

To rephrase what you just asked...

Comcast did "SOMETHING" to resolve this.

The something is where there is a mystery. I hope they do "SOMETHING" to resolve this for you so you do not have to spend capital on standard modem. 

[SBT]

If its not to much trouble, I would be curious what your router software version is, I wonder if the downgraded or upgraded your firmware?