High-latency link between hard-wired modem and router

I'm seeing a long ping latency between a directly connected Sophos UTM 9 router to a CGA4341COM Gigabit modem. Both report a Gigabit link. Speedtest gives 400Mbps range results from a hardwired on-LAN server. After a long test on the UTM 9 to the directly-connected modem using a 10-foot Cat5e cable:

Out of 756 packets transmitted, I get 0% loss but an average ping time of 22ms and a max of 200ms where it should be a steady 0.2 ms.

Trying this all day long changed nothing. All extraneous features of the modem are disabled: port forwarding, port triggering, firewall, MAC access control, dhcp, wifi, etc.

On the sophos, no packet errors.

On eth0, the router is pinging a server on-LAN with the typical 0.1-0.2ms very steady latency. There's no loading (CPU or memory or disk) on the Sophos at all, nor anything remarkable in the logs, nor anything in dmesg.

The modem has almost nothing to report when searching 90 days of logs.

Much more detail can be found on serverfault: https://serverfault.com/questions/1021775/high-ping-latency-over-direct-gigabit-ethernet-link-on-enterprise-class-hardware