Saturday, February 21, 2009

Comcast issues in Palo Alto

My setup at home is Comcast for ISP, and Vonage for phone service. I haven't noticed any severe degradation in my Comcast service recently... But then again, I probably don't use my home phone often, and perhaps I'm not online when outages occur.
However, that all changed on Thursday, and according to my neighbors this wasn't a one-time thing. On Thursday Feb 19, 2009, after 2pm and 4pm or so, my connection was pretty unusable. I couldn't use Vonage in a reasonable manner, and had real problems transferring data. This wasn't because of a complete inability to transfer data, it was dropping packets periodically. See the graph that I generated from about that time.
So I called 1-800-COMCAST and I hit the right buttons to report trouble with my "high speed Internet." A recorded informed me that there was trouble in my area, that technicians were working on it, and offered to call me back when it was better. It got better before 11pm that night, but I got my automated call back on Friday at 11am.
Some theories about what's happening:
Recently, about 2 or 3 weeks ago, Comcast made a change and blocked my incoming port 25. Ever since I've had Comcast I could never make an outgoing connection on port 25, which is "normal" for an ISP. But blocking incoming port 25 is deadly if you're attempting to run your own mail server.
The reason I mention the blocked port 25 is that I believe that these problems are related to Comcast's recent changes to control their bandwidth utilization. Comcast isnotorious for sending forged TCP control packets to upset your P2P transfers: that's like blowing out the tires of cars on a congested highway because cars might be carrying illegal contraband. In any case, they have been rightfully remorseful and punished, at least in reputation, for this behavior.
Because of this "turnaround," they've come up with a new scheme to control their traffic. Of course, every ISP has a right to control their traffic so a single subscriber doesn't swamp all other users. But perhaps their current implementation is is still a little green... And thus our current connectivity troubles.
And you can see here we (Bay Area) were scheduled for the switchover at the end of November 2008 or so, but it probably happened more recently, or there are more changes than a simple switch, and takes time to convert all neighborhoods in the Bay Area. But apparently according to this the new system is 100% online a month and a half ago.
I just did another test, and at this time it's much better, but still not great. I should probably set up a monitoring schedule with; it costs a little bit of money, but no big deal.
I've set up the monitoring tool. Here are my up-to-date line monitoring results:
East Coast Hourly Daily
West Coast Hourly Daily