[colug-432] possible LAN issues - tcp retransmits

[Rick Hornsby]

Background (skip this for the short version below) -
Had a wide spread residential ISP outage a few weeks ago.  It was only 
down for a couple of hours, but when it came back the IPTV service was 
bad - lots of pixelation and lost audio, etc - but it was also 
intermittent.  The problem is not related to the TV, the HDMI cables or 
anything like that - the same DVR'd program shows the same problems at 
the same time on two different TVs (common DVR storage).  The problem 
also happened on live tv (so not a DVR storage-specific issue).

After a few calls, they sent a tech out.  By the time he came out, the 
problem seemed to mostly be cleared up on its own - I couldn't 
demonstrate the problem on live tv.  He said that I wasn't the only one 
complaining, that they'd found some part common to several houses that 
was misbehaving (OLT maybe?).  He also did a couple of things in the 
house - changed an RG-6 filter, re-did an ethernet cable.

Everything looked fine when he left, but a day or so later a TV show 
that was recorded had the really bad pixelation and audio again.  I 
called and they escalated the ticket.  Since then, it seems to be fine - 
a minor glitch or two is all I've noticed.  The ISP called yesterday to 
follow up with me, told them it seemed to be fine.  They queried me 
about what kind of gear was on the network in the house, and relayed 
that they noticed a large amount of "chatter".  When I pressed the guy 
on the phone, he said that the notes said it looked like a lot of tcp 
retransmit packets and that it might be a source of a problem.  He also 
said that it usually happens because some piece of equipment is failing.

Query -
I found some info on the interwebs about using tshark like so:

     tshark -Y "tcp.analysis.retransmission" -Tfields -e ip.src -e ip.dst

I'm not sure what to do with this information, or what constitutes an 
unacceptable volume of retransmits. (This isn't showing me anything 
about the rest of the un-retransmitted traffic for comparison.)  tshark 
on the basement server (archer) is spitting out a pair of IP addresses 
(one itself, one remote) - maybe 250 of them in the last 20 minutes?  
I'm running the same tshark command on my OSX box upstairs (aztec), but 
it doesn't have a counter.  Maybe 50 in the last 15 minutes?

I ran an ISP speed test from aztec (Flash based, so can't run it on 
archer) and there was a burst of transmit packets (50? 100?) says 
tshark.  No idea if this is normal?

I don't know if this is related, but on archer, the arp table is full of 
LAN IP addresses for devices that don't exist.  There are 64 devices in 
the table (The DHCP server shows only 12 devices with leases, which 
seems more accurate), 45 of which are showing "incomplete".  aztec only 
has 1 incomplete entry, the rest look like they belong there.

Any thoughts on what to look at or how to better use tshark?  Is 
archer's arp table weirdness a red herring?


thanks!
-rj