After the last test, which wasn’t much improvement, my suspicion was that the tcp keepalives for all those connections were swamping the interface. So i installed iftop and reran the tests to check:

echo Install iftop
sudo yum -y install libpcap*
sudo yum -y install ncurses*
cd ~
tar -xzvf iftop-0.17.tar.gz
cd iftop-0.17
./configure && make && sudo make install
sudo ./iftop

I didn’t get any further, but it did confirm that once the sockets were open that there is no load on the network. Seems like the keepalives aren’t major (or are nonexistent).

Another thing i found to do was ss -s to show how many sockets are open at one time. Very handy for screenshots :)

Thanks for reading! And if you want to get in touch, I'd love to hear from you: chris.hulbert at gmail.