It’s been a while… There are some tools you can use that will give you a trace of the ping values from your location to an outside IP address. I suggest exploring it that route so that you can figure out if it is your router or a port that is wrong, or if it is a bottle-neck somewhere outside of your house.
Google: tracing my IP route.
Sorry I am not more help. Networking issues blow chunks.