G'evening Qlimax and welcome to the forums!
Cable--->Router(10/100)[Linksys]--cat5-->5 Port Switch(10/100/1000)[Netgear]--cat6-->Computer
that's pretty much what i have, except the router isn't linksys but another 4-port netgear switch.

try plugging your computer directly into the linksys and see if you get lower latency. if so, then there is something up with the switch. try moving to another port (you never know.. i've had ports stop working)

if you still get the bad latency.. try replacing the cat6 with the cat5 directly. still a problem? could try directly from the cable modem... but make sure to have your software firewalls updated and running.

if you still get a problem, i'd look for a new machine to confirm it's not the machine or nic on the box i was testing.. and if both have the same latency, i'd look to the isp to explain it.

on cable modems... you can get 'leakage' which can radically increase your ploss which in turn will drive up your latency

i hope that helps.

Rob