yes, sorry, you're right.
I just wanted to say that there are so many factors outside of your control that affect your ping and may change suddenly (like your/your servers ISP setup, how others servers at your servers hoster are accessed in parallel with your trem server, how many of your neighbours are downloading stuff etc.). Most ISPs don't even guarantee a minimum bandwidth, and I haven't seen any that guarantee good ping times.
I would ping the server for some time (ping -n 360 <server-ip>) to see if there are the same lag spikes. But if they are I don't think you can't do anything about them.