It's the time taken in milliseconds for data to travel between you and the server, it's important when games don't use any kind of lag compensation as the lower your ping the less you need to compensate manually and the better you're able to play, with unlagged it's still an issue but a relatively high ping is no longer a serious barrier to play, it just causes some peculiarities.
Ping is primarily down to your physical distance from the server, if you connect to servers on the same continent you'll often find that you get a better(lower) ping.
This is all very simplistic and i'm sure things like packet loss and it's effect on ping are important, but i don't really know enough to attempt to explain that.
This may help,
http://en.wikipedia.org/wiki/Ping
