Is it balanced?

[Chrysops]

I'm sure you've heard the criticism that better players tend to be stacked on the alien team since mediocre players are more likely to choose the human team. Therefore the win/loss stats might be 50/50 even though the game is biased towards humans. It occurred to me that there might be a different way of looking at the statistics that would eliminate this variable.

If you could look at individual players win ratio on each team, you might be able to see a bias, if there is one. So if PlayaNoob wins 59% of the time on the human team, but only wins 39% of the time on the alien team, it doesn't matter that this player spends 90% of his time on the human team. I don't know if its possible to track individual players like that, and it doesn't take into account who he is playing against; but if there is a consistent pattern of human win ratios being higher than alien win ratios across numerous players it would reveal a bias that the differences in players' skills is concealing.

What do you think? Is my reasoning logical? Is it possible to look at the stats for particular players this way?

[jm82792]

We can always dynamically adjust things(within a reasonable limit) to compensate for this.
If someone wanted to they could code something that dynamically changes a few variables every month to balance things out.

[OhaiReapd]

Great idea, how would you implement it.

[jm82792]

It wouldn't be horrific but we would have to pinpoint some variables(HP or whatever) that can be changed that shift the balance over some.
Norfenstien would know off hand a few variables that could be automatically tweaked I'm guessing.

[Ripple]

We can always dynamically adjust things(within a reasonable limit) to compensate for this.
If someone wanted to they could code something that dynamically changes a few variables every month to balance things out.

I think this would work, but I don't remotely like the sound of it.

[F50]

What do you think? Is my reasoning logical? Is it possible to look at the stats for particular players this way?

That could work, but in order to get decent results you need a player who has played relatively many (100? 1000? 10000?) games on the team you're looking at stats for, and you have to get statistics for relatively many 'noob' players in order to generalize to noob players in general. since a player really needs to actually be trying for this to work. So if a player plays 90% humans, that player will have to have played a heck of a lot of games to get useful information from his stats.

Lies, damn lies, and statistics. You're reasoning sounds relatively sound, but I'm sure tremulous is not a very ideal situation for statistics, so I'd be very careful with this.

I think looking at total games is better, since you get to analyze more data that way. For various reasons, you're not going to get useful stats for tremulous anyways, so I'm very, very skeptical of this sort of compensation.

[Dr. A. Goon]

I think that a system where the game responds to the player's actions is ripe to be exploited, and a system where the devs change variables regularly is harder to implement and less reliable.

[OhaiReapd]

What do you think? Is my reasoning logical? Is it possible to look at the stats for particular players this way?

That could work, but in order to get decent results you need a player who has played relatively many (100? 1000? 10000?) games on the team you're looking at stats for, and you have to get statistics for relatively many 'noob' players in order to generalize to noob players in general. since a player really needs to actually be trying for this to work. So if a player plays 90% humans, that player will have to have played a heck of a lot of games to get useful information from his stats.

Lies, damn lies, and statistics. You're reasoning sounds relatively sound, but I'm sure tremulous is not a very ideal situation for statistics, so I'd be very careful with this.

I think looking at total games is better, since you get to analyze more data that way. For various reasons, you're not going to get useful stats for tremulous anyways, so I'm very, very skeptical of this sort of compensation.

Like This?

[jm82792]

I mean a long term deal over a month or two of data collection.
Not a day or a week.

[A Spork]

a month or 2 != long term.

[Chrysops]

Whether you won or not seems like such an obvious thing to keep track of that I'm surprised Tremstats doesn't already keep a record of it. Maybe there is something in the code that makes it difficult. Not sure how useful it would really be since players tend to leave the losing team after their spawns are down.

Not sure how I feel about dynamically tweaking variables! Probably the easiest way to give a newbie team a handicap would be a command to let an admin give them kills toward the next stage. You could give them credits, but that would likely benefit the other team when the newbies fed those credits.

It's good to hear that people will still be working on balance, since it might shift as people get more experience with the new game mechanics. Keep up the good work!

[F50]

Tremstats keeps track of that per map. Win percentage per-player was suggested at one point in a previous thread I think, but was rejected because if anyone decides to try to self-evaluate based on win-percentage then we will be screwed. Having people try to stack games to boost their stats would not be a good thing.

[Nux]

A month ago:

I think it's possible that the human/alien win-loss statistics are not being used correctly and that changes to create more balance are not creating a more balanced game. From scanning through individual player stats, it looks like the more experienced, better players are playing alien more often than human. Similarly, newer and overall worse players tend to play humans more often. There are clearly exceptions to this. However, as a general trend, this makes sense. If your background is with other FPSs, humans are simple to pick up, but learning to use a basilisk or the narrow goon chomp is a trick.

How does this impact the stats? If the trend I'm describing is true, aliens win more often because better players are playing aliens, not necessarily because aliens are more powerful. Rather than simply charting win-loss stats, you need to statistically control for the quality of the players in the games. You could use each player's average score per game as a proxy for their skill. It wouldn't be perfect, but it may help you better understand the game's balance. The technical way to see whether this trend is true is to use a statistical method called logistic regression to test how well the score-per-game of a team's players predicts wins and losses. Norf, if you can help me get the data in an easy to use format, I'd be happy to do this.

I don't see anything wrong with Prince_Andrei trying to test his hypothesis. The very fact he wants to test it makes him stand apart from all the speculation shouters. If he's anything like the stats geek he says he is, he will state his assumptions and give his method of analysis and then we can decide whether it's fair or not.

The current balance stats are pretty limited in thier explaining power; far too much information is pooled together in my opinion. I believe they can be greatly improved upon (timbo himself said how they 'aren't very scientific') and if ever I get the data from timbo I'd be happy to try my hand at some statistical analysis on it too.

If anyones worried about how to define 'a skilled player' or other vague concepts like it, this has always been a necessary part of analysis. The best you can hope to do is pick a standard that enough people agree upon and see what the numbers tell you under that assumption.

I have a copy of the code timbo used to graph 'balance' and I've decided that if I could just get my hands on the game data he applied it to, I would be willing to try and code a more explanatory statistical overview.