Author Topic: Gameplay Preview Phase 1 Results  (Read 197803 times)

Norfenstein

  • Posts: 628
  • Turrets: +81/-78
Gameplay Preview Phase 1 Results
« on: December 13, 2009, 08:03:09 pm »
Well the results are in: we've collected data on more than 200 hundred games on both official servers, and can now be confident enough about the trendlines to move forward. Here are the graphs, as of a few hours ago (each cross is one game, the distance from 0 on the vertical axis indicates how close a game was to the mean duration for all games on the graph):

Both servers combined


Euro server (unlagged OFF)


US server (unlagged ON)


Obviously not quite what we hoped for (and, apparently, the opposite of what many of you expected) but this exactly why didn't use the development games alone to balance the game. This data tells me two things:

  • Aliens are winning significantly more games than humans
  • The win ratios on the US and EU servers are very similar

About unlagged
The graphs on the EU and US graphs look oddly different, but this may be because of the smaller number of games in the upper playercounts skewing the trendlines. However, I think the outcome is close enough to justify not having made balance changes specific to unlagged settings. To be certain about this, the best thing to do would be to swap the unlagged settings on the two servers without making any other changes, but considering that both are unbalanced in the same way anyway I think this would be a waste of time. We're going to turn unlagged on for both servers for three reasons:

  • Unlagged does not appear relevant enough to spend more time on right now.
  • Changing the setting on one server is almost as good as changing it on both because the difference between the two servers can then be compared to see if the disparity has more to do with player populations.
  • I've had requests to turn unlagged on for the EU server and know of people that would stop playing on the US server if it had unlagged turned off. I can't say the same for the reverse of either. Unlagged is the default we're going with and it looks to be highly likely that balancing for unlagged on is the same thing as balancing for unlagged off.

About the balance
Given how complicated Tremulous is, knowing what to change is a much harder question than knowing what needs to be fixed. There's no getting around needing to use intuition at some level, and more importantly: multiple cycles of tweaking and re-testing (and no, the time spent changing things and playing on the development server wasn't enough, it always had to be with a much larger playerbase). And we do have the time for these cycles, as there's still plenty of work being done on everything not gameplay-related.

From the data we've collected and my experience playing and watching people on both servers, I believe the imbalance shown in the graphs is because of a deficiency in human offense, and to much lesser extent the continued usefulness of egg spam. To see if this really is the right idea, I'm going to make just a few small changes and we'll test again to see if the balance moves in the right direction.

  • Chaingun damage reverted to 6 - The difference wasn't enormous when I lowered it some time ago, but I believe now it was probably on the wrong side of "almost right". Note that the spread is still slightly better than it was in 1.1 (900 instead of 1000).
  • Flamer damage increased to 25 - Even being 50 credits cheaper, the flamer doesn't seem be more useful than other weapons (for four times the price it's gotten about as many kills as the painsaw). It was, however, very useful when the dretch health was low enough for it to kill them in one hit, so we'll see how it goes when it's like that again.
  • Granger spit slow time reduced from 5 seconds to 1 second - I've always felt that the granger's slowing ability was far too powerful for a free class not intended for combat of any kind, and alien bases have enough defensive capabilities as it is. More importantly this will make it just a little harder for grangers to run away from humans when they have no base (though if their aim and timing is good enough they can keep a human slowed with repeated shots).

I don't expect these changes to fix the balance entirely, but if it helps then it'll be good indicator that I'm on the right track. After 200 more games on both servers we'll try this again.

David

  • Spam Killer
  • *
  • Posts: 3543
  • Turrets: +249/-273
Re: Gameplay Preview Phase 1 Results
« Reply #1 on: December 13, 2009, 08:09:10 pm »
Kill count and stage up graphs would be nice.
I think humans are getting more kills and getting stages sooner which is what's causing people to think they are stronger.
Any maps not in the MG repo?  Email me or come to irc.freenode.net/#mg.
--
My words are mine and mine alone.  I can't speak for anyone else, and there is no one who can speak for me.  If I ever make a post that gives the opinions or positions of other users or groups, then they will be clearly labeled as such.
I'm disappointed that people's past actions have forced me to state what should be obvious.
I am not a dev.  Nothing I say counts for anything.

FreaK

  • Posts: 295
  • Turrets: +45/-59
Re: Gameplay Preview Phase 1 Results
« Reply #2 on: December 13, 2009, 08:13:03 pm »
I think you should look at the basi gas, and maybe the mara zap. If both are used they are extremely powerful, maybe increase the speed of the human when you get gassed. Interesting stats :P

bob0

  • Posts: 80
  • Turrets: +11/-9
Re: Gameplay Preview Phase 1 Results
« Reply #3 on: December 13, 2009, 08:36:03 pm »
I think you should look at the basi gas, and maybe the mara zap. If both are used they are extremely powerful, maybe increase the speed of the human when you get gassed. Interesting stats :P

Zap is really only effective against cluttered bases.  It's hard as a marauder to use zap against a spread out base; usually chomp would be more effective, although sometimes marauders can't do much damage against well built human bases.  Humans should try to spread out their base especially when aliens are stage 2, much like aliens try to spread out their base against human grenades (s2).  Unfortunately, as the most active player on US1 as of December 10, 2009, I've found that humans sometimes camp inside their closed-in 1.1-styled based, which I think was a major factor in the imbalance, which was, funnily enough, tilted towards the alien team.
« Last Edit: December 13, 2009, 08:38:31 pm by Z+ SM p(EEn- »
bob

Lakitu7

  • Tremulous Developers
  • *
  • Posts: 1002
  • Turrets: +120/-73
Re: Gameplay Preview Phase 1 Results
« Reply #4 on: December 13, 2009, 08:59:51 pm »
These changes are now up on official US1 and EU1. Don't forget that unlagged is now on on both servers. The votable map list has also been expanded considerably on the EU server to match the US server.

kevlarman

  • Posts: 2737
  • Turrets: +291/-295
Re: Gameplay Preview Phase 1 Results
« Reply #5 on: December 13, 2009, 09:09:15 pm »
first graph, AS3 vs HS3 (146.30%)? explain?  ???
that's 146 games, 30%. there is no '.'
Quote from: Asvarox link=topic=8622.msg169333#msg169333
Ok let's plan it out. Asva, you are nub, go sit on rets, I will build, you two go feed like hell, you go pwn their asses, and everyone else camp in the hallway, roger?
the dretch bites.
-----
|..d| #
|.@.-##
-----

StevenM

  • Posts: 292
  • Turrets: +40/-33
Re: Gameplay Preview Phase 1 Results
« Reply #6 on: December 13, 2009, 09:13:00 pm »
missed the combined part :D. i would like to see more data though. eg. weapon data.

AllmanBros

  • Posts: 99
  • Turrets: +5/-2
Re: Gameplay Preview Phase 1 Results
« Reply #7 on: December 13, 2009, 09:47:15 pm »
While these results did surprise me, I believe part of the bias towards aliens could be due to people still trying to play trem like its 1.1. Like has been stated before, mara zap is very good against closed in bases, usually used in 1.1. Other things that used to fly in 1.1 don't work well anymore, and its all about adjusting.

When people get accustomed to the new settings, be it alien ranges or new building styles, and maybe the results will even out more.

Who knows, though.

rotacak

  • Posts: 761
  • Turrets: +39/-64
Re: Gameplay Preview Phase 1 Results
« Reply #8 on: December 13, 2009, 11:02:58 pm »
Granger spit slowdown reduced to 1 sec? Granger is too dangerous? You can easilly hit dodging human with granger spit? I don't think so. It's hard and with 1 sec duration it is useless, you can remove it entirely like hovel...

your face

  • Community Moderators
  • *
  • Posts: 3843
  • Turrets: +116/-420
Re: Gameplay Preview Phase 1 Results
« Reply #9 on: December 13, 2009, 11:07:14 pm »
Granger spit slowdown reduced to 1 sec? Granger is too dangerous? You can easilly hit dodging human with granger spit? I don't think so. It's hard and with 1 sec duration it is useless, you can remove it entirely like hovel...

+1
spam spam spam, waste waste waste!

Winnie the Pooh

  • Posts: 442
  • Turrets: +45/-85
Re: Gameplay Preview Phase 1 Results
« Reply #10 on: December 14, 2009, 01:22:44 am »
Granger spit does more to encourage people to try and hit humans when they don't need to.
Quote
I also realize that this is the internet, but even more so this is the forum for a video game on an internet, then even beyond that this is TREMULOUS forums the Satan version of all video game forums for a video game that is ON the internet.

KamikOzzy

  • Posts: 742
  • Turrets: +317/-172
Re: Gameplay Preview Phase 1 Results
« Reply #11 on: December 14, 2009, 02:39:51 am »
I don't know how the games are going down with you, but I know any time we have had a good group of players, even in 1.1, if a certain team is losing more, then the good players will switch to that team the next round. To be honest, when I see a great players assembled on AA, the balance is 1:1, as the game is decided more by who is playing than what team they're playing on. Is it possible that your players are skewing the balance towards aliens because they have been told it is the less likely team to win? If that's the case, the balance may not be leaned towards aliens as much as it seems from the stats. I still hold the belief that good players have a better feeling of balance than numbers and statistics ever will, because there are too many independent variables.
|AoD|Ozzyshka at your service.
Still using Windows XP and still playing 1.1
click this: http://cornersrocks.shop-pro.jp/?pid=16232798

Repatition

  • Posts: 332
  • Turrets: +11/-196
    • http://jbjarts.blogspot.com/
Re: Gameplay Preview Phase 1 Results
« Reply #12 on: December 14, 2009, 05:16:29 am »
Length of games would be nice...

mooseberry

  • Community Moderators
  • *
  • Posts: 4005
  • Turrets: +666/-325
Re: Gameplay Preview Phase 1 Results
« Reply #13 on: December 14, 2009, 05:43:07 am »
It says the mean game length is 15 min. That seems a bit low to me but w/e.

Seeing game length for every game would be _really_ complicated/confusing and not so helpful.
Bucket: [You hear the distant howl of a coyote losing at Counterstrike.]

मैं हिन्दी का समर्थन

~Mooseberry.

kevlarman

  • Posts: 2737
  • Turrets: +291/-295
Re: Gameplay Preview Phase 1 Results
« Reply #14 on: December 14, 2009, 06:37:05 am »
min/max/standard deviation would be nice to see though.
Quote from: Asvarox link=topic=8622.msg169333#msg169333
Ok let's plan it out. Asva, you are nub, go sit on rets, I will build, you two go feed like hell, you go pwn their asses, and everyone else camp in the hallway, roger?
the dretch bites.
-----
|..d| #
|.@.-##
-----

cron

  • Donators
  • *
  • Posts: 197
  • Turrets: +22/-22
    • GrangerHub
Re: Gameplay Preview Phase 1 Results
« Reply #15 on: December 14, 2009, 08:00:48 am »
  • Aliens are winning significantly more games than humans

You'd still have to try very hard to convince me that this isn't due to the fact that new players DO NOT like to play as aliens. Hence, you often end up with a human team full of noobs, and more experienced players on the aliens.

In large public games with a bunch of players, the individual's role is diminished, but in smaller games with more experienced players (clan scrims, etc), all of the alien nerfs make things very un-fun at times (in my opinion).

It says the mean game length is 15 min. That seems a bit low to me but w/e.

As others have stated, and I agree 100% with, the length of 1.1 games is one of the main reason I like Trem. 1.2 games seem to end much faster. In my mind, if a game doesn't make it to sudden death, then one team just isn't trying hard enough, or one is stacked with experienced players.

Finally, please leave the poor granger alone! ;_;

UniqPhoeniX

  • Spam Killer
  • *
  • Posts: 1376
  • Turrets: +66/-32
Re: Gameplay Preview Phase 1 Results
« Reply #16 on: December 14, 2009, 11:02:34 am »
Could make the graphs more readable by depending less on lots of colors, instead using different kinds of lines (thicker for overall, some dotted; or curves might be even better) and symbols (+/x/o). Also what exactly is the vertical value (bias??) and 'mean duration'?.
Graphs for time (H) vs playercount (V) showing stageups and wins would be nice (to see which team gets stage ups sooner at which playercounts), possibly connecting individual points vertically.
I also don't agree with granger spit being made useless, grangers already have a very hard time egg spamming if humans just work together and don't let the aliens spread through the whole map again and again. If they don't work together at all, do they deserve to win?
Maybe increasing flamer speed would be better then damage, as then self damage will not increase. I would like to see flamer more used against bases then aliens. In 1.1 flamers can actually kill some hives safely if you are careful and use the speed addition from movement. As hives now attack all attackers, what about making flamer kill swarms?

your face

  • Community Moderators
  • *
  • Posts: 3843
  • Turrets: +116/-420
Re: Gameplay Preview Phase 1 Results
« Reply #17 on: December 14, 2009, 08:04:43 pm »
Burning cloud of zombie bees.  That would be a sight to remember. :D
spam spam spam, waste waste waste!

CreatureofHell

  • Posts: 2422
  • Turrets: +430/-126
    • Tremtopia
Re: Gameplay Preview Phase 1 Results
« Reply #18 on: December 14, 2009, 08:08:53 pm »
Burning cloud of zombie bees.  That would be a sight to remember. :D

+1
{NoS}StalKer
Quote
<Timbo> posting on the trem forums rarely results in anything good

Repatition

  • Posts: 332
  • Turrets: +11/-196
    • http://jbjarts.blogspot.com/
Re: Gameplay Preview Phase 1 Results
« Reply #19 on: December 14, 2009, 08:09:21 pm »

F50

  • Posts: 740
  • Turrets: +16/-26
Re: Gameplay Preview Phase 1 Results
« Reply #20 on: December 14, 2009, 08:54:22 pm »
I'm not sure there is a bias toward aliens at all. IMO if a game doesn't make it to HS3 vs AS3 there is probably something very wrong with the player balance (or someone just didn't build, or didn't build right). Those games also tend to be very short. If we eliminate games under 10min or so, what kind of balance statistics would we find? I think it is notable that in AS3vsHS3 the euro server found humans to be slightly more effective, and the reverse for the US server.
"Any sufficiently advanced stupidity is indistinguishable from malice." -- Grey's Law


Superpie

  • Spam Killer
  • *
  • Posts: 339
  • Turrets: +105/-48
    • superpie.org
Re: Gameplay Preview Phase 1 Results
« Reply #21 on: December 14, 2009, 09:37:25 pm »
Where is the good in goodbye? -Meredith Willson

Norfenstein

  • Posts: 628
  • Turrets: +81/-78
Re: Gameplay Preview Phase 1 Results
« Reply #22 on: December 15, 2009, 02:10:30 am »
Granger spit slowdown reduced to 1 sec? Granger is too dangerous? You can easilly hit dodging human with granger spit? I don't think so. It's hard and with 1 sec duration it is useless, you can remove it entirely like hovel...
I also don't agree with granger spit being made useless, grangers already have a very hard time egg spamming if humans just work together and don't let the aliens spread through the whole map again and again. If they don't work together at all, do they deserve to win?
If they clearcut the alien base and reduced the entire alien team to grangers, who themselves are only working together in the sense that they all build eggs as soon as any one of them gets an overmind up, then yes, the humans absolutely deserve to win.

...not that I really think changing the spit will help very much with that, but I do think being able to slow a human for five seconds (that's any human; the basilisk gas -- which you have to pay two frags to get -- will slow a naked human for ten seconds, but a battlesuit only two seconds longer than the granger spit does) is very inappropriate for a builder. And at one second I expect it will still be useful against humans trying to rush through an alien base (i.e. when slowing down can mean the difference between grenading the overmind and getting eaten), and for not much else (i.e. when the granger isn't in its base). If you're only imagining one granger dueling with one human then you have the wrong idea.

Maybe increasing flamer speed would be better then damage, as then self damage will not increase. I would like to see flamer more used against bases then aliens. In 1.1 flamers can actually kill some hives safely if you are careful and use the speed addition from movement. As hives now attack all attackers, what about making flamer kill swarms?
The biggest complaint about the flamer seems to be how much it hurts the user, so maybe addressing that even more will be warranted, but I think the flamer's niche is effectively killing lower level aliens. By its very nature it's not as good against structures as other weapons -- it has a short range but doesn't do as much damage as the painsaw, and if you don't position yourself well you are going to hit yourself (more inevitably than against alien players I think; you can hit them while backing up without risk to yourself).

While these results did surprise me, I believe part of the bias towards aliens could be due to people still trying to play trem like its 1.1. ...
... If that's the case, the balance may not be leaned towards aliens as much as it seems from the stats. ...
You'd still have to try very hard to convince me that this isn't due to the fact that new players DO NOT like to play as aliens. Hence, you often end up with a human team full of noobs, and more experienced players on the aliens.
I thought I'd mentioned this in the top post, but I guess I edited it out before posting: there's no shortage of reasons one could give to explain why the statistics are inaccurate, or scenarios one can imagine in which the balance is perfect already, but I think we're better off using the statistics we have instead of the statistics we might imagine for situations that can't be tested. At least this way we can be confident about being balanced in one set of circumstances, instead of just hoping to be balanced in all of them.

I still hold the belief that good players have a better feeling of balance than numbers and statistics ever will, because there are too many independent variables.
And up until now feelings have been the only thing used to balance 1.2 (and no, not just my feelings). Now we're using them to interpret and act on empirical data, where we feel it's beneficial.

Lies, damned lies, and statistics.
Did you have a look at that wikipedia page, or were you just being cynical? It cuts both ways:
Quote
"Lies, damned lies, and statistics" is the persuasive power of numbers, particularly the use of statistics to bolster weak arguments, and the tendency of people to disparage statistics that do not support their positions.

temple

  • Posts: 534
  • Turrets: +37/-42
Re: Gameplay Preview Phase 1 Results
« Reply #23 on: December 15, 2009, 03:37:11 am »
There is never going to be a perfectly balanced game.

However, this goes to show less about the relative power between the teams and more about how pointless the changes have been.  If the trend continues, what is the point of the changes? 

I personally don't enjoy the changes.

FisherP

  • Posts: 295
  • Turrets: +31/-32
Re: Gameplay Preview Phase 1 Results
« Reply #24 on: December 15, 2009, 06:57:24 am »
I'm not sure there is a bias toward aliens at all. IMO if a game doesn't make it to HS3 vs AS3 there is probably something very wrong with the player balance (or someone just didn't build, or didn't build right). Those games also tend to be very short. If we eliminate games under 10min or so, what kind of balance statistics would we find? I think it is notable that in AS3vsHS3 the euro server found humans to be slightly more effective, and the reverse for the US server.

I agree mostly, though I also thing that all the games that go into SD but finish before time is up should also be eliminated since in my experience SD is geared towards an Alien win. First of all the alien structures regenerate, 2nd all aliens really need to keep going is spawns and the om. Humans need much more infrastructure to keep going in SD, the arm, the rc, spawns and a medi. To get the same functional base as the aliens humans need a DC, arm and medi each of which takes BP away from defensive infrastructure. Ergo it is easier to take down the human base (once you get past the opposition) than it is to take down the alien base. Saying all this though I do acknowledge that aliens do seem to die quicker in the human base than before, especially now that the rant has only 350hp.

Imho the whole argument about the good players normally play as aliens and noobs play humans might be true, and if this is what we are truly seeing irl then the balance should take that into account.

Having said that, is there a way you can re-do the figures and look at them with these games filtered out?

David

  • Spam Killer
  • *
  • Posts: 3543
  • Turrets: +249/-273
Re: Gameplay Preview Phase 1 Results
« Reply #25 on: December 15, 2009, 08:18:16 am »
How about you publish the (anonymous) logs so people can make there own graphs etc?
Any maps not in the MG repo?  Email me or come to irc.freenode.net/#mg.
--
My words are mine and mine alone.  I can't speak for anyone else, and there is no one who can speak for me.  If I ever make a post that gives the opinions or positions of other users or groups, then they will be clearly labeled as such.
I'm disappointed that people's past actions have forced me to state what should be obvious.
I am not a dev.  Nothing I say counts for anything.

==Troy==

  • Posts: 440
  • Turrets: +65/-67
Re: Gameplay Preview Phase 1 Results
« Reply #26 on: December 15, 2009, 08:24:14 am »
To get an accurate statistics, force random join on the teams. I.e. when player wants to join, no matter if he chooses H or A, he is "autoplaced". At least that will give you a bit higher spread, but definitely better values.

Granger spit slowing down only for 1 s seems to be a bit too excessive for a change at the moment. (5 times the value). At least 2 seconds would be reasonable, or even 1.5, with the ability to "continue" the slowdown effect on the human. It will still require 100% attention of the granger, but will be a bit more fair to the builders which can not only build.

I mean, as a hckit you could kill a dretch with a blaster with some reasonable effort, as granger, you barely have ANY chance to kill naked rifle even if the rifle is a complete newbie.

SlackerLinux

  • Spam Killer
  • *
  • Posts: 555
  • Turrets: +41/-62
Re: Gameplay Preview Phase 1 Results
« Reply #27 on: December 15, 2009, 12:41:35 pm »
To get an accurate statistics, force random join on the teams. I.e. when player wants to join, no matter if he chooses H or A, he is "autoplaced". At least that will give you a bit higher spread, but definitely better values.

i like this idea for beta could give better results.
Slackware64 13.1
SlackersQVM/

rotacak

  • Posts: 761
  • Turrets: +39/-64
Re: Gameplay Preview Phase 1 Results
« Reply #28 on: December 15, 2009, 04:30:09 pm »
I think that developers too much rely on balance. And that cannot be perfect. Wining ration depending mainly on team. I see many good changes in 1.2, but some of them making me sad. Like luci, hovel, turrets, basi grab turret, granger spit, granger speed, granger size, granger slash, pulse speed... Yes, that changes are probably better for game balance, but Tremulous losing his variability. Player should need to learn how to use basilisk - not fixing basilisk for noobs. Same for pulse rifle. In 1.2 we will lose big part from Tremulous - we cannot kill turret by circling around, because we cant bite turret anymore. We cannot be hurted by turrets like granger or dretch, because we are almost-instant killed. We not need to predict alien movement with luci, because luci is fixed for noobs. We cant be near luci humans with goon and provoke, because we will be instant luci killed. We cant evil laughing when we hit human with granger spit, because 1 sec slowdown is not enough time for laungh. We cant hop in hovel - if we doing it only for fun, now we cant do even that small fun. Pulse rifle was weapon for good players or for base rushes and now is fixed for noobs. Flamer - should be still weapon what can burn yourself if you don't know how to use it. Regular granger have attack, but he should only have fear and run away, not fight.

This is my opinion and I am sad when I see how Tremulous losing his variability only due to balance changes. I think Tremulous should be "learn, practice and play" not "take some weapon and shoot" like quake. But on the other side I am happy when I see bunch of improvements in 1.2 (not balance changes).

Troy: I think autoplacing will cause only join -> leave -> join -> leave.... spam until player will join requested team.

Liskey

  • Posts: 112
  • Turrets: +7/-4
Re: Gameplay Preview Phase 1 Results
« Reply #29 on: December 15, 2009, 11:20:24 pm »
Also what exactly is the vertical value (bias??) and 'mean duration'?

It does need some explanation - obviously the range of game durations is not 14 - 16 minutes (15 +/-1), nor is the vertical axis scaled from -1 = 0 minutes to +1 = timelimit.  WTF?

I agree that measuring balance with players new to 1.2 is better than guessing with no empirical data, but also agree that balance with new players will be significantly different than balance with experienced ones.  The most disconcerting thing for me has been stamina - it is far easier to black out in 1.2 than in 1.1, and impossible to recover.  The alien differences (slow healing without booster) are easier to adjust for, and I love the boosted heal speed :-).