Tremulous Forum
Mods => Mod Releases => Tools and Utilities => Topic started by: DASPRiD on November 24, 2006, 10:24:00 am
-
Heyas,
I know, my plans were different, but there were many little nasty bugs in the parser, which couldn't wait 3 more weeks. So this evening (say, 21 UTC), you will get the newest version of Tremstats.
Here's a list of all improvements:
- Fixed bug #1 (http://bugzilla.dasprids.de/show_bug.cgi?id=1)
- Fixed bug #2 (http://bugzilla.dasprids.de/show_bug.cgi?id=2)
- Fixed bug #3 (http://bugzilla.dasprids.de/show_bug.cgi?id=3)
- Player search moved to top of the page
- All columns in the output are sortable now
- Integrated game-time factor for total efficiency
I think with this you'll get really nice stats now, which are also all right. The update from 0.4.* requires an entire re-parsing of the logfiles. Also, the database structure has changed a little, so you have to rebuild it.
If you find any bugs in it, please don't make the mistake to post them in the comments. Better use the Bugtracker for this, you can find it on http://bugzilla.dasprids.de. There you may also state feature request.
The demo of Tremstats 0.5 can be seen on http://tremstats.dasprids.de/
Download it here:
http://dev.dasprids.de/programs/tremstats/download/
-
*Lava Croft huggles DASPRID
-
Nice work! Nice Nice! lol
But you should exclude all "UnnamedPlayer"s...
Most Active Player UnnamedPlayer (Games: 252)
...
But like i said, NICE! :)
-
thank you, I can farm my stats again! at last it counts everything :)
the only thing to make my a super happy panda now is tremstats on Aliens Wrath :)
-
would be cool if u could make the trem stats world- wide for all servers? lol
-
would be cool if u could make the trem stats world- wide for all servers? lol
I can't.
-
It's done, get it while it's hot!
-
would be cool if u could make the trem stats world- wide for all servers? lol
I can't.
but someone else could start a thing, If all the server admins gave them access to the logs....
-
Wow, TremStats got a nice visual overhaul since I last looked at it. Very nice!
-
would be cool if u could make the trem stats world- wide for all servers? lol
I can't.
but someone else could start a thing, If all the server admins gave them access to the logs....
What admins would never do ;>
-
would be cool if u could make the trem stats world- wide for all servers? lol
I can't.
but someone else could start a thing, If all the server admins gave them access to the logs....
What admins would never do ;>
why? what is there in a log that is so secretive?
all it would need is a daily cron job to upload the script to some server somewhere.
-
Important note for all server admins:
To get propper results, you have to turn of unbuffered logging. So add the following line to your server.cfg:
set g_logsync 0
-
Uhm, just found another bad bug with the parser and copying the logfiles. I will write a patched version tomorrow.
-
Tremstats 0.5.1 out.
-
still http://tremstats.dasprids.de/player_details.php?player_id=77 shows peak kill count in the 40s, and Im sure you remember yesterdays game where I had 80 kills
-
Sure, but the logs of that are corrupted thanks to the bug. So I can't count them anymore, sorry.
-
would be cool if u could make the trem stats world- wide for all servers? lol
I can't.
but someone else could start a thing, If all the server admins gave them access to the logs....
What admins would never do ;>
why? what is there in a log that is so secretive?
all it would need is a daily cron job to upload the script to some server somewhere.
For that big server logs parser, I can provide about 150 MB of logs, filled with insults, tking but nice things also :8)
-
am I stupid or do you have a nice field called `building_icon`containing file name in buildings table, and then in maps table you retarted'ly (:P) store a FOKIN PNG image inside a database????? btw my mysql just throws out on that part and refuses to import.
-
... btw my mysql just throws out on that part and refuses to import.
Got the same pb, remove INSERT into maps lines to import data.sql without errors.
Perform a ./tremstats.py --pk3only to refill the maps table.
Ty DASPRiD for this new version and for help to upgrade it :)
http://tremx.bricosoft.com
-
am I stupid or do you have a nice field called `building_icon`containing file name in buildings table, and then in maps table you retarted'ly (:P) store a FOKIN PNG image inside a database????? btw my mysql just throws out on that part and refuses to import.
Thats because the levelshots have to be getable without having parser and webserver on the same server.
Btw, the SQL-bug is fixed in SVN version.
-
Any one have stats on this parser - How long does it take to parse a large log file (>100Mb)?
Just curious if it's worth using at this point.
-
This depends on your machine. Parsing a 300 MB logfiles takes my server 5 minutes.
-
Running into some errors testing the tool. I haven't looked at your code, but I assume you are trying to move the original file instead of just copying it? To my knowledge, this won't work on Windows unless the server is shutdown.
Renaming original logfile ...
Traceback (most recent call last):
File "D:\temp\trem\logger\tremstats.py", line 142, in <module>
app.Main()
File "D:\temp\trem\logger\tremstats.py", line 63, in Main
parser.Main(self.dbc, self.Check_map_in_database, self.games_log, self.stati
c_log)
File "D:\temp\trem\logger\internals\log_parse.py", line 49, in Main
self.Log_read()
File "D:\temp\trem\logger\internals\log_parse.py", line 106, in Log_read
os.rename(self.games_log, source)
WindowsError: [Error 13] The process cannot access the file because it is being
used by another process
-
True, I'm renaming it, because the pointer of the application should move with to a temporary position. But who is really running a dedicated server with windows os?
-
Just found your static variable. Might be worth putting that in your config file with a bit of instruction.
//nevermind. this still doesn't fix the issue.
-
But who is really running a dedicated server with windows os?
To my knowledge, quite a few of us!
-
Made a quick mod to your code to get things working. Instead of moving/copying the log all over the place, I just set things to copy the log to an archive directory. The code then sifts through the "last version" and compares it with the current copy. This ensures that we don't process the same data twice. Once done, it turns the current copy into the "last version".
This is the only solution I can think of for a Windows setup - without shutting down the Trem server to release the log that is.
This is how my current stats solution works, and I figured it'd work here while I'm testing your stats tool.
-
Well, and how do you ensure, that the parser gets only logfiles beginning with an "init" and ending with a "shutdown"? Or how do you ensure, that you don't copy the file while Tremulous is writing to it?
-
I'm not sure I see your solution for such issues. Unless I'm overlooking it.
-
The renaming is the solution ;). Because on Linux the file-pointer is moved. Read the 3rd news on dev.dasprids.de for explanition.
-
Thanks, DASPRiD. That's helpful, but let me see if I get this straight.
You are renaming the games.log to games.log.tremstats.tmp. Tremulous will continue to write to this TMP file until a game completes. Tremulous then creates a new games.log. Now TremStats will process the tmp file?
I assume this is a non-Windows tremded feature.
The only option I could work out in Windows would be to add a check for the last instance of ShutdownGame, and ignore the rest until next time around.
-
You got it, that's the way I work :)
By the way, if you got a running workaround for windows, I may implement it in the svn-version. Just send me the patch.
If you do so, please checkout the latest version from SVN:
http://svn.dasprids.de/repos/tremstats/trunk/
-
Finally got things working. First I copy the log to the archive directory. Then I strip unwanted ending lines from the log (everything after the last ShutdownGame) - this removes unfinished games. Then we compare the file with the "last version" and process new games.
I'll try to work my method into the latest SVN and post for your review.
May need to create a variable in config for the user to tell whether they are using a Windows server or not.
-
No need to:
import sys
if sys.platform == 'win32':
print "I am windoze."
else:
print "I am some good OS."
And btw, wht exacly do you do with the stripped line? Do you drop them?
-
They are dropped. They would get picked up the next time around.
-
Ah ok, then it's fine.
-
Just pulled down the SVN and input my changes. Here is a ZIP file including the changes and a small changelog noting the lines I updated.
Feel free to contact me via ICQ or PM to discuss...
//**URL REMOVED**//
//I also added a tally of "says" for the players table, so I could display Most Talkative on my stats page. This change is represented in structure.sql and data_calc.py
-
Alright, I'm going to contact you per ICQ.
-
I think that "time factor" is wrong and should be abandoned. Its a buggy patch over a bug. It makes average server campers look better than good players with real life :P.
Better way is to sort ppl according to the kill effeciency, and then (multiply kill effeciency of ppl player killed[or number from the sort above] x times he killed them)/ total kills, and then sort on this number. ppl with 2-5 games played usually have low chance to meet good players, and if they do they dont make >10 kill effeciency :), and if they DO then they are really good (they managed 10 kill effeciency against other good players).
That would factor out the time spend on the server, and really compare players against each other. Killing feeder n00bs would lose importance.
Of course it would be even better if the parser would store class of killed player (now it only stores class/weapon of the attacker), then we could also factor tyrrant pwning naked rifles and dretch killing BSes. Imo it should be in the next version.
PS1:
and just to back up with a strong argument :P look at
29 5555555555555555555555555]
Game-time factor 0.01
Kills 20
Total Deaths 2
Kill Efficiency 6.67
34 Xonya[FI]
Game-time factor 0.04
Kills 395
Total Deaths 228
Kill Efficiency 2.47
I mean LOL, Xonya is a good player, she kills me a lot, and there we have some lamer who happened to play twice pwning n00bs left and right (maybe he joined at S3 and ran arround with camped pulse/lucy after dretches?) 5 places higher than her ... cmon, the main table MUST be relative, each player compared with every other equally. Its not fair to just factor time spend on a server and act like its all good in the hood.
PS2: and no, im not ranting because Oli is about to take first place because he camped bigger time factor than me :P :] hehe. Actually I didnt even tried my formula yet on the dataset you send me and dont know where I will end up :/
-
So you want a player with one 1 game played and 100 kills / 0 death against noobs to be on first place? :roll:
-
Make it (kills-deaths)/time or whatever, and then do <25 games = not on the list.
-
//NEVERMIND!
-
One negative I've found is that player_games_played includes spec-only connections.
It doesn't. If you spec, you get no kills/deaths/teamkills. If you have zero of them overal, you don't get plus one player_games_played.
Make it (kills-deaths)/time or whatever, and then do <25 games = not on the list.
Thats nearly the way I do it, but it's more complex.
-
I just saw the line in the source. Sorry!
-
So you want a player with one 1 game played and 100 kills / 0 death against noobs to be on first place? :roll:
OH F... I wrote A4 poem about it and fokin browser refreshed the page, oh CRAP !!!
anyway
(multiply kill effeciency of ppl player killed x times he killed them)/ total kills
so for your player, lets say he killed 100 times this n00b (sorry Tric :P)
http://tremstats.dasprids.de/player_details.php?player_id=2161
0.86 x 100/100 = 0.86, he is as "good" as the ppl he killed. Now I see that my equation is missing the second part
(multiply kill effeciency of ppl that killed player x times they killed him)/ total deaths
so we have to ignore ppl with ~<10 deaths to not skew the statistic, or ~<10 deaths and above 100 kills.
then we can do (multiply Destruction Efficiency of ppl player killed x times he killed them)/ total kills * player Destruction Efficiency .. or not :P
first both numbers are better while bigger, we could multiply them with each other or do some other fancy stuff with them, but essentially I want THOSE numbers to decide, not some artificial "I play here every day so I pwn" factor :)
Its kinda hard to avoid killing noobs, so its highly unlikly that someone will thrive selectively killing only good players :), on the other hand avoiding getting killed is a skill no matter who tries to kill you :)
hmmm hmm, ok, good players playing only with good players, this can be a problem. It will make ppl playing with n00bs look bad. There must be something smart, that has nothing to do with how much you camped the server :)
hmmm maybe we should compute kill efficiency individually for every map played, and THAT number would multiply against
(multiply kill effeciency of ppl player killed x times he killed them)/ total kills on that particular map
Oh cmon, work with me, throw me a bone here.
ps: one PAINFULL thing is lack of parsing into the database the class/weapon of the player that died (I know it needs patching to have that info). Its one thing to dretch a BS and die 3 times in the process, its the other to tyrr 30 naked rifles. Same goes the other way arround, one thing to die by the tyrr, the other one to get dretched while armored/hatted/shotgunned/and nade loaded :)
-
Btw, you totaly forgot the destruction efficiency. Tremulous is not about just killing everyone. Anyway, the game time factor has just another effect: If you stop playing on the server, your efficiency gets down and you get out of the top-stats.
But we could modify the algorithm so regular players don't see a big difference in their gtf.
-
Btw, you totaly forgot the destruction efficiency. Tremulous is not about just killing everyone.
no I didnt :P
>then we can do (multiply Destruction Efficiency of ppl player killed x
>times he killed them)/ total kills * player Destruction Efficiency .. or not
But we could modify the algorithm so regular players don't see a big difference in their gtf.
Could you at least make those
(multiply kill effeciency of ppl player killed x times he killed them)/ total kills
(multiply kill effeciency of ppl that killed player x times they killed him)/ total deaths
a part of the equation? :P
-
ah, to compare them with eachother... hmmm, sounds like a good addition, i'll test it if i find some time. btw, let's not use the forum as chat, contact me over icq, jabber or irc.
-
soo wazzup with Tremstats v0.6.0?
what is a Games paused?
anyway I like it (since Oli is no longer a threat :P)
-
does it list how many tyrants you killed?
-
rasz_pl:
Games paused is how many games you have paused. There are 2 treshold variables in the config now: games_played and games_paused. The first indicates, with how many games min played you get in the list. The standard for this is 25. the games_paused treshold means, how many games you can max have paused before you get out of the list, standard is 500. If you start playing again, you get immediently back in the list with your old stats.
Also, I found a good use for the Game-time factor: It's used for the most active players, instead of just the games played.
vcxzet:
It's not listed yet, what exacly you killed how often, but this will follow.
-
rasz_pl:
Games paused is how many games you have paused. There are 2 treshold variables in the config now: games_played and games_paused. The first indicates, with how many games min played you get in the list. The standard for this is 25. the games_paused treshold means, how many games you can max have paused before you get out of the list, standard is 500. If you start playing again, you get immediently back in the list with your old stats.
Also, I found a good use for the Game-time factor: It's used for the most active players, instead of just the games played.
vcxzet:
It's not listed yet, what exacly you killed how often, but this will follow.
1 so if i pause for >xx games i get dropped back? sounds ok
2 Queen of scotland landed higher than me, so we need to tweak the algo once again :]
>ps: one PAINFULL thing is lack of parsing into the database the class/
>weapon of the player that died (I know it needs patching to have that >info). Its one thing to dretch a BS and die 3 times in the process, its the
>other to tyrr 30 naked rifles. Same goes the other way arround, one
>thing to die by the tyrr, the other one to get dretched while armored/
>hatted/shotgunned/and nade loaded
MDing poor dretches with 500-600 credits worth of equipment should not be equal to dretching 500-600 cred humans :)
if we could only parse the info about class/equipment of the killer and the killed person, we could come up with some weight system. For example the percentage ratio between alien class HP and dps vs human hp * armory class(can be tricky, different armory areas/hit zones) and weapon dps.
-
And can't repeat often enough, there are no hp's logged, and I can't base the entire kill efficiency on something non-standard. Only with the Client_TeamClassPatch I firstly always know, who's playing which class. Before, I only know, whom you killed with which weapon.
-
And can't repeat often enough, there are no hp's logged
>I know it needs patching to have that info
-
Right, and not everyone want's to patch his server, so I can't base the calculation on this.
-
Right, and not everyone want's to patch his server, so I can't base the calculation on this.
that could be an option for anal numbern00bs like me :) just add 3 more fields to the KILL table and let admins use normal logs (ignoring those fields) or pathed (using them)
or not, just throwing ideas out loud
edit: actually you can partially kinda almost guess what was the class/weapon of the killed person, just make the parser ass_ume that after the last person the killed one killed he didnt evo/buy, and write to the database his last known evo/weapon :)