Tremstats: Statistics for tremulous servers

[DASPRiD]

Hi all and good evening (to german people).

The last two days, I figured out some way for a good parsing of the games.log with the help of Ingar. After applying tjw to my server, I am now able to log deconstructions and destructions, which are very important for a tremulous statistic I think. The only problem is, that I cannot differ between normal kills and team kills at the moment, but WolfWings promised me to write a patch, so they will be marked as teamkills in the logfile.

The current status of the Tremstats is very low at the moment. It's version 0.1 and misses still many things in the output, but the parser is full functional. If you want to see it, take a look at the following URL:

http://tremstats.dasprids.de/

You can also play on the server, located in Germany, Frankfurt. It's ping is very good in and around Germany, and maybe also in the USA, test it. As I said, TJW is installed on the server. You may find it in the gamebrowser, called [DB@]DASPRiD's, or simply do a /connect dasprids.de.

If you have any suggestions, what I could implement in the statistics, just post your suggestions here. If they are possible (the log-file limits me), and they are ok, then I will add them to the statistics. But don't post things twice; the following things are still planned:

Overall:

 
• A search for player names
  

For the player details:

 
• Single "weapon" statistics
   
• Decones
   
• Destructions
  

After Tremstats has a good development status, it will be published under the GPL.

[Paradox]

Cool cool.

When can we get this?

[Smokey]

Good start, good idea, good work

[squishy]

Feel free to look at the Boom Boom stats page for some ideas. I'm waiting until GUIDs are implemented in the base install of Tremulous before working on my system anymore.

http://fragify.com/stats.php

[Paradox]

Is this AJAX? because it behaves very fast.

[DASPRiD]

No, it isn't, I hate AJAX. But since its a really good XHTML-code plus nice CSS, good browsers like Firefox don't re-render the full page, but only the changed parts.

[Paradox]

Depends on what you call AJAX. If only one part of the page changes, and it isnt in a frame, its ajax to me.

It isnt the formal, dictionary def, but its still what i call ajax.

For example, on my site http://test.halolivewire.com (halo live wire is a podcast, im developing their site, the subdomain is for testing purposes), i have a chatbox on the left part of the page. The chatbox is just hooked to the sql database tightly, so it refreshes, but doesnt reload the whole page.

[DASPRiD]

I load a full new site on reordering the top 10. Javascript is only used to change window.location.href.

[Undeference]

Gecko doesn't rerender pages instantly unless specifically set to do this. Gecko based browsers appear not to update when reloading pages with unchanged data even when they are not caching. This is nothing to do with AJAX, but with the renderer.

[DASPRiD]

Ok, enough for today. I worked the last 8 hours on the parser and the frontend. Now there are nearly all things parsed. WolfWings wrote the patch, so I can track teamkills now, I only need someone to compile it for me. You can see the current results at the known URL and expect the first release soon.

[robug]

http://tremstats.dasprids.de/

You gotta check this stats package out!!  DASPRiD did a great job!!

I cant wait to install it on yaknet lair!

[DASPRiD]

Well, thx to WolfWings, his patch is working now and I can log all teamkills. I made a switch, so it will also read the logs of nonpatched servers without complaining, but teamkills are logged as normal kills then. I will now add "top teamkillers" next and then put some more information into the player details, then I can release the entire package. It will contain the following things:

• .patch file written by WolfWings for propper logging of teamkills
  
• patched game.qvm for all lazy guys (based on current TJW)
  
• Parser and data-evaluator, written in Python, requires the site-package MySQLdb
  
• Output-system, written in PHP, requires PHP 5.* and MySQL 4.0 or higher. Includes a simple template-system (its called PHP ) for easy changes of the layout.
  

Everything, except WolfWings' patch and the tremulous icons, will be released under GPL Lesser Public Licence, so you are free to modifie anything of it. But I recommend to not do too many changes to the parser itself, because their could be follow some more changes until the final release.

Ok, thats so far, good night and stay tuned for the release.

[Rawr]

ok so does a syop have to install somethign into server to get stats? what do i need to do to see like my name up there?

[DASPRiD]

No, you missunderstood me I think. You only need the patch, if you want teamkills to be logged. The stats still work on every Trem-version, but TJW version is minimum recommed, to log deconnings, and more important, destructions. A guy, who only sits on his turret and spamming lucifers should not have more points than someone, risking his life a cuple of times, which sawing overmind after overmind.

[Rawr]

ok i understand now. im not very good with patches ^.^

[DASPRiD]

Just another thing I considered about, will it do tomorrow:

For always having the levelshot and real names of all maps, the parser will run through the PK3-files, and put map-name and it's levelshot (resized) into the database. I put the image into the database, because it should still work for remote machines. But I won't let them stay there. The output checks for new images in the maps table, and moves the images into real files.


... well, I think, the name `parse_log.py` will not be right any longer

[DASPRiD]

Ok, the single file got too big, just splitted the "parser" into 3 submodules:

tremstats.py - main file
internals.log_parse - parser for log files
internals.data_calc - calculator for MySQL data
internals.pk3_read - reader for PK3 files

Now the requirements are the following:
Python Version 2.4 or 2.5

Plus the following site-packages:
MySQLdb
PIL

------------------------------------------------------------------------

For the less guys waiting for a release:
You will get Tremstats this saturday, promised!

[DASPRiD]

Alright, the PK3 reader is done and supports TGA and JPEG now. Take a look on the output:

http://tremstats.dasprids.de/most_played_maps.php

Now I have to finish only some other little parts, then you will get it.

[durand]

wow, very cool idea. however, i did notice that the graphs aren't right. aliens and humans are always equal..

[DASPRiD]

Uhm, what you mean?  :-?

-Edit-
Oh I see, the legends. Well thx for the info, I correct it

[DASPRiD]

Did I say saturday? I meant today ! You heard right. And beause I've written enough in the readme file, I don't repeat myself here again. So download it now and have fun. Please reports all bugs you find. Suggestions and things like that are also wanted .

http://dev.dasprids.de/downloads/tremstats_0.4.tar.gz

[Paradox]

Sweet. Ill try getting this up for SST.

[DASPRiD]

Damn, there were a little bug in it, which let the parser crash on bad log data. And a directory was missing so the logs didn't get archived. Here your get a fixed version:

http://dev.dasprids.de/downloads/tremstats_0.4.0.2.tar.gz

[DASPRiD]

I just setup a project page for tremstats on dev.dasprids.de, so the download link changed. Take a look:

http://dev.dasprids.de/programs/tremstats/

[DASPRiD]

A note on version 0.5:

It will include my library for real-time server statistics. So you will see in the overview, who is currently playing on your server and which map.

[durand]

nice player features!

[Greudin]

TY Dasprid ! The best tremulous stats ever ! Your code is clean and readable. Separation of parser(python+mysql) and php+css makes hacks easier.

Used there :
http://tremx.bricosoft.com

[Shafe]

agreed!  This is turning out awesome.. Nice work man!

[DASPRiD]

Thx guys, I'm proud of it that this gets really used .

Some other notes on version 0.5:

 
• Integration game factor into total effiiency. The calculation of the game factor will be something like square root of game numbers of player with some other numbers calculated. So a player, whos playing only one or 2 games, but does many kills in this single game, maybe because of noobs on the other side, gets a low efficiency. The more someone plays, the game factor gets less significant, because of the square root. So with 500 games total on a server, it is nearly unimportant, if you have played 400 or 450 games.
   
• Change to the calculation of destruction efficiency. It isn't well calculated yet.
   
• Games with no actions in it get dropped by the parser.
   
• Integration of team statistics for player details. There you can see, in which teams the player mostly wins / get tied games.
   
• As told, real time view of running server.
  

The first two points only require a `tremstats.py --calconly`, and you get the new changes. The 2 points after need a reparsing of the logfiles. If you did not clear the archived_logs directory, you may run `cat ./archived_logs/* > ../games.log && rm ./archived_logs/* && ./tremstats.py` to reparse all logs... Well, I should integrate another argument to reparse all archived_logs, could be useful.
And yeah, the last points only needs to replace the overview template.

Thats so far.

[vcxzet]

GAY
AAA
YAY

good job dasprid