Author Topic: Tremstats reloaded  (Read 17145 times)

codernem

  • Posts: 73
  • Turrets: +4/-0
Tremstats reloaded
« on: May 23, 2008, 08:18:31 am »
Hi all!  :)

I want to show you a new tremstats engine output. I wrote it because our italian community site has little DB space, and Dasprid's tremstats use it extensively. So I developed this parser/stats calculator (written in Java). The processing time of a 380 Mb log file is rather quick (about 5 minutes...), and it exports all data in plain HTML format, without the need of DB space. It needs about 200 Mb of memory to store data during parsing, while the output (for about 410 players) is 11 Mb large (without map images and icons).

At the moment I'm rewriting some pieces of code, and refactoring, and after that I'll release it as GPL and put a google-code repository.
I want to really thanks Dasprid for let me use his icons for weapons and buildings.

Main features:

- no DB required
- advanced stats (expected value and variance)
- GUID aggregator by config
- quick processing time

What I'm planning:

- xml export of processed data
- charts generator
- more statistics
- map image auto-extractions (given a map repository dir)

You can see it here:

http://www.capponcino.it/tremulous/statsprolinux/index.html

Feel free to send me suggestions  :-* and critics  >:(

PS: there is a bug on the index page, if opened by IE. I've already address this issue  :angel:
« Last Edit: May 23, 2008, 08:23:34 am by codernem »

doomagent13

  • Posts: 506
  • Turrets: +18/-18
Re: Tremstats reloaded
« Reply #1 on: May 23, 2008, 04:18:42 pm »
I suggest that you have it run once a day and SAVE the results.  Then you could just show the results upon request rather than needing to recalculate them EVERY time someone wants to look at them.

NiTRoX

  • Posts: 1453
  • Turrets: +41/-200
Re: Tremstats reloaded
« Reply #2 on: May 23, 2008, 05:39:40 pm »
Seems cool.

I like the website layout and stuff.

codernem

  • Posts: 73
  • Turrets: +4/-0
Re: Tremstats reloaded
« Reply #3 on: May 23, 2008, 08:54:41 pm »
I suggest that you have it run once a day and SAVE the results.  Then you could just show the results upon request rather than needing to recalculate them EVERY time someone wants to look at them.

It's what I do :P

Lakitu7

  • Tremulous Developers
  • *
  • Posts: 1002
  • Turrets: +120/-73
Re: Tremstats reloaded
« Reply #4 on: May 24, 2008, 07:26:01 am »
Cool to see someone adding to the dormant stats code.

I know the european folks don't use it much, but I've added a lot of additional things to games.log in my qvm and it'd be pretty cool to see someone use them to tabulate new stats / improve existing ones. Particularly, I'd like to see stats on the number of games that end before/during SD, and also stats on vote pass/fail rates + most kicked user etc. Just a thought. If I were you I probably wouldn't do it either though. Someday, perhaps. :)

Hellrider

  • Posts: 61
  • Turrets: +5/-2
    • My personal website
Re: Tremstats reloaded
« Reply #5 on: May 24, 2008, 07:31:09 am »
Good job codernem  ;) ;D

Lava Croft

  • Guest
Re: Tremstats reloaded
« Reply #6 on: May 24, 2008, 09:17:21 am »
Good work Codernem! <3

codernem

  • Posts: 73
  • Turrets: +4/-0
Re: Tremstats reloaded
« Reply #7 on: May 24, 2008, 03:22:58 pm »
Cool to see someone adding to the dormant stats code.

I know the european folks don't use it much, but I've added a lot of additional things to games.log in my qvm and it'd be pretty cool to see someone use them to tabulate new stats / improve existing ones. Particularly, I'd like to see stats on the number of games that end before/during SD, and also stats on vote pass/fail rates + most kicked user etc. Just a thought. If I were you I probably wouldn't do it either though. Someday, perhaps. :)

If you print right log informations, it's a matter of minutes extends the system to generate events from new log entries. I'm putting efforts on writing easily extendible and clean code.

I guarantee nothing, but I'll look at your log messages, after the refactor process I'm doing.

Thanks a lot for this suggestion!

wannabe

  • Posts: 44
  • Turrets: +2/-18
Re: Tremstats reloaded
« Reply #8 on: May 25, 2008, 12:48:19 am »
wd (well done)
source?

Oblivion

  • Posts: 410
  • Turrets: +27/-22
    • Gamez-Host.com
Re: Tremstats reloaded
« Reply #9 on: May 25, 2008, 03:07:24 am »
codernem have you released it yet? If not are you going to?
World Wide Server Host

Lava Croft

  • Guest
Re: Tremstats reloaded
« Reply #10 on: May 25, 2008, 09:53:20 am »
wd (well done)
source?
Who cares about the source.

ghostisback

  • Posts: 139
  • Turrets: +8/-1
Re: Tremstats reloaded
« Reply #11 on: May 25, 2008, 01:32:12 pm »
Good Work Mate !!! xD

wannabe

  • Posts: 44
  • Turrets: +2/-18
Re: Tremstats reloaded
« Reply #12 on: May 25, 2008, 05:44:57 pm »
wd (well done)
source?
Who cares about the source.

lava.. do you have fans? I dont think so.. they just dont want to argue with you... :)

Bajsefar

  • Posts: 597
  • Turrets: +49/-39
Re: Tremstats reloaded
« Reply #13 on: May 25, 2008, 06:17:26 pm »
Wannabe, he has. And I have argued with him more than once.. Still a fan. :)

Paradox

  • Posts: 2612
  • Turrets: +253/-250
    • Paradox Designs
Re: Tremstats reloaded
« Reply #14 on: May 25, 2008, 09:17:31 pm »
Looks promising, but the original design is very clean. Perhaps you could use it along with this new system.

Also, its slower than my old blog.

∧OMG ENTROPY∧

khalsa

  • Administrator
  • Posts: 597
  • Turrets: +187/-132
    • http://www.mercenariesguild.net
Re: Tremstats reloaded
« Reply #15 on: May 25, 2008, 09:29:47 pm »
Looks very promising, can't wait to see a release.


Khalsa
}MG{ Mercenariesguild
ਮਨੁ ਜੀਤੇ ਜਗੁ ਜੀਤਿਆ

codernem

  • Posts: 73
  • Turrets: +4/-0
Re: Tremstats reloaded
« Reply #16 on: May 26, 2008, 08:24:57 am »
source?

As I said before, I'm going to release source code after cleaning it up. I don't want to release hard-to-understand code. But, yes, sources will be released for sure!

Quote
Looks promising, but the original design is very clean. Perhaps you could use it along with this new system.

Sure, Dasprid design is superb. I need to talk to him about this :)


@ all: thanks for appreciation and suggestions :-*

NiTRoX

  • Posts: 1453
  • Turrets: +41/-200
Re: Tremstats reloaded
« Reply #17 on: May 26, 2008, 10:52:36 am »
source?

As I said before, I'm going to release source code after cleaning it up. I don't want to release hard-to-understand code. But, yes, sources will be released for sure!

Quote
Looks promising, but the original design is very clean. Perhaps you could use it along with this new system.

Sure, Dasprid design is superb. I need to talk to him about this :)


@ all: thanks for appreciation and suggestions :-*

His was written in Python.

codernem

  • Posts: 73
  • Turrets: +4/-0
Re: Tremstats reloaded
« Reply #18 on: May 26, 2008, 11:07:25 am »
source?

As I said before, I'm going to release source code after cleaning it up. I don't want to release hard-to-understand code. But, yes, sources will be released for sure!

Quote
Looks promising, but the original design is very clean. Perhaps you could use it along with this new system.

Sure, Dasprid design is superb. I need to talk to him about this :)


@ all: thanks for appreciation and suggestions :-*

His was written in Python.

I know but, over all, Dasprid's stats need a DB, while my exporter doesn't. This is the main difference between the two. Maybe what Paradox would say is to use the same layout and style for the web part.
« Last Edit: May 26, 2008, 11:18:11 am by codernem »

Paradox

  • Posts: 2612
  • Turrets: +253/-250
    • Paradox Designs
Re: Tremstats reloaded
« Reply #19 on: May 26, 2008, 09:26:48 pm »
Yes, thats what i mean. Same front end, but the backend would be totally different.

∧OMG ENTROPY∧

doomagent13

  • Posts: 506
  • Turrets: +18/-18
Re: Tremstats reloaded
« Reply #20 on: May 27, 2008, 02:26:10 am »
And please make it not need to re-parse/compile/whatever the log every time someone tries to check it.

codernem

  • Posts: 73
  • Turrets: +4/-0
Re: Tremstats reloaded
« Reply #21 on: May 27, 2008, 08:40:53 am »
And please make it not need to re-parse/compile/whatever the log every time someone tries to check it.

As I said before (two times) my engine DOES NOT PARSE log files every time a user see the website. Parsing is made ONCE, and after that, the engine output STATIC html pages, with all infos.

Maybe many of you think that because of the progress bar that loops and loops on the first page. It's only a graphical effect (buggy on IE, but already solved in development), introduced because of players indexing is done with a JS file (that is quite big... about 1 Mb). If that gif creates so much problems, I can simply delete it. You can skip it by pressing Player Index on top menu. Try to see the main page with firefox ;)

Again, the system DOES NOT REPARSE log files at every access :P

doomagent13

  • Posts: 506
  • Turrets: +18/-18
Re: Tremstats reloaded
« Reply #22 on: May 27, 2008, 12:13:31 pm »
Okay.  Just had to make sure...

BTW, it looks quite good.

David

  • Spam Killer
  • *
  • Posts: 3543
  • Turrets: +249/-273
Re: Tremstats reloaded
« Reply #23 on: May 27, 2008, 03:04:28 pm »
Maybe many of you think that because of the progress bar that loops and loops on the first page. It's only a graphical effect (buggy on IE, but already solved in development), introduced because of players indexing is done with a JS file (that is quite big... about 1 Mb). If that gif creates so much problems, I can simply delete it. You can skip it by pressing Player Index on top menu. Try to see the main page with firefox ;)

1.3MB of javascript... :o
Have you thought about ajax?  Would drop the downloaded amount a fair chunk.
Any maps not in the MG repo?  Email me or come to irc.freenode.net/#mg.
--
My words are mine and mine alone.  I can't speak for anyone else, and there is no one who can speak for me.  If I ever make a post that gives the opinions or positions of other users or groups, then they will be clearly labeled as such.
I'm disappointed that people's past actions have forced me to state what should be obvious.
I am not a dev.  Nothing I say counts for anything.

codernem

  • Posts: 73
  • Turrets: +4/-0
Re: Tremstats reloaded
« Reply #24 on: May 27, 2008, 03:43:48 pm »
Maybe many of you think that because of the progress bar that loops and loops on the first page. It's only a graphical effect (buggy on IE, but already solved in development), introduced because of players indexing is done with a JS file (that is quite big... about 1 Mb). If that gif creates so much problems, I can simply delete it. You can skip it by pressing Player Index on top menu. Try to see the main page with firefox ;)

1.3MB of javascript... :o
Have you thought about ajax?  Would drop the downloaded amount a fair chunk.

Sure, but I dont know ajax technology atm. In future reworking (after the release of source code), why not? :P

Colkadome

  • Posts: 25
  • Turrets: +2/-2
Re: Tremstats reloaded
« Reply #25 on: May 28, 2008, 12:55:13 am »
How many people are currently not on the list? I didn't see my name  :(

codernem

  • Posts: 73
  • Turrets: +4/-0
Re: Tremstats reloaded
« Reply #26 on: May 28, 2008, 07:42:38 am »
How many people are currently not on the list? I didn't see my name  :(

You can configure the engine to export only players with a minimal presence (in Prolinux case, at least 20 games). Maybe this is why you can't see your name on the list.

slux_ITA

  • Posts: 38
  • Turrets: +0/-0
    • http://www.capponcino.it/alessio
Re: Tremstats reloaded
« Reply #27 on: May 28, 2008, 11:40:56 pm »
good job brother!  ;D

codernem

  • Posts: 73
  • Turrets: +4/-0
Re: Tremstats reloaded
« Reply #28 on: May 31, 2008, 10:16:40 am »
I've done a style cleanup, using some css from Dasprid's stats.

http://www.capponcino.it/tremulous/statsprolinux/index.html

Cheers :)

NiTRoX

  • Posts: 1453
  • Turrets: +41/-200
Re: Tremstats reloaded
« Reply #29 on: May 31, 2008, 04:08:20 pm »
Just my opinion.

Why don't you make the navigation bar buttons of equal length/height and try to space them centrally over the black boxy thing?