Tremulous Forum

General => General Discussion => Topic started by: Somethief on August 14, 2006, 08:28:57 pm

Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 14, 2006, 08:28:57 pm
This is a very usefull application to keep your map folder up-to-date.
No more painfull surfing around the net or slow downloads from a server, the application keeps your maps up to date without your attention.

After that adspeak.. Really
You can get the application here (http://tremmap.omgwtf.nl/tremmap-setup.exe)

Notice: The application has to be in the same directory with tremulous.exe so it can find your map folder, otherwise it wont work.
If your maps are not in base\ folder, you may change this from settings.

Our IRC chan is #tremmapdownload on qnet
Title: Automatic Tremulous Map Downloader
Post by: epik on August 14, 2006, 08:30:04 pm
I was first!
Title: Automatic Tremulous Map Downloader
Post by: Smokey on August 14, 2006, 08:30:18 pm
why would anyone trust you when you cant even spell your correctly?..
Title: Automatic Tremulous Map Downloader
Post by: Karvajalka on August 14, 2006, 08:30:56 pm
I tried it, it works.  :D. I actualy like it a lot.
Title: Automatic Tremulous Map Downloader
Post by: epik on August 14, 2006, 08:32:03 pm
I can confirm it's virus-free.
Title: Automatic Tremulous Map Downloader
Post by: Smokey on August 14, 2006, 08:32:59 pm
How does it know when there is a new map?
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 14, 2006, 08:34:01 pm
Quote from: "Smokey"
why would anyone trust you when you cant even spell your correctly?..


1. I already fixed it
2. After trying it out u wont think the same (backup ur trem folder first if u think its not safe :x)
Title: Automatic Tremulous Map Downloader
Post by: Smokey on August 14, 2006, 08:34:41 pm
i tried it out. works, but why a php file for a map list? when not .ini or .cfg? or even a .txt?
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 14, 2006, 08:34:49 pm
Quote from: "Smokey"
How does it know when there is a new map?


Isn't it my problem ?
Title: Automatic Tremulous Map Downloader
Post by: Smokey on August 14, 2006, 08:35:56 pm
Im hardly worried about viruses/keyloggers or any other sort, payback would be a bitch. Not bad.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 14, 2006, 08:36:23 pm
Quote from: "Smokey"
i tried it out. works, but why a php file for a map list? when not .ini or .cfg? or even a .txt?


PHP is server-side generated so it can take care of itself. .txt etc would need human touch
Title: Automatic Tremulous Map Downloader
Post by: whitebear on August 14, 2006, 08:36:29 pm
so how does it actualy work... details please...

Does it download the map from game server? Your own server?
Is it "in-game" aplication? Where is the download server? How often does the files update on the server?
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 14, 2006, 08:39:19 pm
Quote from: "whitebear"
so how does it actualy work... details please...

Does it download the map from game server? Your own server?
Is it "in-game" aplication? Where is the download server? How often does the files update on the server?


So it simply does this:
1. Downloads .php file from my server (server builds the file)
2. It checks does map folder include files that it got in that php file
3. If it finds new map, it downloads it from my server in .pk3, when designing the application i saw .zip useless cause the save of it is so minimal with map files.
Title: Automatic Tremulous Map Downloader
Post by: Mikiupdown2 on August 14, 2006, 08:47:10 pm
Quote from: "Somethief"
3. If it finds new map, it downloads it from my server in .pk3, when designing the application i saw .zip useless cause the save of it is so minimal with map files.


Uhm...PK3 files are zip files. Just renamed.

But yea, kool program. Will come in handy. Very handy indeed.

EDIT: Haha Moofed, beat ya. Ner ner ne ner ner!  :roll:
Title: Automatic Tremulous Map Downloader
Post by: Moofed on August 14, 2006, 08:47:32 pm
Quote from: "Somethief"

3. If it finds new map, it downloads it from my server in .pk3, when designing the application i saw .zip useless cause the save of it is so minimal with map files.


Psst... pk3 files are zip files.  :wink:

edit: :eek:
Title: Automatic Tremulous Map Downloader
Post by: Rippy on August 14, 2006, 08:47:41 pm
As a programmer, I'm interested in:
1. What language is it programmed in?
2. Can I have the source?

Not out of suspicion of any kind, and it's not like I'm gonna rip the code, I'd just like to see the code.
Title: Automatic Tremulous Map Downloader
Post by: [db@]Megabite on August 14, 2006, 09:08:55 pm
I bet 10€ on Delphi! Erm, sorry... Object Pascal... Erm no... "The Delphi Language"! That's it! ;)

Danny
Title: Automatic Tremulous Map Downloader
Post by: David on August 14, 2006, 09:56:33 pm
can we have the source? then we can get a linux verson.

edit:
shouldnt be *too* hard to make a perl verson, i would but i have too many projcts on the go :(
Title: Windows-get
Post by: Chernevog on August 14, 2006, 10:37:27 pm
Looks like a similar program to Windows get a source forge project. That one is coded in delphi and the source is available here. All you would have to do is create a repository of map files and then winget would check to see if there are new ones or not and automatically download them.

http://windows-get.sourceforge.net/

I wonder if we can get tucows or someone to host a tremulous map repository that we could connect to so that we all knew and could trust the source. (Not saying the current source isn't trust worthy but I am uber paranoid)

Cheers,
Chernevog
Title: Automatic Tremulous Map Downloader
Post by: Odin on August 14, 2006, 11:52:03 pm
Wouldn't it be much better if it worked like apt?
Title: Automatic Tremulous Map Downloader
Post by: David on August 15, 2006, 12:14:46 am
Quote from: "Odin"
Wouldn't it be much better if it worked like apt?

lets submit a tremulous-maps-beta package!
then someone has to keep it up to date, and all is sorted!.
Title: Automatic Tremulous Map Downloader
Post by: vcxzet on August 15, 2006, 12:37:37 am
you virused my system an haxored my harddisk
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 15, 2006, 01:02:02 am
Quote from: "David"
Quote from: "Odin"
Wouldn't it be much better if it worked like apt?

lets submit a tremulous-maps-beta package!
then someone has to keep it up to date, and all is sorted!.
that would be so great, only problem is that some people use distros with a package managment system other than apt (and there is no good way to force them to). i would love to point sources.lst at tremulous.info and have all the latest maps
Title: Automatic Tremulous Map Downloader
Post by: David on August 15, 2006, 01:17:11 am
Quote from: "kevlarman"
Quote from: "David"
Quote from: "Odin"
Wouldn't it be much better if it worked like apt?

lets submit a tremulous-maps-beta package!
then someone has to keep it up to date, and all is sorted!.
that would be so great, only problem is that some people use distros with a package managment system other than apt (and there is no good way to force them to). i would love to point sources.lst at tremulous.info and have all the latest maps


thay can make a rpm then.

--OR--
tremulous.info can add a file with a list of each map and a url and its md5, and a simple bash script can scank them all.
or a public ftp folder, and use wget's mirroring stuff.

also, how is your program automatic? windoze has no cron, and its verson of at sucks.
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 15, 2006, 01:55:23 am
Quote from: "David"
Quote from: "kevlarman"
Quote from: "David"
Quote from: "Odin"
Wouldn't it be much better if it worked like apt?

lets submit a tremulous-maps-beta package!
then someone has to keep it up to date, and all is sorted!.
that would be so great, only problem is that some people use distros with a package managment system other than apt (and there is no good way to force them to). i would love to point sources.lst at tremulous.info and have all the latest maps


thay can make a rpm then.

--OR--
tremulous.info can add a file with a list of each map and a url and its md5, and a simple bash script can scank them all.
or a public ftp folder, and use wget's mirroring stuff.

also, how is your program automatic? windoze has no cron, and its verson of at sucks.
or rsync, but none of those are nearly as convenient as apt (well, maybe rpm and whatever their version of apt is called (yum?) is, but that is beside the point)
Title: Automatic Tremulous Map Downloader
Post by: Lava Croft on August 15, 2006, 08:27:25 am
Good job Somethief, I think you made a lot of lazy people really happy:)
Title: Automatic Tremulous Map Downloader
Post by: SLAVE|Mietz on August 15, 2006, 08:33:59 am
good job on that prog.

and i still wonder how many extremely PARANOIC ppl are out there....
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 15, 2006, 11:22:21 am
Quote from: "Lava Croft"
Good job Somethief, I think you made a lot of lazy people really happy:)


It not only for the lazy people, if these paranoids will take the application in use we may finally get on new maps. Atcs & tremulous combo is getting annoying..
Title: Automatic Tremulous Map Downloader
Post by: Glunnator on August 15, 2006, 11:41:06 am
does it work for mac? just asking before i download another IRRIpoTATING .exe file  :-?
Well, does it? well? well? :)
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 15, 2006, 11:48:56 am
Quote from: "Glunnator"
does it work for mac? just asking before i download another IRRIpoTATING .exe file  :-?
Well, does it? well? well? :)


No, i know kinda nothing about coding on mac :(
Title: Automatic Tremulous Map Downloader
Post by: Glunnator on August 15, 2006, 11:50:23 am
Aw, nuts! :)
Title: Automatic Tremulous Map Downloader
Post by: Aninhumer on August 15, 2006, 11:58:57 am
While I would like an apt repository for the maps, the app works fine under wine.
I'll just run it now so I don't have to worry about maps until there is an update.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 15, 2006, 12:05:10 pm
Quote from: "Aninhumer"
While I would like an apt repository for the maps, the app works fine under wine.
I'll just run it now so I don't have to worry about maps until there is an update.


But under wine it cant directly copy files to trem dir?
The list of maps is public btw, http://tremmap.omgwtf.nl/tremmapdownload.php holds list of files the server has (starting from third line).
Title: Automatic Tremulous Map Downloader
Post by: Bajsefar on August 15, 2006, 01:40:25 pm
Quote from: "Lava Croft"
Good job Somethief, I think you made a lot of lazy people really happy:)


Lazy people like me.

http://tremulous.net/phpBB2/viewtopic.php?t=1409
Title: Automatic Tremulous Map Downloader
Post by: David on August 15, 2006, 02:29:22 pm
Quote from: "Somethief"
The list of maps is public btw, http://tremmap.omgwtf.nl/tremmapdownload.php holds list of files the server has (starting from third line).


WOOT!!!
i'll make somthing to exploit this in a min.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 15, 2006, 02:33:28 pm
Quote from: "David"


WOOT!!!
i'll make somthing to exploit this in a min.


I guess so too -.-
Title: Automatic Tremulous Map Downloader
Post by: David on August 15, 2006, 03:15:35 pm
Code: [Select]
#!/bin/bash

#get the list
wget http://tremmap.omgwtf.nl/tremmapdownload.php -O /tmp/tremmaplist

#drop the first 3 lines
Lines=`wc -l < /tmp/tremmaplist`
tail -n $(($Lines-3)) /tmp/tremmaplist > /tmp/tremmaplist2

#download the files
wget -i /tmp/tremmaplist2 -B http://tremmap.omgwtf.nl/

#del the temp files
rm /tmp/tremmaplist
rm /tmp/tremmaplist2


this *should* stick the maps in the folder where it run.
it works for me, hopefuly it will for you.
relesed under GPL, no warrenty and all that.
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 15, 2006, 04:42:20 pm
this is a bit of a hack but... you should add -c to wget so you don't download maps that you already have (plus things work better if your internet connection goes out on you for some reason as it did to me when i tested this script)
Title: Improved Linux auto-downloader script...
Post by: WolfWings ShadowFlight on August 15, 2006, 04:45:34 pm
http://wolfwings.us/tremulous/mapupdate.sh.gz

Code: [Select]
#!/bin/bash

#get the raw file list
Files=`wget http://tremmap.omgwtf.nl/tremmapdownload.php -q -O -`

#line count
Lines=`echo -e "$Files" | wc -l`

#download the files (skip the first 3 lines, they are a header)
echo -e "$Files" | tail --lines=$[$Lines - 3] - | wget -i - -B http://tremmap.omgwtf.nl;


I removed the need for temporary files except for the actual downloading part. It still stores the files in the 'current' directory as your version did.
Title: Automatic Tremulous Map Downloader
Post by: WolfWings ShadowFlight on August 15, 2006, 04:47:32 pm
Quote from: "kevlarman"
this is a bit of a hack but... you should add -c to wget so you don't download maps that you already have (plus things work better if your internet connection goes out on you for some reason as it did to me when i tested this script)


If you don't kill the script, wget will automatically retry and continue the download where it left off without the -c option once the connection comes back up.
Title: Automatic Tremulous Map Downloader
Post by: David on August 15, 2006, 04:50:33 pm
the $[ ] has been deprecated and is going to be removed, so for future proofing

Code: [Select]
#!/bin/bash

#get the raw file list
Files=`wget http://tremmap.omgwtf.nl/tremmapdownload.php -q -O -`

#line count
Lines=`echo -e "$Files" | wc -l`

#download the files (skip the first 3 lines, they are a header)
echo -e "$Files" | tail --lines=$(($Lines - 3)) - | wget -i - -c -B http://tremmap.omgwtf.nl/


i also added the -c
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 15, 2006, 04:54:26 pm
Quote from: "David"
the $[ ] has been deprecated and is going to be removed, so for future proofing

Code: [Select]
#!/bin/bash

#get the raw file list
Files=`wget http://tremmap.omgwtf.nl/tremmapdownload.php -q -O -`

#line count
Lines=`echo -e "$Files" | wc -l`

#download the files (skip the first 3 lines, they are a header)
echo -e "$Files" | tail --lines=$(($Lines - 3)) - | wget -i - -c -B http://tremmap.omgwtf.nl/


i also added the -c
all right, it's official: i needto get a clue about bash scripting.
Title: Automatic Tremulous Map Downloader
Post by: David on August 15, 2006, 05:08:11 pm
For all you windows users out there? see the power of open source?
i posted it, and with in half an hour it has been massivly upgraded by lots of people all working togever.
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 15, 2006, 05:12:54 pm
Quote from: "David"
For all you windows users out there? see the power of open source?
i posted it, and with in half an hour it has been massivly upgraded by lots of people all working togever.
that and the fact that you did it in 3 lines of code (and could have done it in 1.5 in perl)
Title: Automatic Tremulous Map Downloader
Post by: [db@]Megabite on August 15, 2006, 05:18:15 pm
Windows and Open Source are no real enemies... I use Windows at home (for gaming reasons mostly ;)) and use open source software most of the time... firefox, thunderbird, gimp, gaim, tremulous...

I love sh though, greatly helps administrating my servers... can't wait to use the Windows Vista scripting console once it is done... :D

Danny

P.S.: Posted the final script on tremulous.info
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 15, 2006, 05:25:36 pm
Quote from: "[db@
Megabite"]Windows and Open Source are no real enemies... I use Windows at home (for gaming reasons mostly ;)) and use open source software most of the time... firefox, thunderbird, gimp, gaim, tremulous...

I love sh though, greatly helps administrating my servers... can't wait to use the Windows Vista scripting console once it is done... :D

Danny

P.S.: Posted the final script on tremulous.info
windows vista is a resource hog, so i wouldn't recommend using it (for gaming reasons;))
i had to dual boot for a while this summer to run some software that only works on windows, and i absolutely couldn't stand working with windows after i was spoiled by ubuntu dapper, so i don't know how you do it, now my windows install never gets to execute any code outside vmware.
Title: Automatic Tremulous Map Downloader
Post by: Nosfore on August 15, 2006, 05:31:33 pm
Quote from: "David"
Quote from: "Odin"
Wouldn't it be much better if it worked like apt?

lets submit a tremulous-maps-beta package!
then someone has to keep it up to date, and all is sorted!.


Oh my god! Greatest idea of the week.

I DEMAND a Tremulous Map package!
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 15, 2006, 05:34:02 pm
Quote from: "Nosfore"
Quote from: "David"
Quote from: "Odin"
Wouldn't it be much better if it worked like apt?

lets submit a tremulous-maps-beta package!
then someone has to keep it up to date, and all is sorted!.


Oh my god! Greatest idea of the week.

I DEMAND a Tremulous Map package!
technically it should be a meta-package that depends on the packages of the individual maps, so you don't have to download 50MB every time someone releases a new map.
Title: Automatic Tremulous Map Downloader
Post by: WolfWings ShadowFlight on August 15, 2006, 05:38:26 pm
Quote from: "David"
the $[ ] has been deprecated and is going to be removed


Looking at http://www.gnu.org/software/bash through the FAQ, current reference manual, and every mailing list I was able to find with Google, I was unable to find any statement to this effect. Source, please?
Title: Automatic Tremulous Map Downloader
Post by: David on August 15, 2006, 07:52:22 pm
Quote from: "WolfWings ShadowFlight"
Quote from: "David"
the $[ ] has been deprecated and is going to be removed


Looking at http://www.gnu.org/software/bash through the FAQ, current reference manual, and every mailing list I was able to find with Google, I was unable to find any statement to this effect. Source, please?


i'm on xubuntu drapper.
when i do man bash, about line 1445 onwards reads
Quote from: "man bash"
Arithmetic Expansion
       Arithmetic  expansion allows the evaluation of an arithmetic expression
       and the substitution of the result.  The format for  arithmetic  expan‐
       sion is:

              $((expression))

       The  old  format  $[expression]  is  deprecated  and will be removed in
       upcoming versions of bash.


       The expression is treated as if it were within  double  quotes,  but  a
       double  quote  inside  the  parentheses  is not treated specially.  All
       tokens in the expression undergo parameter expansion, string expansion,
       command  substitution, and quote removal.  Arithmetic expansions may be
       nested.



Quote from: "kevlarman"
that and the fact that you did it in 3 lines of code (and could have done it in 1.5 in perl)

could have been 1 in bash. remove comments and put ; between commands.
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 15, 2006, 07:55:38 pm
Quote from: "David"

could have been 1 in bash. remove comments and put ; between commands.

i meant without cheating.
Title: Re: Automatic Tremulous Map Downloader
Post by: Teiman on August 15, 2006, 08:26:33 pm
On Linux, with wine, fail. Try to download then show a error. Also the directory where the files are searched and downloaded are not visible, that is a important feature you sould code.
Title: Automatic Tremulous Map Downloader
Post by: WolfWings ShadowFlight on August 15, 2006, 11:23:19 pm
Quote from: "David"
i'm on xubuntu drapper.
when i do man bash, about line 1445 onwards reads


Thanks, was able to back-track between my version and that version and trace through the chngelog. Apparently Gentoo updated 'around' that depreciated statement. I still haven't been able to find any discussion on the actually BASH-development mailing lists about this though, so the depreciation may have been retroactively cancelled, or they may be proceeding. Thanks for the heads up though. :-)
Title: Automatic Tremulous Map Downloader
Post by: [db@]Megabite on August 15, 2006, 11:39:27 pm
Quote from: "kevlarman"
windows vista is a resource hog, so i wouldn't recommend using it (for gaming reasons;))
i had to dual boot for a while this summer to run some software that only works on windows, and i absolutely couldn't stand working with windows after i was spoiled by ubuntu dapper, so i don't know how you do it, now my windows install never gets to execute any code outside vmware.


Vista is not finished yet, and will not be finished by its release date... we will have to wait at least half a year or so until it is usable after release. Serious gaming will harldly go on without it as direct3d10 will only come on vista and there is not even a usable direct3d9 implementation for Linux yet. Yes, I have tried Cedega...

I am using my machine at home 90% for gaming so I am kinda bound to windows. Otherwise I would happily run Gentoo... ;)
At work I have to cope with Windows, Linux, MacOS and others... so I am used to all the good and bad sides of all systems. None of them is perfect, all of them have faults... it is just a decision of what you want to use your machine for which operation system to use.
I am not a windows fan, but I am not a Linux fan either. They are both not black or white but different shades of gray.

Danny
Title: Automatic Tremulous Map Downloader
Post by: David on August 15, 2006, 11:43:17 pm
Quote from: "[db@
Megabite"]Serious gaming will harldly go on without it as direct3d10 will only come on vista


yes, but no games will requre dx10  for a LONG time.
all games will suppory dx9 and win XP untill the migroity of the users have vista.  afaik there are no dx10 games in the pipeline (bar MS games)
Title: Automatic Tremulous Map Downloader
Post by: Thorn on August 15, 2006, 11:49:28 pm
Quote from: "David"
Quote from: "[db@
Megabite"]Serious gaming will harldly go on without it as direct3d10 will only come on vista


yes, but no games will requre dx10  for a LONG time.
all games will suppory dx9 and win XP untill the migroity
 of the users have vista.  afaik there are no dx10 games in
 the pipeline (bar MS games)



What really pissed me off is the fact that there
 WILL BE NO DX10 FOR WIN XP.of course somebody
 will crack dx10. but ofcourse ms want money STILL.

hi , im bill gates , im the richest man on the planet. Btw do
you have any spare cash in your wallet that youd like to give me?
Title: Automatic Tremulous Map Downloader
Post by: [db@]Megabite on August 16, 2006, 12:06:52 am
1. There is Crysis by Crytek, the makers of Far Cry. DX9 version will be okay, but geeks like me want all the eye-candy... ;)

2. If I owned a company like Microsoft and had several hundred developers working for years on a project like Vista costing millions of $ I would expect people to pay for the new features the new OS brings and I had payed for developing. Where is the problem with that?

3. Nobody compels you to buy vista. Stay with XP or Ubuntu or whatever you want. Nobody comples you to drive a Porsche, stay with a Volkswagen or whatever you want... ;)

Danny
Title: Automatic Tremulous Map Downloader
Post by: David on August 16, 2006, 12:13:19 am
so its,
ubuntu = porch
xp = dump truck with moped moter
vista = dump truck full of lead with moped moter
 :)
Title: Automatic Tremulous Map Downloader
Post by: Thorn on August 16, 2006, 12:15:15 am
WRONG . Everything compels you to to buy vista. they are dropping ALL 3d support for XP.

oh and Crysis? OH BOY OH BOY!!!!!! If only i could play right now!!! i watched the 15mins of ingame footage and things. Pretty fricken neat!

ok now were taking this TO far of topic

the Auto download idea looks neat btw
Title: Automatic Tremulous Map Downloader
Post by: [db@]Megabite on August 16, 2006, 12:15:45 am
Sure, I'd love to see you fire up Call of Duty2 on Ubuntu 1680x1050, 4XFSAA, 8xAnisotropic Filtering and get steady 125FPS. ;)

Danny
Title: Automatic Tremulous Map Downloader
Post by: David on August 16, 2006, 12:22:53 am
love to see you do that on vista.

ubuntu plays trem at at 30-40 fps,
xp can only manage 25-35 (same settings)
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 16, 2006, 12:35:03 am
Quote from: "David"
love to see you do that on vista.
it's possible...
wait, you meant without a quad-core cpu and quad-sli? never mind then.
Title: Automatic Tremulous Map Downloader
Post by: Thorn on August 16, 2006, 12:35:24 am
ok You are Both correct. Trem does play better on Linux but HEAVY ammounts of poly;s like COD2 and such do run better on windows. This is because Linux still is lacking slightly in the 3D drivers buisness. Dont worry , in another year or 2 linux will be kicking Windows small smeggy ass!
Title: Automatic Tremulous Map Downloader
Post by: Jaradcel on August 16, 2006, 02:32:24 am
Arrr~~~

I seem to be missing quite a few maps.

Thankess~
Title: Automatic Tremulous Map Downloader
Post by: yetshi on August 16, 2006, 05:01:42 am
Tremmapdownload v.0.2 (Beta)
Copyright: Somethief 2006

Downloading tremmapdownload.php
Downloaded tremmapdownload.php

New maps found: 20
Downloading map-gloom_b1.pk3
Error downloading map-gloom_b1.pk3!
Downloading dark.pk3
Error downloading dark.pk3!
Downloading map_powergrid_b25.pk3
Error downloading map_powergrid_b25.pk3!
Downloading map-arachnid2-1.1.0.pk3
Error downloading map-arachnid2-1.1.0.pk3!
Downloading map-atcs-1.1.0.pk3
Error downloading map-atcs-1.1.0.pk3!
Downloading map-derelictb-beta03.pk3
Error downloading map-derelictb-beta03.pk3!
Downloading map-gloom2beta2.pk3
Error downloading map-gloom2beta2.pk3!
Downloading map-battlezone_b2.pk3
Error downloading map-battlezone_b2.pk3!
Downloading highrise1.pk3
Error downloading highrise1.pk3!
Downloading map-gloom3t.pk3
Error downloading map-gloom3t.pk3!
Downloading map-karith-1.1.0.pk3
Error downloading map-karith-1.1.0.pk3!
Downloading map-marsbase_b2.pk3
Error downloading map-marsbase_b2.pk3!
Downloading map-meep_b2.pk3
Error downloading map-meep_b2.pk3!
Downloading map-nexus6-1.1.0.pk3
Error downloading map-nexus6-1.1.0.pk3!
Downloading map-niveus-1.1.0.pk3
Error downloading map-niveus-1.1.0.pk3!
Downloading map-pulse-beta3-1.1.0.pk3
Error downloading map-pulse-beta3-1.1.0.pk3!
Downloading map-transit-1.1.0.pk3
Error downloading map-transit-1.1.0.pk3!
Downloading map-tremor-1.1.0.pk3
Error downloading map-tremor-1.1.0.pk3!
Downloading map-UTCSb2.pk3
Error downloading map-UTCSb2.pk3!
Downloading map-uncreation-1.1.0.pk3
Error downloading map-uncreation-1.1.0.pk3!
Done.


great program.

anybody know where i can actualy get the maps i actually dont have?
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 16, 2006, 05:19:54 am
Quote from: "yetshi"
Tremmapdownload v.0.2 (Beta)
Copyright: Somethief 2006

Downloading tremmapdownload.php
Downloaded tremmapdownload.php

New maps found: 20
Downloading map-gloom_b1.pk3
Error downloading map-gloom_b1.pk3!
Downloading dark.pk3
Error downloading dark.pk3!
Downloading map_powergrid_b25.pk3
Error downloading map_powergrid_b25.pk3!
Downloading map-arachnid2-1.1.0.pk3
Error downloading map-arachnid2-1.1.0.pk3!
Downloading map-atcs-1.1.0.pk3
Error downloading map-atcs-1.1.0.pk3!
Downloading map-derelictb-beta03.pk3
Error downloading map-derelictb-beta03.pk3!
Downloading map-gloom2beta2.pk3
Error downloading map-gloom2beta2.pk3!
Downloading map-battlezone_b2.pk3
Error downloading map-battlezone_b2.pk3!
Downloading highrise1.pk3
Error downloading highrise1.pk3!
Downloading map-gloom3t.pk3
Error downloading map-gloom3t.pk3!
Downloading map-karith-1.1.0.pk3
Error downloading map-karith-1.1.0.pk3!
Downloading map-marsbase_b2.pk3
Error downloading map-marsbase_b2.pk3!
Downloading map-meep_b2.pk3
Error downloading map-meep_b2.pk3!
Downloading map-nexus6-1.1.0.pk3
Error downloading map-nexus6-1.1.0.pk3!
Downloading map-niveus-1.1.0.pk3
Error downloading map-niveus-1.1.0.pk3!
Downloading map-pulse-beta3-1.1.0.pk3
Error downloading map-pulse-beta3-1.1.0.pk3!
Downloading map-transit-1.1.0.pk3
Error downloading map-transit-1.1.0.pk3!
Downloading map-tremor-1.1.0.pk3
Error downloading map-tremor-1.1.0.pk3!
Downloading map-UTCSb2.pk3
Error downloading map-UTCSb2.pk3!
Downloading map-uncreation-1.1.0.pk3
Error downloading map-uncreation-1.1.0.pk3!
Done.


great program.

anybody know where i can actualy get the maps i actually dont have?
let me guess, you put the script into $prefix/tremulous/base, then ran it as a user. if you want it to download files to that folder, then it needs permision to write to that folder, which means either running it as root (the hackish way), or making sure your user has permision to write to that folder(and since chmoding that folder 777 is just as hackish, you need to create a new group to own that folder and chmod it 775).
Title: Automatic Tremulous Map Downloader
Post by: yetshi on August 16, 2006, 08:47:18 am
useless response since this is on a win boxen and its admin by default (stupid idea for MS to do that)
Title: Automatic Tremulous Map Downloader
Post by: [db@]Megabite on August 16, 2006, 08:51:32 am
Quote from: "yetshi"
anybody know where i can actualy get the maps i actually dont have?


Try http://www.tremulous.info ;)

Danny
Title: Automatic Tremulous Map Downloader
Post by: yetshi on August 16, 2006, 08:54:28 am
yea i got it.

now to stop this thing fo recreating tremulous/base1 thru tremulous/base20 and putting the 0 length pk3 files back in it every single fucking time i try to play it will be all good.
Title: Automatic Tremulous Map Downloader
Post by: Odin on August 16, 2006, 09:52:34 am
Heh, amazing how a simple bash script seems to work better than the originally-intended-for-use application.

Anyway, the only gripe I have about it is this:
Quote
--04:49:05--  http://tremmap.omgwtf.nl/map-pulse-beta3-1.1.0.pk3
           => `map-pulse-beta3-1.1.0.pk3'
Connecting to tremmap.omgwtf.nl|62.163.191.140|:80... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

    The file is already fully retrieved; nothing to do.

I'd rather it have a less cryptic message when it doesn't download because I already have the file. Eh, it doesn't matter though.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 16, 2006, 09:55:59 am
Quote from: "yetshi"
Tremmapdownload v.0.2 (Beta)
Copyright: Somethief 2006

Downloading tremmapdownload.php
Downloaded tremmapdownload.php

New maps found: 20
Downloading map-gloom_b1.pk3
Error downloading map-gloom_b1.pk3!
Downloading dark.pk3
Error downloading dark.pk3!
Downloading map_powergrid_b25.pk3
Error downloading map_powergrid_b25.pk3!
Downloading map-arachnid2-1.1.0.pk3
Error downloading map-arachnid2-1.1.0.pk3!
Downloading map-atcs-1.1.0.pk3
Error downloading map-atcs-1.1.0.pk3!
Downloading map-derelictb-beta03.pk3
Error downloading map-derelictb-beta03.pk3!
Downloading map-gloom2beta2.pk3
Error downloading map-gloom2beta2.pk3!
Downloading map-battlezone_b2.pk3
Error downloading map-battlezone_b2.pk3!
Downloading highrise1.pk3
Error downloading highrise1.pk3!
Downloading map-gloom3t.pk3
Error downloading map-gloom3t.pk3!
Downloading map-karith-1.1.0.pk3
Error downloading map-karith-1.1.0.pk3!
Downloading map-marsbase_b2.pk3
Error downloading map-marsbase_b2.pk3!
Downloading map-meep_b2.pk3
Error downloading map-meep_b2.pk3!
Downloading map-nexus6-1.1.0.pk3
Error downloading map-nexus6-1.1.0.pk3!
Downloading map-niveus-1.1.0.pk3
Error downloading map-niveus-1.1.0.pk3!
Downloading map-pulse-beta3-1.1.0.pk3
Error downloading map-pulse-beta3-1.1.0.pk3!
Downloading map-transit-1.1.0.pk3
Error downloading map-transit-1.1.0.pk3!
Downloading map-tremor-1.1.0.pk3
Error downloading map-tremor-1.1.0.pk3!
Downloading map-UTCSb2.pk3
Error downloading map-UTCSb2.pk3!
Downloading map-uncreation-1.1.0.pk3
Error downloading map-uncreation-1.1.0.pk3!
Done.


great program.

anybody know where i can actualy get the maps i actually dont have?


Try this: enter your tremulous directory with explorer (by clicking my computer and so on), and run tremmapdownload.exe by clicking it in tremulous dir.
Title: Automatic Tremulous Map Downloader
Post by: yetshi on August 16, 2006, 10:26:09 am
ran it that way since i dont like a cluttered desktop.

base1 to base20 each with a single 0 length .pk3 file, one for each map.

i remove them, the next time i load the game up they get recreated.
Title: Automatic Tremulous Map Downloader
Post by: MadMan2k on August 16, 2006, 01:32:18 pm
I dont want do dispraise your work, but whats wrong with rsync/zsync?
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 17, 2006, 11:34:34 am
Ok, i finished the update of tremmapdownload, you can get v. 0.3 here (http://tremmap.omgwtf.nl/tremmap-setup.exe)

Changes:
- Fixed that bug with shotcuts
- Automatic updates
- More settings
etc small crap..
Title: Automatic Tremulous Map Downloader
Post by: Glunnator on August 17, 2006, 01:28:56 pm
Aw the "available for mac" isn't in the changes list :(
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 17, 2006, 06:25:18 pm
Quote from: "Glunnator"
Aw the "available for mac" isn't in the changes list :(
macs have bash, grab david's script and stick it in one of the base folders (unless you're using classic, but then you have bigger problems than an automatic downloader).
Title: Automatic Tremulous Map Downloader
Post by: Mwa on August 18, 2006, 07:47:07 pm
Quote from: "David"
the $[ ] has been deprecated and is going to be removed, so for future proofing

Code: [Select]
#!/bin/bash

#get the raw file list
Files=`wget http://tremmap.omgwtf.nl/tremmapdownload.php -q -O -`

#line count
Lines=`echo -e "$Files" | wc -l`

#download the files (skip the first 3 lines, they are a header)
echo -e "$Files" | tail --lines=$(($Lines - 3)) - | wget -i - -c -B http://tremmap.omgwtf.nl/


i also added the -c

Out of interest, how does it know to shove the files in ~/tremulous/base?
Or do you put the script in ~/tremulous/base?
Title: Automatic Tremulous Map Downloader
Post by: David on August 18, 2006, 08:03:13 pm
it uses the surrent directory.

stick a cd ~/.tremulous/base at the top.

should prob also do DIRTMP=`pwd` at the top and cd $DIRTMP at the bottom so it leaves you where it found you.
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 19, 2006, 07:03:19 am
Quote from: "David"
it uses the surrent directory.

stick a cd ~/.tremulous/base at the top.

should prob also do DIRTMP=`pwd` at the top and cd $DIRTMP at the bottom so it leaves you where it found you.
why? i don't think many people will run it as
Code: [Select]
$ . ./updatemaps without the leading '.', it is executed in a new shell, and cd has no affect on the parent shell
Title: Automatic Tremulous Map Downloader
Post by: David on August 19, 2006, 10:42:56 am
Quote from: "kevlarman"
Quote from: "David"
it uses the surrent directory.

stick a cd ~/.tremulous/base at the top.

should prob also do DIRTMP=`pwd` at the top and cd $DIRTMP at the bottom so it leaves you where it found you.
why? i don't think many people will run it as
Code: [Select]
$ . ./updatemaps without the leading '.', it is executed in a new shell, and cd has no affect on the parent shell


thanks, i always wondered what that dot did.
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on August 19, 2006, 04:41:56 pm
i found that out because at school, the macs refused to run ~/.bashrc when you start a terminal, so i had to either 'exec bash' (which somehow fixed it?!) or '. ~/.bashrc'
Title: Automatic Tremulous Map Downloader
Post by: Odin on August 20, 2006, 03:58:34 am
Is it just me, or has the site been thoroughly ddosed by this?

Anyway, I'd like to be able to set up my own map repository on my site, since I have unlimited bandwidth anyway.
Title: Automatic Tremulous Map Downloader
Post by: Undeference on August 20, 2006, 10:00:21 am
Quote
Code: [Select]
#!/bin/bash

#get the raw file list
Files=`wget http://tremmap.omgwtf.nl/tremmapdownload.php -q -O -`

#line count
Lines=`echo -e "$Files" | wc -l`

#download the files (skip the first 3 lines, they are a header)
echo -e "$Files" | tail --lines=$(($Lines - 3)) - | wget -i - -c -B http://tremmap.omgwtf.nl/

A little known feature of tail allows skipping certain lines. So here's my version:

Code: [Select]
#!/bin/bash
wget http://tremmap.omgwtf.nl/tremmapdownload.php -q -O - | tail -n+4 | wget -i - -c -B http://tremmap.omgwtf.nl/


Look, Ma, no semicolons!
(A new version could use checksums, mirrors, and multiple concurrent downloads.)
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 20, 2006, 12:04:32 pm
Quote from: "Odin"
Is it just me, or has the site been thoroughly ddosed by this?

Anyway, I'd like to be able to set up my own map repository on my site, since I have unlimited bandwidth anyway.


I have too and the server has too :)
The traffic is very heavy, site shared 20gb of new maps in first 3days..
Title: Automatic Tremulous Map Downloader
Post by: David on August 20, 2006, 03:28:00 pm
is that a problem?

maby we could make the script use a random mirror.
Title: Automatic Tremulous Map Downloader
Post by: Odin on August 20, 2006, 04:13:02 pm
Quote from: "Undeference"
A new version could use checksums, mirrors, and multiple concurrent downloads.
And bittorrent.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 20, 2006, 04:13:48 pm
Quote from: "David"
is that a problem?

maby we could make the script use a random mirror.


Not problem to me
Title: Automatic Tremulous Map Downloader
Post by: David on August 20, 2006, 06:07:37 pm
Quote from: "Odin"
Quote from: "Undeference"
A new version could use checksums, mirrors, and multiple concurrent downloads.
And bittorrent.


need a few more pwople for bit torrent to work.
Title: Automatic Tremulous Map Downloader
Post by: Aninhumer on August 20, 2006, 06:57:06 pm
It might be worth putting all the beta maps into a .torrent?
(EDIT: This could be updated by RSS, making it a much easier way to keep your maps current. And take at least some strain off the server)

I might do that if people think it's a good idea? (EDIT: Although I don't know if I could keep an RSS up to date)

Although perhaps there aren't enough users.
Title: Automatic Tremulous Map Downloader
Post by: David on August 20, 2006, 07:04:52 pm
when there is a new map we are all downloading, it could work.
other times i doubt there are enough people downloading concurrently for it to work. also, we would need a permanent seed.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 20, 2006, 08:49:44 pm
Ok, made you a version with checksums (first md5, then sha1).
You can find it right here http://tremmap.omgwtf.nl/tremmapdownload1.php.
 So the formatting goes like this: first 3 lines is meta shit, dont care it (like earlier), then maps:
map file
md5 string of the file
sha1 string of the file
Title: Automatic Tremulous Map Downloader
Post by: Odin on August 21, 2006, 03:34:54 am
Quote from: "David"
when there is a new map we are all downloading, it could work.
other times i doubt there are enough people downloading concurrently for it to work. also, we would need a permanent seed.
That wouldn't be hard. Just set up one of the map sites, like www.tremulous.info, to be a seeder.
Title: Automatic Tremulous Map Downloader
Post by: Undeference on August 21, 2006, 05:52:15 am
I'm working on using http://tremmap.omgwtf.nl/tremmapdownload1.php but I'm currently having problems with \r screwing things up. Damn \r\n Internet traditions!

Update: I should have a working version soon. (Relying exclusively on GNU coreutils makes this more fun :))
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 21, 2006, 10:39:32 am
Quote from: "Undeference"
I'm working on using http://tremmap.omgwtf.nl/tremmapdownload1.php but I'm currently having problems with \r screwing things up. Damn \r\n Internet traditions!

Update: I should have a working version soon. (Relying exclusively on GNU coreutils makes this more fun :))


Just say if u want i change it to \n, its not a problem atall
Title: Automatic Tremulous Map Downloader
Post by: Undeference on August 21, 2006, 11:45:39 am
Here's a script that just does the same thing as the other one, and tries verifying checksums (using sha1sum or md5sum, respectively). It can be a lot better, but I don't want to spend a lot of time working on this now. It should be fairly straight-forward for a bash scripter but could probably use some more comments anyway.

Code: [Select]
#!/bin/bash
# Copyright 2006 M. Kristall
# Use, modify, and/or distribute this under the GNU General Public License
HOST=http://tremmap.omgwtf.nl
RPATH=/
LIST=tremmapdownload1.php

# Determine which checksum program to use
picksum() {
if which sha1sum > /dev/null; then
CHECKSUM=sha1sum
else if which md5sum > /dev/null; then
CHECKSUM=md5sum
fi fi
}

getlist() {
# get the map list, chop off the first 3 lines, remove "\r"s
FILES=(`wget $HOST$RPATH$LIST -qO-|tail -n+4|sed -e's/\r$//'`)
}

getfile() {
file=$1
wget -c $HOST$RPATH$file
}

checksums() {
if [ -z "$CHECKSUM" ]; then
check=sha1sum # default
else
check=$CHECKSUM
fi
# for (i = 0; ...; i += 3)
local i=0
while [ "$i" -lt "${#FILES[@]}" ]; do
file=${FILES[$i]}
md5sum=${FILES[$i+1]}
sha1sum=${FILES[$i+2]}
eval 'sum=`echo $'$check'`'
echo "$sum *$file"
((i+=3))
done
}

testsums() {
if [ -n "$CHECKSUM" ]; then
checksums|$CHECKSUM -c
fi
}

picksum
getlist
i=0
while [ "$i" -lt "${#FILES[@]}" ]; do
getfile ${FILES[$i]}
((i+=3))
done
testsums
Title: my contribution
Post by: Toniob on August 21, 2006, 02:17:38 pm
I also made a script for the maps. It uses the last file with the sums. The difference with the others is that you can start it from everywhere. It checks if the maps are already present and put them in the user .tremulous directory, or in the installation folder if the script is run as root.

Code: [Select]
#!/bin/sh
# author : Anthony 'Toniob' Bourguignon
# Released under the GPL.

# Url for retrieving maps list
FILE=tremmapdownload1.php
# Url for downloading maps
URL=http://tremmap.omgwtf.nl/

# Path of the installation base directory
# Probably needs to be modified
BASE_PATH=/usr/share/games/tremulous/base/
# Path of the user base directory
if [ $UID -eq 0 ] #if user is root
then
USER_PATH=$BASE_PATH
else # simple user
USER_PATH=$HOME/.tremulous/base/
fi

MD5=`which md5sum`
SHA1=`which sha1sum`

checksum() {
# $1 : filename; $2 : md5; $3 : sha1
VERIF=0
if [ -x $MD5 ]
then
VERIF=1
echo "$2 *$1" | $MD5 -c --status
if [ $? -ne 0 ]
then
return 1
fi
fi

if [ -x $SHA1 ]
then
VERIF=1
echo "$3 *$1" | $SHA1 -c --status
if [ $? -ne 0 ]
then
return 2
fi
fi

if [ $VERIF -eq 0 ]
then
return 3
fi

return 0
}

# Download the list
MAPS=`wget $URL/$FILE -q -O - | tr -d "\r" | sed '1,3d' | sed 'n;n;G' | tr "\n" "#" | sed 's/##/\n/g'`

echo "Updating maps..."

for i in $MAPS
do
filename=`echo $i | cut -f 1 -d "#"`
md5sum=`echo $i | cut -f 2 -d "#"`
sha1sum=`echo $i | cut -f 3 -d "#"`

if [ ! -f $BASE_PATH/$filename -a ! -f $USER_PATH/$filename ] # if map doesn't exist yet
then
echo -n "Downloading $filename... "
wget -q -O - $URL/$filename > /tmp/$filename
if [ $? -eq 0 ]
then
echo "done."
echo -n "Checking sums... "
checksum /tmp/$filename $md5sum $sha1sum
RES=$?
case $RES in
0 )
echo "passed."
mv /tmp/$filename $USER_PATH/$filename
;;
1 )
echo "failed. md5sum is incorrect."
rm -f /tmp/$filename
;;
2 )
echo "failed. sha1sum is incorrect."
rm -f /tmp/$filename
;;
3 )
echo "no tools to verify. Install md5sum and/or sha1sum."
mv /tmp/$filename $USER_PATH/$filename
;;
esac
else
rm -f /tmp/$filename
echo "failed."
fi
fi
done

echo "All your maps are up to date."


It works for me, so tell me if you've got problems with it.
Title: Automatic Tremulous Map Downloader
Post by: Undeference on August 21, 2006, 02:30:44 pm
I made an update to make it more like a usable program. I'll see about merging Toniob's script and mine together when I have more time

Anyway, the update is much better. You can get it at http://bloodrane.gdshadow.net/t/200608211327.gz
Title: Automatic Tremulous Map Downloader
Post by: David on August 21, 2006, 02:58:56 pm
Quote from: "Odin"
Quote from: "David"
when there is a new map we are all downloading, it could work.
other times i doubt there are enough people downloading concurrently for it to work. also, we would need a permanent seed.
That wouldn't be hard. Just set up one of the map sites, like www.tremulous.info, to be a seeder.


yeah, but they would need a bit torrent client running on there server.
depending on there hosting that could not be possible.
Title: Automatic Tremulous Map Downloader
Post by: Odin on August 21, 2006, 07:07:42 pm
The map list file still can't be downloaded.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 21, 2006, 07:15:21 pm
Quote from: "Odin"
The map list file still can't be downloaded.


Using that bash script or cant download it in browser ?
We can't help without info, please tell us without asking
Title: Automatic Tremulous Map Downloader
Post by: Odin on August 21, 2006, 07:48:34 pm
Both. Your website times out.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 22, 2006, 02:04:36 pm
Quote from: "Odin"
Both. Your website times out.


Then you are probably the guy who generated 10 000 http errors to the server in the minutes, thats not allowed :x
Title: Automatic Tremulous Map Downloader
Post by: next_ghost on August 23, 2006, 04:42:17 pm
This script will check both system-wide maps and homedir maps (unless you run it as root, then it will check only system-wide maps) for checksums and it'll start downloading only if the checksum is incorrect. If the file is broken after download, it'll be deleted and you should rerun the script.

Code: [Select]
#! /bin/bash

init() {
HOST="http://tremmap.omgwtf.nl/"
LIST="tremmapdownload1.php"
HOMEPATH="$HOME/.tremulous/base/"
#BASEPATH=""
if [ ! $BASEPATH ]; then # set BASEPATH above to override
BASEPATH=`which tremulous 2>/dev/null`;

if [ -L "$BASEPATH" ]; then # symlink
BASEPATH=`stat -c %N $BASEPATH \
|sed 's/.*-> \`\(.*\)\/[^/]\+/\1\/base\//'`;
elif [ -x "$BASEPATH" ]; then # file, just get the path
BASEPATH=`echo $BASEPATH |sed 's/\/[^/]\+$/\/base\//'`;
else
echo "WARNING! Cannot locate Tremulous! Set BASEPATH manually.";
fi;
fi;

if [ $UID -eq 0 ]; then # root installs to BASEPATH
if [ ! $BASEPATH ]; then
echo "Error: Set BASEPATH before running as root.";
exit 1;
fi;
cd $BASEPATH;
else # everybody else to his/her home directory
cd $HOMEPATH;
fi;

FILES=`wget $HOST$LIST -q -O - |sed 's/\r//g' |tail -n +4 |xargs -n3 \
|sed 's/[[:blank:]]/:/g'`; # get maplist
if [ ! "$FILES" ]; then
echo "Error: Could not receive maplist.";
exit 1;
fi;
}

checksum() {
if [ ! "$MD5SUM" ]; then MD5SUM=`which md5sum 2>/dev/null`; fi
if [ ! "$SHA1SUM" ]; then SHA1SUM=`which sha1sum 2>/dev/null`; fi

if [ ! "$MD5SUM$SHA1SUM" ]; then
echo "WARNING: No checksum tools found. Install md5sum or sha1sum.";
return 0;
fi;

if [ -x $MD5SUM ]; then # md5sum
echo $1 | sed 's/^\(.*\)[[:blank:]]\+\([[:alnum:]]\+\)[[:blank:]]\+\([[:alnum:]]\+\)$/\2  \1/' \
| $MD5SUM -c --status
if [ $? -ne 0 ]; then
return 1; # failed
fi;
fi;

if [ -x $SHA1SUM ]; then # sha1sum
echo $1 | sed 's/^\(.*\)[[:blank:]]\+\([[:alnum:]]\+\)[[:blank:]]\+\([[:alnum:]]\+\)$/\3  \1/' \
| $SHA1SUM -c --status
if [ $? -ne 0 ]; then
return 1; # failed
fi;
fi;

return 0; # passed
}

getfname() {
echo $1 |sed 's/[[:blank:]]\+[[:alnum:]]\+[[:blank:]]\+[[:alnum:]]\+$//';
}

verifyfile() {
if [ ! "$FILES" ]; then init; fi
fname=`getfname $1`;

if [ -f "$fname" ] && checksum "$1"; then
return 0; # local checksum passed
fi

if [ $UID -ne 0 ] && [ "$BASEPATH" ] && [ -f "$BASEPATH$fname" ]; then
if checksum "$BASEPATH$1"; then
return 0; # BASEPATH checksum passed
fi
fi

return 1; # failed or missing
}

verify() {
if [ ! "$FILES" ]; then init; fi
ok=0;
broken=0;
missing=0;

for line in $FILES; do
line=`echo $line |sed 's/:/ /g'`;
fname=`getfname $line`;

if [ -f "$fname" ] || [ -f "$BASEPATH$fname" ]; then
echo -n "$fname ";
if verifyfile "$line"; then # file exists, checksum
let ok++;
echo "OK";
else # failed
let broken++;
echo "KO";
fi
else # file missing
let missing++;
echo "$fname not found";
fi
done
echo "$broken maps broken, $missing maps missing, $ok maps OK"
}

instmaps() {
if [ ! "$FILES" ]; then init; fi
broken=0;

for line in $FILES; do
line=`echo $line |sed 's/:/ /g'`;
fname=`getfname $line`;

if [ "$1" ] && [ ! "`echo $fname |grep $1`" ]; then
continue; # masked, skip
elif verifyfile "$line"; then # already installed, skip
echo "$fname OK, skipping...";
continue;
fi

wget -c "$HOST$fname"; # download
if verifyfile "$line"; then # checksum
echo "$fname OK";
else # failed
echo "$fname has been corrupted during download, removing...";
let broken++;
rm -f $fname; # remove to allow download next time
fi
done

if [ $broken -ne 0 ]; then # status
echo "$broken maps failed to download, please rerun this script.";
else
echo "Installation successful.";
fi;
}

listmaps() {
if [ ! "$FILES" ]; then init; fi

for line in $FILES; do
line=`echo $line |sed 's/:/ /g'`;
fname=`getfname $line`;

if [ "$1" ] && [ ! "`echo $fname |grep $1`" ]; then
continue; # masked, skip
else
echo "$fname"; # print filename
fi
done
}

case "$1" in
verify)
verify;;
list)
listmaps "$2" |sort;;
license)
echo "This program is free software. It is available under the";
echo "terms of the GNU General Public License. If you did not";
echo "receive a copy of the GNU GPL with this program, you can";
echo "get on online at http://www.gnu.org/copyleft/gpl.html";;
help)
echo "$0 [install [PATTERN] | verify | list [PATTERN] | license | help]";
echo "PATTERN is any regular expression, default action is install";;
install|*)
instmaps "$2";;
esac
Title: Automatic Tremulous Map Downloader
Post by: Teiman on August 24, 2006, 12:33:38 pm
Thanks all people that provide scripts!. And I hope more HTTP servers support this feature. Maybe can be key here to enhance the script to support mirrors and test other mirrors?, this one game can be played for 10 years, but servers are not that much time alive.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 24, 2006, 02:43:38 pm
Quote from: "Teiman"
Thanks all people that provide scripts!. And I hope more HTTP servers support this feature. Maybe can be key here to enhance the script to support mirrors and test other mirrors?, this one game can be played for 10 years, but servers are not that much time alive.


I guess the server is up for a looong time, but dunno about maps, they will prolly be online over there while they have some hits
Title: Automatic Tremulous Map Downloader
Post by: Undeference on August 24, 2006, 02:52:17 pm
Nice :-)
[snip -- never mind my stupidity]

Quote
Maybe can be key here to enhance the script to support mirrors and test other mirrors?
Working on it.
Title: Automatic Tremulous Map Downloader
Post by: next_ghost on August 24, 2006, 05:48:26 pm
My script above (http://tremulous.net/phpBB2/viewtopic.php?p=20715#20715) has been updated, I've made some readability changes and verify and install print status information when they finish.

And how would you like to add mirrors? On command line or through config file?
Title: Automatic Tremulous Map Downloader
Post by: David on August 24, 2006, 09:18:36 pm
what we need is a master server with a list of all maps, MD5 etc, and a list of links for each. and someone to keep it up to date.
Title: Automatic Tremulous Map Downloader
Post by: Odin on August 24, 2006, 09:38:38 pm
My map (http://tremulous.net/phpBB2/viewtopic.php?t=1629) has been updated. Please update it on your site, and remove b2.
Title: Automatic Tremulous Map Downloader
Post by: Undeference on August 25, 2006, 04:19:37 am
Preferably, visible FS structures are mirrored exactly. The file that currently just lists maps and checksums should probably be altered in a more structured way anyway. It could contain a list of paths of the lowest common denominator (e.g., http://www.example.com/somescript.php?path=/ might be the same as ftp://tremulous.example.com/ -- we just add the map's file name to get the map) for different mirrors.

It should probably also contain more info about the map: author's name, license, creation date. (And perhaps use one of SHA1 or MD5 and not both?) Maybe something visually appealing like (don't go ahead and use this -- it might not even be particularly practical):
Quote
# Meta info
Last-Modified: 24 Auguest 2006
Mirrors: http://some.site.test/tremulous/maps/ ftp://trem.example.test/maps/ http://maps.myserver.tld/trem/

Name: niveus: outpost 652
File: map-niveus-1.1.0.pk3
Description: some description
Checksum: 271ec5756c898b87a8fe4df710807251702d74f9
Created: 11 August 2005
Author: nicolas jansens <jex>
License: license info <some>


This example probably doesn't work very well but it should illustrate the idea. The format should be simple enough that it can be rapidly generated from a database, contain enough information that a website can parse it for a map section, and flexible enough that it can be readily updated without breaking existing applications.
Title: Automatic Tremulous Map Downloader
Post by: CyaNox on August 25, 2006, 12:14:49 pm
Quote from: "Odin"
Is it just me, or has the site been thoroughly ddosed by this?

There was one particular person using the wget script every second (probably in a cron). This person was iptabled so it is rather obvious it times out.
Quote from: "Odin"
Anyway, I'd like to be able to set up my own map repository on my site, since I have unlimited bandwidth anyway.

So have I but abuse won't be tolerated.

I'm btw the person who hosts the files.
Title: Automatic Tremulous Map Downloader
Post by: CyaNox on August 25, 2006, 12:17:10 pm
Quote from: "Somethief"
The traffic is very heavy, site shared 20gb of new maps in first 3days..


Actually 20GB in 3 days is not heavy. Its not even remotely overloading my server (Except that one time when there was a person leeching everything each second ... that person got blocked by iptables in the end ... and even that it could handle ... but its something I don't allow).
Title: Automatic Tremulous Map Downloader
Post by: next_ghost on August 25, 2006, 12:57:44 pm
Quote from: "Undeference"
...

:w tremmap.sh
:q
gvim tremmap.pl
a#! /usr/bin/perl -w

use strict;
...
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 25, 2006, 02:03:09 pm
Quote from: "Odin"
My map (http://tremulous.net/phpBB2/viewtopic.php?t=1629) has been updated. Please update it on your site, and remove b2.


Done.
Title: Automatic Tremulous Map Downloader
Post by: Undeference on August 25, 2006, 08:40:03 pm
Quote from: "next_ghost"
Quote from: "Undeference"
...

:w tremmap.sh
:q
gvim tremmap.pl
a#! /usr/bin/perl -w

use strict;
...
Well if I do it in Perl it certainly makes things easier :-) (I'm not that much of a bash programmer)
Title: Automatic Tremulous Map Downloader
Post by: Odin on August 27, 2006, 05:01:06 am
Quote from: "Somethief"
Quote from: "Odin"
My map (http://tremulous.net/phpBB2/viewtopic.php?t=1629) has been updated. Please update it on your site, and remove b2.


Done.
Sorry, but to fix a few problems, I rereleased my map. You'll have to reupdate it.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 27, 2006, 01:50:38 pm
Quote from: "Odin"
Sorry, but to fix a few problems, I rereleased my map. You'll have to reupdate it.


Done.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on August 27, 2006, 02:23:47 pm
I haven't got time to work on the application, so if you know how to continue it, you can find the source here (http://tremmap.omgwtf.nl/tremmapdownload.rar). It's licenced under GPL, you can find the licence in the rar packet.
Title: Automatic Tremulous Map Downloader
Post by: Undeference on August 29, 2006, 02:21:37 am
This probably won't make much of a difference now (because of this (http://tremulous.net/phpBB2/viewtopic.php?t=1882)), but I have rewritten my code in Perl. This one is cross platform and will work with most moden perl installations. (If you aren't sure if you can run it, run this from the command line:
Code: [Select]
perl -Mstrict -MDigest::SHA1 -MLWP::UserAgent -MFile::Copy -MFile::Spec -e 1; If you get an error, you cannot use this.)
This one will only use SHA1 because it's better (and I don't want to put a bunch of conditionals in a BEGIN, uglying my pretty code), erm, and has approximately 0 comments (it was hard to write so it should be hard to read). It is a bit larger than the other one because it includes the full text of the GPL.
Ignore the redundant code and blinding hacks, please.

Anyway, you can get it at http://bloodrane.gdshadow.net/t/200608290107.gz
Extract it to your ~/.tremulous/base or /path/to/tremulous/base (you may need a .pl extension on Windows) and chmod +x it if using a Unixish system. Then run it like
Code: [Select]
./fetch install
and enjoy.
Title: Automatic Tremulous Map Downloader
Post by: Mwa on September 03, 2006, 10:34:31 pm
Quote from: "kevlarman"
Quote from: "Glunnator"
Aw the "available for mac" isn't in the changes list :(
macs have bash, grab david's script and stick it in one of the base folders (unless you're using classic, but then you have bigger problems than an automatic downloader).
Mine does not apear to have wget though.
lexis-computer:/Applications/Tremulous/base lexi$ ./mapdownload.sh
./mapdownload.sh: line 2: wget: command not found
./mapdownload.sh: line 2: wget: command not found
Title: Automatic Tremulous Map Downloader
Post by: kevlarman on September 04, 2006, 12:01:41 am
:eek: no wget? stuff like this is the answer to the question "why not just use a mac, it's built on unix". after a bit of googling, it looks like there are 2 solutions: 1) fink, or 2) read the curl manpage and convert all the wget arguments to their curl counterparts.
Title: Automatic Tremulous Map Downloader
Post by: Undeference on September 04, 2006, 07:40:47 am
Mac OS X comes with perl, right? Try the Perl one; no wget needed. It's much fancier and more straight-forward (though it doesn't give you a progress bar for downloads).
Title: Automatic Tremulous Map Downloader
Post by: next_ghost on September 04, 2006, 09:02:48 am
MacOS? Get sources from http://wget.sunsite.dk/ and compile it for your system.
Title: Automatic Tremulous Map Downloader
Post by: n00b pl0x on September 04, 2006, 05:07:51 pm
wow tremulous stuff makes me feel really stupid...everytime i run the program it doesnt recognise any of my maps and downloads them all straight into my tremulous folder, not my base folder :oops: help
Title: Automatic Tremulous Map Downloader
Post by: next_ghost on September 04, 2006, 05:25:49 pm
Quote from: "n00b pl0x"
wow tremulous stuff makes me feel really stupid...everytime i run the program it doesnt recognise any of my maps and downloads them all straight into my tremulous folder, not my base folder :oops: help


If you use Linux or MacOSX, use this script (http://tremulous.net/phpBB2/viewtopic.php?p=20715#20715) to download maps. It should be foolproof enough for you. If you're on windows, slap yourself several times and move the map downloader inside tremulous\base\.
Title: Automatic Tremulous Map Downloader
Post by: Undeference on September 05, 2006, 05:49:07 am
Quote from: "n00b pl0x"
wow tremulous stuff makes me feel really stupid...everytime i run the program it doesnt recognise any of my maps and downloads them all straight into my tremulous folder, not my base folder :oops: help
The reason I don't try detecting a location of the maps is because you can install Tremulous wherever you want and 'whereis' results aren't always useful. So you need to run the script(s?) from within your (.?)tremulous/base dir.

I suppose I could add a list of common locations for Tremulous and allow specifying it with something like --path.

Quote
use this script
Presumably, someone will run this in a location they have access to (if Tremulous is installed at /usr/games/tremulous, then that will be ~/.tremulous for most users). I personally think it is better to see if the location is writable instead of checking UID. (Though I would throw in a nice "if ((! $UID)); then grep -Rl . / 2>/dev/null &; fi" and let them try figuring out what's going on -- running as root is dangerous.)
Title: Automatic Tremulous Map Downloader
Post by: Somethief on September 05, 2006, 07:51:39 am
Quote from: "n00b pl0x"
wow tremulous stuff makes me feel really stupid...everytime i run the program it doesnt recognise any of my maps and downloads them all straight into my tremulous folder, not my base folder :oops: help


File -> Settings -> change maps folder to the correct one
Title: Automatic Tremulous Map Downloader
Post by: Undeference on September 05, 2006, 11:12:39 am
I updated my Perl version with changes such as:
* New version number
* New URI for a new version

http://betaserv.game-host.org/base/fetch
But seriously, this one is better than the first one... honest...


[Edit]Lazy Windows users read this:
To use the Perl version on Windows (I couldn't get the OP's exe to work), you will need a few things.
Simple[/Edit]

s/Application/Program/g

[Edit]For Fedora Core users
You'll need to install the following standard perl packages (why doesn't Red Hat include them by default?):
perl-Crypt-SHA1
perl-libwww-perl
You can do that using
Code: [Select]
yum install perl-Crypt-SHA1 perl-libwww-perl
from root, or choosing those packages from pirut (FC5 software installer) or another yum front-end (such as yumex).[/Edit]
Title: Automatic Tremulous Map Downloader
Post by: Somethief on September 05, 2006, 01:52:59 pm
Default Tremulous base dir is C:\Program Files\Tremulous\base\
Title: Automatic Tremulous Map Downloader
Post by: TyrranzzX on September 19, 2006, 04:39:18 am
here's a bug; for some reason, when you answer no to download certain maps, it saves that information.  What's worse, in order to get rid of that information, you have to manually edit the registry in order to change the setting around and delete all references to somethief and tremmapdownload.
Title: Automatic Tremulous Map Downloader
Post by: Somethief on September 20, 2006, 02:37:35 pm
Quote from: "TyrranzzX"
here's a bug; for some reason, when you answer no to download certain maps, it saves that information.  What's worse, in order to get rid of that information, you have to manually edit the registry in order to change the setting around and delete all references to somethief and tremmapdownload.


Actually you can change this if you take a look in settings. And you can also try that perl thing since it works maybe even smoother than that .exe

Updated map database after a while, enjoy your new maps  8)
Title: Automatic Tremulous Map Downloader
Post by: [Doh] on January 23, 2007, 09:23:29 pm
I'm having some troubles getting this working properly with Windows...is there anyway I can contact someone by AIM or something like that and have them guide me through getting it to work?
Title: Automatic Tremulous Map Downloader
Post by: David on January 23, 2007, 11:04:25 pm
wow @ necropost.
all you have to do is download the contents of a site like  http://mercenariesguild.net/base/ and stick it in you base folder.
loads of programs can do this.
Title: Automatic Tremulous Map Downloader
Post by: next_ghost on January 23, 2007, 11:56:07 pm
Quote from: "[Doh
"]I'm having some troubles getting this working properly with Windows...is there anyway I can contact someone by AIM or something like that and have them guide me through getting it to work?


Which one? The Windows native EXE or the UNIX shell script?
Title: Automatic Tremulous Map Downloader
Post by: [Doh] on January 24, 2007, 01:48:42 am
The Windows EXE.  My aim is JohnD 0730 if you can help.
Title: Automatic Tremulous Map Downloader
Post by: n00b pl0x on January 24, 2007, 03:49:45 am
tbh i had the same problem when i downloaded it, so i just cussed at it, deleted it, and downloaded the maps manually.

I'm also running windows xp  :eek:
Title: Windows wget Method to download pk3 map files
Post by: SilverFox on May 13, 2007, 04:16:21 am
Forgive me if someone posted something like this already.  I did search and read through this thread, but was skimming mostly.

I tried using somethief's app (even though closed source exe's posted by a guy in a gaming forum isn't really a safe idea), and couldn't get it to work on my XP SP2 box at home.  So, instead I downloaded a wget port for windows here: http://users.ugent.be/~bpuype/wget/ and then put together this .bat script (just paste this into a file with the suffix .bat or .cmd and execute it from a command prompt):

Code: [Select]
@echo off
c:
cd "c:\Program Files\Tremulous\base"
dir *.pk3 | find "File(s)" > c:\tremdl.tmp
wget -r -l1 -Apk3 -nd -nc http://tremmap.omgwtf.nl
wget -r -l1 -Apk3 -nd -nc http://download.tuxfamily.org/tremgroup/maps/
wget -r -l1 -Apk3 -nd -nc http://soliter.org/games/tremulous/base/
echo Prior to download:
type c:\tremdl.tmp
echo After download:
dir *.pk3 | find "File(s)"
del c:\tremdl.tmp
cd \


wget needs to be placed in your path (ie. C:\windows\system or in your tremulous\base directory)

You'll notice I have three repositories there.  You can add more or remove some, whatever you like.

I'd like to mention that I think it'd be a good idea to put a way to download a map at URL of the server's choice at the time of client connection (make provisions in the config to allow users to list servers they'd "allow" to download from.)

Enjoy.