Minecraft Alpha Server Backup Utility

[Shocks]

Still doesn't seem to work, gives me the same "Access Denied" thing. Apparently the host we go through requires the @ sign in the user name too.

Actually the issue was with the password, the semi colon on it was messing it up. So WINSCP works now, my problem now is with Overviewer, which I got a feeling you can't do much about.

but here it is:

Microsoft Windows [Version 6.0.6001]

Copyright © 2006 Microsoft Corporation. All rights reserved.

C:\Users\Administrator>e:

E:\>cd Overviewer

E:\Overviewer>c:\python27\python gmap.py "c:\Users\Adminitrator\Minecraft\World1" "e:\Current Map"

Traceback (most recent call last):

File "gmap.py", line 33, in <module>

import world

File "E:\Overviewer\world.py", line 23, in <module>

import numpy

ImportError: No module named numpy

Haven't seen this before.

Bah, I think the issue is that Numpy and PIL default to Python2.6 and not Python2.7, I changed the path to point to 2.6 and it works

\

[Dateranoth]

Still doesn't seem to work, gives me the same "Access Denied" thing. Apparently the host we go through requires the @ sign in the user name too.

Actually the issue was with the password, the semi colon on it was messing it up. So WINSCP works now, my problem now is with Overviewer, which I got a feeling you can't do much about.

but here it is:

Haven't seen this before.

Bah, I think the issue is that Numpy and PIL default to Python2.6 and not Python2.7, I changed the path to point to 2.6 and it works

\

Yeah, Overviewer says it works with 2.7, but I had issues with it. So I use 2.6 myself. So, is everything working well for you now?

[Shocks]

Well I got it to work correctly, with one minor issue. For some reason, it decided to merge my old map which I ran Overviewer and uploaded manually, with the map that your utility ran automatically.

I got a feeling that the old tiles folder that was on the FTP server wasn't erased properly or something and it just overwrote what was there (old manual run) with what was new (new auto run). So I am in the (slow) process of deleting the tiles folder from the FTP, and getting ready to rerun the whole backup utility again. Only took 11 hours to run Overviewer and upload last time, yay for 30kb up!

[Dateranoth]

Well I got it to work correctly, with one minor issue. For some reason, it decided to merge my old map which I ran Overviewer and uploaded manually, with the map that your utility ran automatically.

I got a feeling that the old tiles folder that was on the FTP server wasn't erased properly or something and it just overwrote what was there (old manual run) with what was new (new auto run). So I am in the (slow) process of deleting the tiles folder from the FTP, and getting ready to rerun the whole backup utility again. Only took 11 hours to run Overviewer and upload last time, yay for 30kb up!

Ah, sorry for that. I thought it should have synced and deleted anything on the ftp that wasn't on the local drive. I'll look into it, but it might have just been a fluke.

[Shocks]

Well I think the issue was with the cache file or something, I just decided to delete everything, all the backup folder, the cache folder, the output folder and everything on the FTP. Ran the program again and my chunk count is down to 3103, from 26000 last night. I think I might have some how got my old world file jammed into the new one or something. Not really sure what happened, but it looks like it is fixed now.

[Dateranoth]

Well I think the issue was with the cache file or something, I just decided to delete everything, all the backup folder, the cache folder, the output folder and everything on the FTP. Ran the program again and my chink count is down to 3103, from 26000 last night. I think I might have some how got my old world file jammed into the new one or something. Not really sure what happened, but it looks like it is fixed now.

That's great! I'm glad it's finally working for you. Sorry for the trouble, but you have helped me work out a lot of bugs. Any links to the map?

[Shocks]

The map will be at http://map.zjnewbs.com when it is finished

It is a relatively new map, and we haven't done a lot to it since we were waiting on the Halloween update (which didn't do half the things for SMP we had hoped), so there isn't a lot to see.

[Dateranoth]

The map will be at http://map.zjnewbs.com when it is finished

It is a relatively new map, and we haven't done a lot to it since we were waiting on the Halloween update (which didn't do half the things for SMP we had hoped), so there isn't a lot to see.

Yeah, completely understandable. His latest blog says he will be concentrating on SMP and pushing for beta. So, maybe we will see some good improvements over the next few weeks.

[Dateranoth]

::V2.4-Added double quotes around FTP usernames and passwords. Common @ symbols in-----::

::-----user names were messing up the script. Should work with these now. Added option-::

::-----to wait for X seconds before running backup after server save-all. This will-----::

::-----make sure servers with lots of changes have time to finish saving before the-----::

::-----files are copied and compressed.-------------------------------------------------::

[Dateranoth]

::V2.5-Added backup of Minecraft Server directory files. Only the files, and not the---::

::-----directories in your Server Directory will be copied into the backup world in a---::

::-----folder call _ServerFiles. This way you will have the configs if you have a crash::

::-----or need to restore when using mods/etc. There is also the option to use an------::

::-----exclude file. This file contains files or filetypes (one per line) that you-----::

::-----don't wish to have backed up. I have included a default one that skips .jar .bat::

::-----and .vbs. You can choose to use it or not. Default is to back up all files.----::

If wanting the exclude feature then download the full package from the links above. This patch should make it easier to use mods/plugins. Since they will now be backed up to the world directory for future restoration if needed.

[kwijy]

many thanks for making the scripts and making the available!

i've slowly been ironing out all the little issues i've had along the way and it's getting quite close.

Even though I've set the -passive switch in the switches variable, I still get winscp prompting me to ®etry the connection. I get this message:

....Copying files to remote side failed.

Entering Passive Mod (some kind of IP number or such)

(a)abort, ®etry.. etc etc..

Any ideas?

Also, with the SMP server I'm hosting, the map rendering took 8 hours with a total file size of 1.5GBs. It's quite large.. and not really feasible for me to upload to an

FTP. I've been toying around with the -z (zoom) command, but I'm not exactly sure how it works. I was hoping for a function that could reduce the resolution or remove the

most detailed layer of the map, as with such a large map, we don't really need to see the textures of each tiny little block, or the details in the trees foliage - just more

of an overview of the areas.

Many thanks again for all your time invested in the utility.

[Dateranoth]

<br />many thanks for making the scripts and making the available!<br />i've slowly been ironing out all the little issues i've had along the way and it's getting quite close.<br /><br />Even though I've set the -passive switch in the switches variable, I still get winscp prompting me to ®etry the connection. I get this message:<br /><br />....Copying files to remote side failed.<br />Entering Passive Mod (some kind of IP number or such)<br />(a)abort, ®etry.. etc etc..<br /><br />Any ideas?<br /><br /><br />Also, with the SMP server I'm hosting, the map rendering took 8 hours with a total file size of 1.5GBs. It's quite large.. and not really feasible for me to upload to an<br />FTP. I've been toying around with the -z (zoom) command, but I'm not exactly sure how it works. I was hoping for a function that could reduce the resolution or remove the<br />most detailed layer of the map, as with such a large map, we don't really need to see the textures of each tiny little block, or the details in the trees foliage - just more<br />of an overview of the areas.<br /><br />Many thanks again for all your time invested in the utility.<br />

<br /><br /><br />

Thanks for the appreciation. Id be happy to help. Remove any passwords and post your user variables here. Ill look into it after work.

[kwijy]

<br /><br /><br />

Thanks for the appreciation. Id be happy to help. Remove any passwords and post your user variables here. Ill look into it after work.

::Set to use Incremental Backup. Only supports 7zip compression method. (yes or no)

::Incremental will create one base file and then only the changed updates from that point forward in 7-zip format.

::In order to restore the world you MUST extract the base file and then the changes in order until the date/time you wish to restore is reached.

SET UseIncr=yes

::Set to the number of Incremental Backups you would like created before a new folder and master file is created.

::If you multiply this number by your backup Time you will get the amount of time between new incremental folder creation.

::ie. Backup time of 10 minutes, and an Incremental Backup # of 6 would create a new master and folder every 60 minutes.

SET IncrCounterMax=72

::Set Server Directory (The folder your "World" folder lives in...NOT the path inside your world folder)

SET WorldDIR=C:\Minecraft\hMod Server mod\bin

::Set World Name (Copy your World Folder name here)

SET WorldNAM=KwijyServer

::Set Backup Directory (Where do you want to save your backups?)

SET BackupDIR=C:\Minecraft\Backup

::Set Time between Backups (in minutes)

SET Timer=60

::Set Compression Method (rar or 7zip)

SET Method=7zip

::Set Directory for 7z.exe

SET _7zipdir=C:\Program Files\7-Zip

::Set the format you would like the date to appear on backup files. Default is year(%%g)-month(%%e)-day(%%f). Your windows time format may differ.

:: If it does or you would like something different then rearrange the parameters below until satisfied. Results will be format1-format2-format3

set dateformat1=%%g

set dateformat2=%%f

set dateformat3=%%e

::Would you like to use the VBS file to force a save-all to your server? yes or no

SET UseVBS=yes

::Set directory where saveall.vbs is located

SET VBSDir=C:\Minecraft\hMod Server mod\bin

::Set how long you would like to wait for Minecraft Server to finish save-all before backing up.

::Maps with lots of players and changes occuring will need more time than maps

::with only a few players and changes. (Time in Seconds, 0 to start immediately)

SET VBSDelay=15

::Will you be using Sleep.exe? (yes or no)

SET UseSleep=yes

::Set Sleep EXE directory if using Sleep.exe

SET SleepExe=C:\windows\system32

::Would you like to use Cartographer to map your backups? yes or no

SET UseCart=no

::Set directory where Cartograph.exe is located

SET CartDIR=C:\Minecraft\Cartograph

::Set Cartograph map type. Refer to Cartograph Readme.txt for more info or leave at default setting.

SET CartMap=normal 0 0 0 0

::Set how often you would like Cartograph to be ran. (in minutes) Minimum is the time between backups.

::Will only be ran when a backup runs. ie. If your backup time is every 30 minutes, then Cartograph

::can be set to run every 30 minutes, 1 hour, 1 hour and 30 minutes, 2 hours, etc.

::Odd times will round forward, running after the time has been met but not until the backup starts.

::Time can be different from Overviewer Time.

SET CartTimer=480

::Set to 0 if you DON'T want Cartograph to run when the batch file is initially opened.

SET CartFirstRun=1

::Would you like to use Overviewer to map your backups? First run can take a LONG time. (yes or no)

SET UseOVWR=yes

::Which method of Overviewer? Windows Executable or Python Script? (exe or python)

SET OVWRmethod=python

::Set directory where Overviewer is located

SET OVWRDIR=C:\Overviewer

::Set Overviewer Image Directory (Overviewer Output)

SET OVWROUTDIR=C:\Overviewer\Output

::Set Overviewer CACHE Directory (Where Overviewer Cache files are stored)

SET OVWRCACHEDIR=C:\Overviewer\Cache

::How many processors should Overviewer Use? (This should be set low or it will cause lag on the server)

SET Procs=1

::Set how often you would like Overviewer to be ran. (in minutes) Minimum is the time between backups.

::Will only be ran when a backup runs. ie. If your backup time is every 30 minutes, then Overviewer

::can be set to run every 30 minutes, 1 hour, 1 hour and 30 minutes, 2 hours, etc.

::Odd times will round forward, running after the time has been met but not until the backup starts.

::Time can be different from Cartograph Time.

SET OVWRTimer=960

::Set to 0 if you DON'T want Overviwer to run when the batch file is initially opened.

SET OVWRFirstRun=1

::Would you like to use WinSCP FTP to upload the Overviewer map to a web server? (yes or no)

::If connecting to a secure ftp the first run will ask you to save the key to cache.

::Once saved it will run automatically without user input from that point on.

SET useftp=yes

::WinSCP is required. Please set the path to WinSCP

SET ftpwinscpdir=C:\Program Files\WinSCP

::Set your ftp Protocol (ftp or sftp or scp)

SET ftpprotocol=ftp

::This is where you can set WinSCP switches for connecting. Refer to http://winscp.net/en...t_commands#open for details.

::Main uses are to leave blank for default OR -passive for ftp protocol passive mode.

SET ftpswitch=-passive

::Set your ftp address here WITHOUT ftp:// (site:port)

SET ftpurlport=*******

::Set the login name for your FTP.

SET ftploginname=*******

::Set the password for your FTP.

SET ftploginpass=*******

::Set the max upload bandwidth you want WinSCP to use (kibps)

SET ftpspeed=30

::Set your REMOTE ftp directory for Overviewer. Should be base directory containing index.html. (Full Remote Path)

SET ftpovwrdirectory=/minecraft/

and just to be sure...

::Attempts to kill WinSCP in case it was still running or stuck from last time.

::Starting the FTP Upload Process. It is opened as a seperate process and is not waited for because it can take a long time.

:startftp

ECHO Starting FTP Upload of Overviewer. Process will open in a new window and backup batch will continue to run without waiting.

taskkill /im winscp.exe

cd /d "%ftpwinscpdir%"

start winscp.exe /console /command "open %ftpprotocol%://%ftploginname%:%ftploginpass%@%ftpurlport% %ftpswitch%" "synchronize remote -delete -criteria=both -speed=%ftpspeed% %OVWROUTDIR% %ftpovwrdirectory%" "exit"

goto endbackup

::Python method was selected. Overviewer will now be ran using the selected options. Run timer will then be reset.

:OVWRpythonstart

cd\

c:\Python26\python c:\overviewer\gmap.py --cachedir="%OVWRCACHEDIR%" -p "%Procs%" -z 8 "%TempWorldDir%" "%OVWROUTDIR%"

SET RemainingOVWRTimer=0

IF /i %useftp%==yes (goto startftp) ELSE (goto endbackup)

many thanks in advance.

[Dateranoth]

and just to be sure...

many thanks in advance.

If you're sure your folder is in the root of the remote directory then everything looks good. Try removing the trailing slash in the remote directory

SET ftpovwrdirectory=/minecraft/

to

SET ftpovwrdirectory=/minecraft

The full directory can be deceiving as well. If the above doesn't work, then open WinSCP (GUI version that was installed). Connect to your ftp, and then navigate to the folder you want to place the files in. The full path that you need to use is in the blue bar at the top of the remote directory.

As far as overviewer is concerned. I can't really help much with that. The zoom function will crop your map if you use too low of a number. The overall map will be smaller, but only because there is less map shown. The only other option is to check the readme here:

https://github.com/b...raft-Overviewer

Near the bottom is a method to 'crush' the output files which basically compresses them so they take up less space. However, if you leave it be and take the time to upload then it won't be as bad the next times. Overviewer uses a great cache, and the ftp setup in my code syncs the files so only new or updated files are transferred to the remote directory.

Finally, I see that you have your backups set to every 60minutes. With incremental backups you could probably backup more often without much extra disk usage or load on the server.

Hope that helps.

[kwijy]

ah good one. i'll give the 'crush' function a go. like you said, once it's initially uploaded, it uploads incrementally so it won't be as much of an ordeal next time.

a couple more quick questions.

does the incrcountermax create another directory and set of files once it's been triggered? so if i set that low, will it mean i'll use up a lot of disk space with previous backups?

also. does the cache for Overview have to be cleared if i change a setting like -z? or will it use the layers and alter the previous cache?

thanks for having a look at the batch for me.

champ!

i'll let you know how it goes and shoot you a link to the map when it's online.

If you're sure your folder is in the root of the remote directory then everything looks good. Try removing the trailing slash in the remote directory

SET ftpovwrdirectory=/minecraft/

to

SET ftpovwrdirectory=/minecraft

The full directory can be deceiving as well. If the above doesn't work, then open WinSCP (GUI version that was installed). Connect to your ftp, and then navigate to the folder you want to place the files in. The full path that you need to use is in the blue bar at the top of the remote directory.

As far as overviewer is concerned. I can't really help much with that. The zoom function will crop your map if you use too low of a number. The overall map will be smaller, but only because there is less map shown. The only other option is to check the readme here:

https://github.com/b...raft-Overviewer

Near the bottom is a method to 'crush' the output files which basically compresses them so they take up less space. However, if you leave it be and take the time to upload then it won't be as bad the next times. Overviewer uses a great cache, and the ftp setup in my code syncs the files so only new or updated files are transferred to the remote directory.

Finally, I see that you have your backups set to every 60minutes. With incremental backups you could probably backup more often without much extra disk usage or load on the server.

Hope that helps.

[Dateranoth]

ah good one. i'll give the 'crush' function a go. like you said, once it's initially uploaded, it uploads incrementally so it won't be as much of an ordeal next time.

a couple more quick questions.

does the incrcountermax create another directory and set of files once it's been triggered? so if i set that low, will it mean i'll use up a lot of disk space with previous backups?

also. does the cache for Overview have to be cleared if i change a setting like -z? or will it use the layers and alter the previous cache?

thanks for having a look at the batch for me.

champ!

i'll let you know how it goes and shoot you a link to the map when it's online.

I'm not sure on the -z setting. You might have to check his site or just try it. If you can't get any difference between the settings then you will likely have to delete both the map and the cache to get it to work.

The incrcountermax will create another directory and set of files once it's been triggered. If you set it to 1 then you will be creating a full sized backup but it will still be cached and thus quicker than the normal full backup. However, that would take up a lot of space. The best implementation is a low backup time ( ie every 5 to 30 minutes ) and a higher increcountermax number. This way you have more frequent backups without much space being consumed.

[saladin1980]

Hate to bring up old topics but was wondering if there was a way to make this upload the zip just created to the ftp as well? but also I LOVE THIS it helps SOOOO much i don't use the imaging but i do use the backup and it makes it so simple... and well on a server that actually has no monitor hooked up helps even more LOL thnx for this and again thnx for this:) lol

[Dateranoth]

I'm glad you brought it up. We had our server down for a month or more due to waiting for a decent admin software ( bukkit now ) and I had meant to add this. I'm getting good and drunk tonight, but I'll look into it tomorrow. It should be very simple. I'm glad you like it!

[saladin1980]

Yea only just got into making servers for minecraft and well using a mix... ur is for backup and dropbox so i can have 2 IP's on differnet networks that are linked for world:) so no matter where they connect it saves the information accross them both or that is the plan:)

[Dateranoth]

VERY IMPORTANT !!!

Delete you Incremental Cache folder after the 1.3 update! The world save format has changed! You will be backing up the old format and new if you do not delete the cache.

On a good note, the file change reduced the file count enormously. So now even full backups are pretty fast.

[saladin1980]

yea i only do full backup was going to make a script to auto upload the backup to the ftp not just the images which i don't use anyways:) LOL

[Dateranoth]

I'm releasing this for testing as I won't have time to test it tonight myself. Once I've tested it myself I will release the final. Please report any bugs you find. Testing backup of entire SERVER directory and FTP upload of BACKUP files.

http://www.dateranot...package2.6B.zip

::V2.6-Added backup of entire Minecraft Server Directory. Refer to Variables below for -::

::-----options an instructions. Also added option to upload backups to FTP directory.--::

::-----These have been completely seperated in case overviewer and your other backups---::

::-----reside one two different servers. Refer to the variables for instructions.------::

BACKUP THE PREVIOUS VERSION! I HAVE NOT TESTED THIS AT ALL!

[saladin1980]

i will test this tonight so u will have guine pig so to speak ... i want to use this as.. well i am going to ftp the backups to my min server while the "game server" will be just doing game files:) that way saves room which costs more than on my webhosted server:) LOL if that makes sense?

[Dateranoth]

::V2.7-Fixed Incremental FTP uploads. The ftp folder is now successfully created.------::

::-----Allowing ftp backup to work as it should for incremental backups. Also fixed----::

::-----remote backup directory option. Wasn't placing backup in correct directory.-----::

::-----If using FTP upload of backup, incremental backups with a very large max is------::

::-----recommended. This way the first backup will be large and take time uploading----::

::-----but the following incremental backups will be much smaller and save time and-----::

::-----bandwidth.-----------------------------------------------------------------------::

Download the latest version from the link in the OP. There were problems with incremental uploads to ftp and they have been fixed.

Backup of config files by default has been removed as of 2.7 . You either have the option to backup just the world directory or the entire Minecraft directory. The exclude file can still be used to exclude certain files from the backup. Enjoy!

[Dateranoth]

::V2.8-Updated saveall.vbs to V1.2 - Condensed Keystroke Commands & Added Server--------::

::-----Backup notification. Download at www.gamercide.org/minecraft/MCBackup/saveall.zip::

::-----Change Cartograph to Cartograph G to support the new save format. Can be found at::

::-----http://www.minecraftforum.net/viewtopic.php?f=1022&t=165779&hilit=Cartograph+G --::

::-----Please refer to Readme.txt included with Cartograph G for map render settings----::

::-----Added option to change output directory of the Cart maps. Default is BackupDIR---::

Changed vbs script to work a little faster and also notify server that backup has started. You can change this simply by opening up the VBS file and changing the appropriate line or commenting it out.

Also, Cartograph maps are working with the new Region save format. You MUST UPDATE to Cartograph G for it to work!!!

You can find it here:

http://www.minecraftforum.net/viewtopic.php?f=1022&t=165779&hilit=Cartograph+G

Be sure to read the readme as the options have changed. Though you can still just use the defaults if you don't have a particular render that you want. The only difference in the script is it renders North as Up instead of East. Refer to the User Variables to change it.

I'm still waiting on a working Overviewer, but none of the options should change for it when it's released.

ENJOY!