HowTo: NExtFW-Paket backup-box (Beta) auf AMS/SBLAN2

TIPPS UND TRICKS, FAQ

Moderator: ModTeam

Deine Meinung zum NExt backup-box Paket

Ich setze backup-box heute ein
3
25%
Ich werde es installieren, wenn die Version nicht mehr Beta ist
1
8%
Momentan brauche ich backup-box nicht, vielleicht später mal
8
67%
Ich habe kein Interesse an backup-box
0
No votes
Ich werde keine modifizierten Firmware-Extensions einsetzen
0
No votes
 
Total votes: 12

User avatar
lochness
Posts: 3224
Joined: Sat Feb 04, 2006 20:42
Product(s): Myrica TV P42-2H
AMC 570 SAT (3.2.419 + VZD2)
AMC 370 SAT (1.6.115+VZD2)
AMP 150 (1.03.28eu)
3 x AMS 150 (V2.00 + NExt V2.0.06)
SBLAN2 R1.0 (V2.00 + NExt V2.0.06)
SBLAN2 R1.1 (V2.00 + NExt V2.0.06)
Storagebird DUO 35-LR (V1.03)
SB-L V1 (80GB, NetHDD007-0802)
Celvin Q800 4x2TB
Celvin Q700 2x1TB
Celvin Q600 1x2TB
Location: Bad Lippspringe

HowTo: NExtFW-Paket backup-box (Beta) auf AMS/SBLAN2

Postby lochness » Tue Aug 12, 2008 22:39

==================================================================================
Download: zurückgezogen! / withdrawn!
Aktuelles Paket: NEXT_backup-box--0.9k--FSC_231.tgz-FSC_222.tgz
Aktuelles Readme: NEXT_backup-box--0.9k--readme.txt

==================================================================================

Beschreibung folgt, von "iwan". Feedback zu diesem Paket bitte in diesem Thread oder direkt an den Autor.
Last edited by lochness on Fri Jul 08, 2011 6:45, edited 3 times in total.

iwan
Posts: 7
Joined: Thu May 08, 2008 16:14
Product(s): SB-Lan2, Rev1.1, V1.01 + Ext1.1n (500GB) - primary
SB-Lan2, Rev1.1, V2.00ß + Ext2.23n (500GB) - backup
AMS-150, Rev1.1, V2.00ß2 + Ext2.23n (500GB)

HowTo: NExtFW-Paket backup-box (Beta) auf AMS/SBLAN2

Postby iwan » Tue Aug 12, 2008 23:10

This software is intended to be used with the "NExt" Firmware Extensions for AMS150/SBLAN2 NAS devices only. You can find further information about NExtFW in the FSC Digital Home Forum
http://forum.ts.fujitsu.com/digital ... =58&t=7471 and at http://nextfw.sourceforge.net/.

NOTE:
To use this package you will need to install the NExtFW base package first!

HOW TO INSTALL THIS PACKAGE:
The package can be installed via the Admin-GUI of the device ("Maintenance" -"load firmware extensions") or via network share. You can reach the Admin-GUI at http://sblan2/ for SBLAN2 devices or at http://ams150/ for AMS150 devices. Version information for this package is displayed via the "NExtFW" status page which you can reach via Admin-GUI right after you have installed the base package.

HOW TO UN-INSTALL THIS PACKAGE:
Re-installing the original Firmware Extensions by FSC will not automatically de-install the "NExt" Firmware Extensions completely. As of now, you can only de-install all NExtFW packages in one go over the Web interface (see above). In order to do so, please, install the NExtFW package "cleanup" package also available with the other packages or re-format the hard disk (loosing all data!) to get rid of the Firmware Extensions at all.

PACKAGE CONTENTS:
This package provides local and remote backups of AMS/SBLAN2 devices.
As of now, it does not provide functions to do backups to / from external USB or SATA disks.

PREREQUISITES:
NONE except the NExtFW "base" package.

ISSUES AND RESTRICTIONS:
By default, this package creates daily backup snapshots of the local Samba shares, which are exported as a new Samba share named [.BACKUP]. Disk space for daily snapshots consists of space for directories, symlinks, and changed files, only. For more details, please, see the Mini Description of "backup-box".

This software is provided AS-IS with no warranty of functionality. We cannot be held responsible for any damage that may occur after installing this extension. You use this Software at your own risk.

Have fun,
iwan and the NExtFW team

iwan
Posts: 7
Joined: Thu May 08, 2008 16:14
Product(s): SB-Lan2, Rev1.1, V1.01 + Ext1.1n (500GB) - primary
SB-Lan2, Rev1.1, V2.00ß + Ext2.23n (500GB) - backup
AMS-150, Rev1.1, V2.00ß2 + Ext2.23n (500GB)

README: NExtFW-Paket backup-box (Beta) auf AMS/SBLAN2

Postby iwan » Tue Aug 12, 2008 23:42

backup-box is yet another rsync-based backup tool.
It is NOT meant to provide backup/miror functionality with external
USB/SATA disks. Instead, it provides backups of local shares or backups
from another remote network server running an rsync daemon
(see also package 'rsyncd').

By default, backup-box creates local backups of all Samba shares in a
new share named '.BACKUP', where it keeps full backup snapshots of all
shares for each day for long-term backup. Backups are started by cron
once a day.

With proper configuration settings backup-box can also mirror and backup
directories FROM other rsync servers which provide a share (module) named
'.DISK' (see also package 'rsyncd').

A backup snapshot consists of hardlinked copies of unchanged files and
copies of all directories, symbolic links, and changed files. Hence, only
changed files and directories require additional disk space.

In order to get access to the backup snapshots you need to define another
share with name '.BACKUP'.

Script developers might like the easy access to internal functions, which
is available via the option '--run' or by symlinking the name of an internal
function to the 'backup-box' executable. (Thus, the name 'backup-box'.)


BACKUP MANAGEMENT

Mirroring and backups are controlled by cron, see /etc/cron.d/backup-box.
This package-specific crontab is also visible under share '.BACKUP'.
It is named '.backup-box.cron' there. A crontab change in share '.BACKUP'
requires a reboot to become effective.

While backups (mirroring) may run multiple times a day,
each snapshot is maintained in a sub-directory with naming pattern

Code: Select all

   $BACKUP_ROOT/$BACKUP_DIR/$BACKUP_HOST/FULL/$BACKUP_DATE

$BACKUP_DATE is a format string, which is used with date(1) to define
the actual time stamp. By default, it reflects the current date, only.

Code: Select all

   BACKUP_DATE="%Y-%m-%d--%H-%M-%S"   # format of date(1)-stamp
   BACKUP_DATE="%Y-%m-%d--%H"      # ...keep one snapshot per hour
   BACKUP_DATE="%Y-%m-%d"         # ...keep one snapshot per day

The number of retained backup snapshots depends on the setting of parameter
BACKUP_DATE and the frequency of backup runs.


There are various options to manage the disk space used for backups[/]

Code: Select all

    (1) use all unused disk space up to a certain threshold (BACKUP_DISK_*)
    (2) define maximum number of snapshots to keep (BACKUP_HIST_MAX)
    (3) define maximum disk space for all snapshots (BACKUP_POOL_MAX)

The various parameters are used by different clean-up procedures which are
applied in the above indicated order.


CONFIGURATION OPTIONS

Various features of backup-box are controlled by internal parameters,
which can be provided on the command line and by config files.

The following loading sequence applies[/]

Code: Select all

   <built-in defaults>
   /etc/backup-box
   /mnt/.backup-box
   $HOME/.backup-box
   ./.backup-box
   <command-line>

Note the missing leading '.' for the config file located in /etc.
Config file /mnt/.backup-box is also visible in share '.BACKUP'.

Under Windows you might need to change your Explorer settings to show also
'hidden' files in order to see the configuration files '.backup-box' and
'.backup-box.cron'.


The automatically generated configuration file is produced by command

Code: Select all

   shell> backup-box  --run backup-defaults  [ file ]

It lists all available configruation options with short usage remarks:

Code: Select all

#
#   .backup-defaults @ froggle
#
#   created on 2008-05-14 21-21-34 by backup-defaults
#

   RSYNC_PATH="rsync"         # path to rsync(1)
   RSYNC_PARTIAL="true"         # keep partial transfers
   RSYNC_TIMEOUT="20"         # I/O timeout in seconds
   RSYNC_VERBOSE="false"         # don't be verbose
   RSYNC_DRY_RUN="false"         # do NOT transfer files
   RSYNC_BWLIMIT=""         # bandwidth limit [kbps]

   RSYNC_UPDATE="true"         # keep newer files
   RSYNC_DELETE="true"         # delete obsolete files
   RSYNC_MODULE=".DISK"         # name of remote rsync module
   RSYNC_PASSWD=""            # password file for auth.
   RSYNC_FILTER=""            # rsync filter file


   BACKUP_COPY="false"         # make data copy/mirror of src
   BACKUP_FULL="true"         # make full backup snapshots

   BACKUP_HOST="localhost"         # FQDN/IP of remote NAS/rsyncd
   BACKUP_ROOT="/mnt"         # local dir with SMB shares
   BACKUP_DIR=".BACKUP"         # name of backup directory

   BACKUP_LOG="false"         # redirect output to a log file
   BACKUP_LOG_TIME="false"         # ... tag output with timestamp
   BACKUP_LOG_GZIP="true"         # ... compress backup logs

   BACKUP_DATE="%Y-%m-%d--%H-%M-%S"   # format of date(1)-stamp
   BACKUP_DATE="%Y-%m-%d--%H"      # ...keep one snapshot per hour
   BACKUP_DATE="%Y-%m-%d"         # ...keep one snapshot per day

   BACKUP_HIST_MIN="3"         # minimum num of backups to keep
   BACKUP_HIST_MAX="99"         # maximum num of backups to keep

   BACKUP_POOL_MAX=""         # maximum GB for backup space
   BACKUP_DISK_MINFREE="60"      # minimum GB free space
   BACKUP_DISK_PERCENT="75"      # maximum percent used space

   BACKUP_SMB_LIKE="Web"      # export '.BACKUP' like this share
   BACKUP_PERM_UMASK="0000"   # umask of new dir/files
   BACKUP_KILL="true"      # kill backup on 'stop' via webadmin

   BACKUP_LIST=""            # list of default shares
                  # ... defaulting to all shares

#
#   the End ...

Beware:
Use of BACKUP_POOL_MAX_GB is SLOW !
It requires a full scan of the disk in order to avoid double counting of
hardlinks. Use this option with care.

Some of the backup parameters take boolean values, i.e. they must be set
to either "true" or "false".

You can define a list of directories to backup by listing them in variable
BACKUP_LIST. The directory names must be separated by spaces. If a directory
names contains spaces, it must be enclosed by single quotes, e.g.

Code: Select all

   BACKUP_LIST="'Hello World' 'How are you'"

Beware the quoting!



COMMAND LINE USAGE

'backup-box' has been developped for FSC AMS-150 / SBLAN2,
but it is also supposed to run under a fully featured Linux or in any
other descent POSIX environment like eg. Win-XP with CygWin-1.5

It aims to support path names with embedded spaces,
thus, you'll find excessive quoting in this script.

The various backup-related functions can be invoked on the command
line by option '--run'. This script is also a multi-call executable, thus,
you may symlink the name of various internal functions to this executable.

The 2 most important command line uses are the following[/]

Code: Select all

   shell> backup-box  --help
   cron>  backup-box  --verbose --backup-log true

In order to cope with the pecularities of AMS-150 / SBLAN2 backup-box
is installed and launched with the helper script named 'backup-ams.sh',
which is also used as start script in the AMS environment.



AVAILABLE FUNCTIONS AND OPTIONS OF 'backup-box'

Code: Select all

backup-box      [ -h ]      # mini help
         [ -v ]      # rsync: verbose
         [ -q ]      # rsync: quiet mode
         [ -n ]      # rsync: dry run
         [ -R host ]   # rsync: server to backup
         [ -M name ]   # rsync: module name (.DISK)

         [ -m ]      # mirror only / no backup
         [ -c file ]   # load config file
         [ --help  ]   # this help
         [ --arg=value]   # set  internal parameter 'arg' to value
         [ --run func ]   # exec internal function
         [ dir ...  ]   # directories / shares to backup

INTERNAL FUNCTIONS AVAILABLE VIA OPTION '--run'

Code: Select all

backup-defaults   [ output-file ]   # create defaults file
backup-install            # install commands & rsyncd
backup-install-cmd         # install commands only
backup-install-cron         # install crontab only
backup-install-rsyncd         # install rsyncd only

backup-uninstall         # uninstall commands & rsyncd
backup-uninstall-cmd         # uninstall commands only
backup-uninstall-cron         # uninstall crontab only
backup-uninstall-rsyncd         # uninstall rsyncd only

backup-disk      [ share ]   # print df(1) of backup disk
backup-disk-dev      [ share ]   # print /dev of backup disk
backup-disk-size   [ share ]   # print size of backup disk
backup-disk-used   [ share ]   # print used blocks
backup-disk-avail   [ share ]   # print available blocks
backup-disk-percent   [ share ]   # print percentage used
backup-disk-dir      [ share ]   # print mount point

backup-check-disk         # backup disk size <= limits ?
backup-check-disk-minfree      # BACKUP_DISK_MINFREE obeyed ?
backup-check-disk-percent      # BACKUP_DISK_PERCENT obeyed ?
backup-check-pool-max         # BACKUP_POOL_MAX     obeyed ?
backup-check-hist-max         # BACKUP_HIST_MAX     obeyed ?
backup-check-hist-min         # BACKUP_HIST_MIN     obeyed ?

backup-cleanup      [ share ... ]   # clean-up backup data
backup-cleanup-history   [ share ... ]   # num backups<= $BACKUP_HIST_MAX
backup-cleanup-disk   [ share ... ]   # ensure required disk space
backup-cleanup-disk-minfree  [ ... ]   # minimum free space
backup-cleanup-disk-percent  [ ... ]   # obey BACKUP_DISK_PERCENT
backup-delete-history      [ count | date ... ]   # delete history
backup-cleanup-pool-max      [ ... ]   # size backups <= $BACKUP_POOL

backup-LED   {backup | full | fan}   {on | off}
backup-LED-blink    {full | fan}   [on-delay]  [off-delay]
backup-LED-blink-stop   [ pid ]

backup-history      [ share ... ]   # print backup history
backup-size      [ share ]   # print size of backup pool
backup-list-size   [ share ... ]   # print backup size of shares
backup-list-files   [ share ]   # print unique file revisions

backup-list-local      [ smb.conf  ]   # print list of  local dirs
backup-list-remote     [ -R server ]   [ -M module ]  [ local-dir ... ]

backup-simple        share ...   # make INCRemental backup
backup-full        share ...   # make FULL backup

backup         [ share ... ]   # backup according to ~/.$0
backup-box      [ share ... ]   # ... defaults to all shares
mirror-box      [ share ... ]   # mirror only, no retention


MISCELLANEOUS REMARKS

The "realtime" clock of an AMS-150 / SBLAN2 lags 13-18 minutes behind
per HOUR. Hence, it is recommended to uncomment the corresponding entry
in the provided CRONTAB if there is a flat-rate Internet connection available
in order to regularly adjust the clock via NTP.


If you want to use backup-box to backup data from multiple servers, then
you need to issue a backup command for every individual server. You might
want to define a separate configuration file for each server (see option '-c')
or overwrite certain parameters of a generic configuration file on the
command line using the syntax '--VARIABLE=VALUE' or '--VARIABLE' 'VALUE'.


By default, the default BACKUP_DIR ".BACKUP" is NOT a visible share.
Parameter BACKUP_SMB_LIKE defines an existing share is used as a template
to export ".BACKUP". After uninstalling backup-box and rebooting the box
the share ".BACKUP" will not be visible any more although it still exists.
In order to make it visible again simply use the management web interface
and add a new share named ".BACKUP".


GERMAN SUMMARY

Das Paket Backup-Box wurde getestet mit NEXTFW-2.0.99 base auf FW-v1.01
und FW-v2.0final für AMS-150 und SBLAN2.

Im AuslieferungsZustand erzeugt es via CRON tägliche Backup-Snapshots aller
lokalen Shares. Die lokale Platte speichert dabei bis zu 99 Snapshots oder
adaptiv weniger, damit die Platte nicht den Füllgrad von 75% übersteigt.

Mindestens 3 Snapshots werden jedoch immer vorgehalten.
Ein Backup-Snapshot verbraucht dabei nur Plattenplatz für Directories,
Symlinks und *geänderte* Dateien.

Die Backup-Snapshots sind mittels eines neuen Shares namens [.BACKUP]
sichtbar. Die Zugriffsrechte werden dabei vom Share [Web] übernommen.
Die "SpeicherVerwaltung" der Backup-Snapshots wird über ein Konfig-File
namens '.backup-box' eingestellt, das ebenfalls wie die zugehörige CRONTAB
im Share [.BACKUP] abgelegt ist ('.backup-box.cron').

Die CRONTAB im Share [.BACKUP] liegt im Windows-Textformat vor, das vom
"InstallationsSkript S75backup-ams.sh" für Linux entsp. umkonvertiert wird.

Das Default-Konfig-File ".backup-box" wird ebenfalls im Windows-Textformat
erzeugt (dabei wird vorerst billigend in Kauf genommen, daß Leerzeilen darin
Fehlermeldungen werfen).

Wenn im Konfig-File anstelle BACKUP_HOST="localhost" ein anderer Host
eingetragen ist, so erwartet das Backup-Skript dort einen Rsync-Daemon
(siehe Paket "rsynd"). Für alle lokalen Verzeichnisse unter /mnt
(= Samba Shares) sucht es dann ein Pendant im Remote Rsync Module [.DISK].

Die Liste der gesicherten Verzeichnisse kann alternativ durch die
Konfigurationsvariable BACKUP_LIST festgelegt werden. Mehrere Verzeichnisse
werden dabei durch Blank getrennt. Wenn ein Verzeichnis Blanks im Namen
trägt, dann muß der Verzeichnisname in *einfache* Anführungszeichen gesetzt
werden, z.B.

Code: Select all

   BACKUP_LIST="'Hello World' 'how are you'"

Die doppelten Anführungszeichen zur Abgrenzung des Variablenwerts und die
gleichzeitige Verwendung von einfachen Anführungszeichen zur Abgrenzung der
Verzeichnisnamen muß wie im obigen Beispiel eingehalten werden.


backup-box ist nach seiner Installation durch sein Default-Konfig-File
sofort aktiv. Es startet jeweils um 23:57h. Andere Startzeitpunkte können
in der CRONTAB im Share [.BACKUP] eingetragen werden.


Der Parameter BACKUP_DATE definiert den Namen des Backup-Verzeichnisses
für einen bestimmten Backup-Lauf. Der Name des Verzeichnisses wird dabei
wie folgt bestimmt[/]

Code: Select all

   DIR="$BACKUP_ROOT/$BACKUP_DIR/$BACKUP_HOST/FULL/`date +'$BACKUP_DATE'`"

Soll heißen: Der Wert des Parameters BACKUP_DATE ist ein Formatstring für
das Kommando date(1), mit dem der "Zeitstempel-Name" des jeweiligen
Backup-Unterverzeichnisses ermittelt wird.


Die Zahl der tatsächlich gespeicherten Backup-Snapshots ergibt sich aus dem
Zusammenspiel von Häufigkeit der Backup-Läufe und der "Genauigkeit" des
generierten Zeitstempels, z.B.[/]

Code: Select all

   BACKUP_DATE="%Y-%m-%d"

Diese Einstellung erzeugt einen Datums-String im ISO-Format. Alle Backup-Läufe
eines Tages verwenden dann das gleiche (aber nicht dasselbe) Unterverzeichnis
für die Backup-SnapShots, so daß nur die Daten des letzten Backup-Laufes
gespeichert bleiben.


Ein CRONTAB-Eintrag der Form

Code: Select all

   57 0,3,6,9,12,15,18,21 * * * root /mnt/.zap/usr/bin/backup-ams.sh cron

startet alle 3 Stunden zur 57. Minute einen Backup-Lauf. In Verbindung mit
der Parameter-Einstellung

Code: Select all

   BACKUP_HOST="%Y-%m-%d--%H-%M"

werden Zeitstempel mit Uhrzeit (hier: Stunde und Minute) erzeugt, so daß
bei jedem Backup-Lauf ein neues Unterverzeichnis für die Backup-Daten
angelegt wird und erhalten bleibt.


Die "Echtzeit-Uhr" eines AMS-150 / SBLAN2 weicht je nach Last zwischen
13-18 Minuten pro STUNDE ab, daher sollte bei Vorhandensein einer Internet-
Anbindung mit FlatRate der Eintrag zur regelmäßigen Uhrzeitkorrektur mittels
NTP in der mitgelieferten CRONTAB auskommentiert werden.

Wenn die Daten von mehreren Servern auf dem NAS gesichert werden sollen,
dann ist für jeden Server ein separater Aufruf der backup-box erforderlich.
Dabei kann für jeden Server ein eigene Konfigurationsdatei mittels der
Option '-c' geladen werden. Alternativ können auf der KommandoZeile einzelne
Parameter einer generischen Konfigurationsdatei mittels der Syntax
'--VARIABLE=VALUE' oder '--VARIABLE' 'VALUE' überschrieben werden.

iwan
Posts: 7
Joined: Thu May 08, 2008 16:14
Product(s): SB-Lan2, Rev1.1, V1.01 + Ext1.1n (500GB) - primary
SB-Lan2, Rev1.1, V2.00ß + Ext2.23n (500GB) - backup
AMS-150, Rev1.1, V2.00ß2 + Ext2.23n (500GB)

README: NExtFW-Paket backup-box (Beta) auf AMS/SBLAN2

Postby iwan » Tue Aug 12, 2008 23:44

(reserved for more documentation)

User avatar
lochness
Posts: 3224
Joined: Sat Feb 04, 2006 20:42
Product(s): Myrica TV P42-2H
AMC 570 SAT (3.2.419 + VZD2)
AMC 370 SAT (1.6.115+VZD2)
AMP 150 (1.03.28eu)
3 x AMS 150 (V2.00 + NExt V2.0.06)
SBLAN2 R1.0 (V2.00 + NExt V2.0.06)
SBLAN2 R1.1 (V2.00 + NExt V2.0.06)
Storagebird DUO 35-LR (V1.03)
SB-L V1 (80GB, NetHDD007-0802)
Celvin Q800 4x2TB
Celvin Q700 2x1TB
Celvin Q600 1x2TB
Location: Bad Lippspringe

Re: HowTo: NExtFW-Paket backup-box (Beta) auf AMS/SBLAN2

Postby lochness » Sat Oct 11, 2008 14:01

Noch eine Bitte vom NExt Team. Bitte stimmt oben bei der Umfrage mit ab. Das wird uns helfen, die Prioritäten für euch passend zu setzen.

lispy
Posts: 5
Joined: Wed Oct 08, 2008 19:40
Product(s): AMS150

Re: HowTo: NExtFW-Paket backup-box (Beta) auf AMS/SBLAN2

Postby lispy » Mon Mar 13, 2017 11:32

Oh Boy, I know it's probably waaaay too late, but this extension is exactly what I'm looking for. Is there some way to obtain the last release? I want to backup my website to my ams.

Support is not needed, I can fix some stuff and if it's broken beyond repair I will have to get my own rsync backup. It would be great for getting started and maybe save me some hassel. :)


Return to “Hints, FAQ”

Who is online

Users browsing this forum: No registered users and 2 guests

cron