steph/duplicity-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases 1 Wiki Activity

🎉 First version!

Browse Source
This commit is contained in:
Stephan Stanisic 2021-01-31 21:52:50 +01:00
commit c37a733313
10 changed files with 287 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored Normal file
View File

@ -0,0 +1 @@
services.conf

15
Dockerfile Normal file
View File

@ -0,0 +1,15 @@
FROM wernight/duplicity
USER root
VOLUME /services
# Setup files
COPY . /data
WORKDIR /data
# Setup cronjob
COPY crontab /etc/cron.d/backup-cron
RUN chmod 0644 /etc/cron.d/backup-cron
RUN crontab /etc/cron.d/backup-cron
CMD ["crond", "-f"]

24
LICENSE Normal file
View File

@ -0,0 +1,24 @@
This is free and unencumbered software released into the public domain.
Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.
In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
For more information, please refer to <http://unlicense.org/>

20
backup.conf.example Normal file
View File

@ -0,0 +1,20 @@
## Backup config file for service. To find the syntax for this file,
## take a look at `man duplicity` or the web docs at this url:
## http://duplicity.nongnu.org/vers8/duplicity.1.html#sect9
##
## Special syntax has been added, a single pound sign line will be
## interpreted like a variable assignment, a line with two pound
## signs will be seen as comments.
## This make daily backups, and keep them for one months, making
## incremental backups and doing a full backup once a week.
# RETAIN=1M
# INCREMENT=1W
# INTERVAL=1D
## The current directory (where this file is in) will get included,
## but the subdirectory mnt will be excluded. (** matches the working
## directory)
+ **
- **/mnt

140
backup.sh Executable file
View File

@ -0,0 +1,140 @@
#!/bin/sh
BASE="/services"
SERVICES=$(ls $BASE)
echo "#### STARTING NEW BACKUP OF ALL SERVICES ####"
echo ""
echo " SERVICES IN TRACKER:"
for SERVICE in $SERVICES; do
echo " - ${SERVICE}"
done
echo ""
echo " BACKUP STARTED ON $(date)"
echo ""
echo ""
echo ""
for SERVICE in $SERVICES; do
LOCAL="$BASE/$SERVICE"
# Skip service if it doesn't have an backup config specified
if [ ! -f "$LOCAL/backup.conf" ]; then
continue
fi
RETAIN="1M"
INCREMENT="1W"
INTERVAL="1H"
# Read config for extra options
while IFS='=' read -r key val; do
[ "${key##\#\#*}" ] || continue # Skip if line starts with two hash signs
[ ! "${key##\#*}" ] || continue # Skip if line doesn't start with a hash sign
key="${key##\#}" # Strip the hash sign
# Key isn't escaped so the space after the hash sign gets consumed
# by the assignment. This way the space is optional
export $key="$val" 2>/dev/null ||
printf ' warning %s is not a valid variable name\n' "$key"
done < "$LOCAL/backup.conf"
echo "==== BACKING UP $SERVICE ===="
echo ""
echo " Retaining backups for $RETAIN"
echo " Maxiumum incremental backup duration is $INCREMENT"
echo " Running every $INTERVAL"
echo ""
INTERVAL_INT=1
case $INTERVAL in
*M) INTERVAL_INT=$(( ${INTERVAL%%M} * 24 * 61 / 2 ));;
*W) INTERVAL_INT=$(( ${INTERVAL%%W} * 24 * 7 ));;
*D) INTERVAL_INT=$(( ${INTERVAL%%D} * 24 ));;
*H) INTERVAL_INT=$(( ${INTERVAL%%H} ));;
*) INTERVAL_INT=1;;
esac
if [ $(( ( $(date +%s) / 3600 ) % $INTERVAL_INT )) -ne 0 ]; then
echo "==== SKIPPING $SERVICE"
echo " Service not yet scheduled for backup."
echo ""
echo ""
echo ""
echo ""
echo ""
continue
fi
echo ""
while read REMOTE; do
# Skip if line is empty
[ -z "$REMOTE" ] && continue
REMOTE="$REMOTE/$SERVICE"
TYPE=$(echo "$REMOTE" | cut -d \: -f 1)
SERVER=$(echo "$REMOTE" | cut -d \: -f 3 | cut -d \@ -f 2 | cut -d \/ -f 1)
echo "==== USING REMOTE:"
echo " $SERVER over $TYPE"
echo ""
echo " > BACKUP"
duplicity \
--verbosity 2 \
--no-encryption \
--allow-source-mismatch \
--full-if-older-than "$INCREMENT" \
--include-filelist "$LOCAL/backup.conf" \
"$LOCAL" "$REMOTE" \
2>&1 | sed 's/^/ /'
echo " > DELETE OLD"
duplicity \
--verbosity 2 \
remove-older-than "$RETAIN" \
--no-encryption \
--allow-source-mismatch \
--include-filelist "$LOCAL/backup.conf" \
--force \
"$REMOTE" \
2>&1 | sed 's/^/ /'
echo ""
echo " > VERIFY"
duplicity \
verify \
--verbosity 2 \
--no-encryption \
--allow-source-mismatch \
--include-filelist "$LOCAL/backup.conf" \
"$REMOTE" "$LOCAL" \
2>&1 | sed 's/^/ /'
if [ $? -ne 0 ]; then
echo " [!] SOMETHING BAD HAPPENED"
echo " Backups where made but don't check out. Is the remote's storage full?"
fi
echo ""
echo ""
done < "/data/services.conf"
echo ""
echo ""
echo ""
done
echo "#### BACKUP FINISHED ####"
echo ""
echo " ENDED ON $(date)"
echo ""
echo ""
echo ""
echo ""
echo ""
echo ""
echo ""
echo ""
echo ""

1
crontab Normal file
View File

@ -0,0 +1 @@
10 * * * * /data/backup.sh 2>/proc/1/fd/2 >/proc/1/fd/1

10
docker-compose.yml Normal file
View File

@ -0,0 +1,10 @@
version: '3'
services:
backup:
build: .
volumes:
- /services:/services
- ./mnt/duplicity/.cache:/home/duplicity/.cache
- ./mnt/duplicity/.gnupg:/home/duplicity/.gnupg
restart: always

2
mnt/duplicity/.gitignore vendored Normal file
View File

@ -0,0 +1,2 @@
.cache/
.gnupg/

72
readme.md Normal file
View File

@ -0,0 +1,72 @@
# Automatic service backup
This docker container will backup services on a schedule, using duplicity as an back end.
## How to configure
To configure file hosting locations you can add storage locations to the `services.conf` file. One per line, you can add duplicity connections. The syntax of these can be found in [their docs](http://duplicity.nongnu.org/vers8/duplicity.1.html).
After changing the `services.conf` file, you'll have to rebuild the container.
The script sends logs to the docker log socket (as is usual with containers).
```
docker-compose build
docker-compose up -d
docker-compose logs -f
```
## How to use
This docker container assumes that you use the following directory structure:
```
some_folder/
service1/
backup.conf
service2/
backup.conf
service3/
```
By default, `some_folder` is mapped to `/services` as this is where my services reside. You can change this in the `docker-compose.yml` file.
If you put a `backup.conf` file into a service's directory it will get backed up. If you omit this file, the service will be skipped.
The contents of the backup file are a duplicity include list. You can find more documentation about them at [their docs](http://duplicity.nongnu.org/vers8/duplicity.1.html#sect9)
You can find an example file with some comments at `backup.conf.example`.
### Backup file syntax
Lines starting with `##` are comments.
Lines starting with `#` are interpreted as variable assignments. There are three variables used to control backup behaviour.
```
## These are also the default values
# RETAIN=1M
# INCREMENT=1W
# INTERVAL=1H
```
`RETAIN` specifies how long backups should be kept around.
`INCREMENT` specifies how long incremental backups are made before creating a full backup again. This will make a full backup every week.
`INTERVAL` specifies how long between backups.
The `RETAIN` and `INCREMENT` variables are directly fed into duplicity and use their syntax. The `INTERVAL` variable is read by the container, and has the following options:
```
1M for once each month
1W for once each week
1D for once each day
1H for once each hour
2H for once each two hours
6H for once each six hours
12H for twice each day
```
Values must be whole integers, and the minimal value possible is `1H`.

2
services.conf.example Normal file
View File

@ -0,0 +1,2 @@
webdavs://nextcloudusername:nextcloudpassword@nextcloudhost/remote.php/webdav/path/to/folder
b2://b2keyId:b2applicationKeyGenerateOneWithoutSlashesJustKeepTrying@b2BucketName/path/to/folder