cPanel (new) Backup doesn't save those deleted accounts
Moderators: BBear, theunknownhost, flaguy
- Arf
- Official Test Penquin
- Posts: 9103
- Joined: Tue Apr 09, 2002 12:00 am
- Location: IDAHO, USA
- Contact:
cPanel (new) Backup doesn't save those deleted accounts
I'm finally transitioning from the old backup to the new. I've found a difference that for me is a cause for concern. The old backup system used to not delete any backup files from deleted accounts. This was handy if the client returned and wanted their account restored. The new system deletes all the old back up files.
I've been writing a utility that archives a copy of any deleted accounts by putting them in a safe place both on the server and on a remote backup server if you're using one.
Is this of any concern to anyone else here? Would such a utility be handy?
BTW - the new backup system is great.
I've been writing a utility that archives a copy of any deleted accounts by putting them in a safe place both on the server and on a remote backup server if you're using one.
Is this of any concern to anyone else here? Would such a utility be handy?
BTW - the new backup system is great.
Re: cPanel (new) Backup doesn't save those deleted accounts
Oh man... this is a concern for me. I have had (and promised) clients who left come back want their old data restored, say a month or two later.
I'd be interested in such an utility but not in a hurry. On a scale of 1-10, with 10 being the most urgent, I say 4.
I'd be interested in such an utility but not in a hurry. On a scale of 1-10, with 10 being the most urgent, I say 4.
- Arf
- Official Test Penquin
- Posts: 9103
- Joined: Tue Apr 09, 2002 12:00 am
- Location: IDAHO, USA
- Contact:
Re: cPanel (new) Backup doesn't save those deleted accounts
I'm beta testing the software now and it's worked well enough that I'm beginning to roll it out of my second production server. In addition to keeping track of deleted accounts it gives a report on any errors that occurred during backup (there are a lot of things that new/old cpanel backup skipped, mostly do to permissions issues, and people don't realize it) as well as a log of the transfers to the remote backup server if you have one. It also duplicates the back up of deleted accounts on the backup server for safe keeping. I'm a pack rat and have accounts dating back to 2010.
Anyway, so far it's going well. Hopefully there will be more interest by someone. Either way, I'll be happy to post the code.
Anyway, so far it's going well. Hopefully there will be more interest by someone. Either way, I'll be happy to post the code.
- Arf
- Official Test Penquin
- Posts: 9103
- Joined: Tue Apr 09, 2002 12:00 am
- Location: IDAHO, USA
- Contact:
Re: cPanel (new) Backup doesn't save those deleted accounts
Almost done.
I learned this: If you're not checking for errors in /usr/local/cpanel/logs/cpbackup/* then you probably don't realize that stuff is not getting backed up. Usually permissions issues. The program will send you errors. Chances are about 100% that you have errors and don't know it.
It will also tell you about your disk usage, and if a backup didn't occur the day before.
I learned this: If you're not checking for errors in /usr/local/cpanel/logs/cpbackup/* then you probably don't realize that stuff is not getting backed up. Usually permissions issues. The program will send you errors. Chances are about 100% that you have errors and don't know it.
It will also tell you about your disk usage, and if a backup didn't occur the day before.
- Arf
- Official Test Penquin
- Posts: 9103
- Joined: Tue Apr 09, 2002 12:00 am
- Location: IDAHO, USA
- Contact:
Re: cPanel (new) Backup doesn't save those deleted accounts
OK, I've been using this for a few weeks on 5 servers. All seems to be working fine. I included as many comments as needed to get a good idea of that the program does and how it works.
Program name: archive.mgr.sh
Program name: archive.mgr.sh
Code: Select all
#!/bin/bash
VERSION="1.1"
# v 1.1 2/3/15 moved disk quota to bottom.
# v.0.1.1 12/24/14
# This program prevents the loss of backup files for accounts that have been deleted
# on a cPanel server. It also sends a report of the current backup
# Author: Thomas Leo :)
##################################################################################
# Documentation at the bottom! #
##################################################################################
############### NO NEED TO EDIT BELOW THIS LINE ##################################
# Make sure $1 is the config file and that the file exists.
if [ "$1" = "" ]; then echo "Syntax ERROR. Use: $0 Archive.host.cfg"; exit; fi
if [ ! -f $1 ]; then echo "ERROR: Cannot find config file: $1"; exit; fi
# Read in the config file.
source $1 # Get the config file info
if [ "$ADMIN" = "" -o "$TARGETUSERNAME" = "" -o "$TARGETHOST" = "" -o "$TARGETBUPATH" = "" -o "$LOCBUDIR" = "" -o "$KEYFILE" = "" ]; then
echo "ERROR: One or more of the variables needed in $1 is empty or blank."
exit
fi
TODAY=`date +%Y-%m-%d`;
YESTERDAY=`date +%Y-%m-%d --date="yesterday"`
LOGFILE="$LOCBUDIR/logs/backup-$TODAY.log"
if [ ! -d $LOCBUDIR/logs/ ]; then mkdir $LOCBUDIR/logs/ ; fi
> $LOGFILE # Clear the log file just in case.
# This section delays the backup for 30 minutes until cpbackup stops running.
BACKUPCMD="/usr/local/cpanel/bin/backup"
BUSY=`ps ax | egrep -v "(cpuwatch|grep)" | grep -c "$BACKUPCMD"`
while [ $BUSY -gt 0 ]; do
echo "Notice: backup is waiting for cpanel backups to stop. `date`" >> $LOGFILE
ps ax | egrep -v "(cpuwatch|grep)" | grep $BACKUPCMD >> $LOGFILE
sleep 1800 #sleep for 30 minutes
BUSY=`ps ax | egrep -v "(cpuwatch|grep)" | grep -c "$BACKUPCMD"`
done
#Log Rotation:
TENDAYSAGO=`date +%Y-%m-%d --date="10 days ago"`
touch --date "$TENDAYSAGO" /tmp/logs.txt
find $LOCBUDIR/logs/ -type f -not -newer /tmp/logs.txt | xargs -i rm -f {}
#### ARCHIVE ANY ACCOUNTS THAT HAVE BEEN DELETED SINCE YESTERDAY.##############################
echo -e "\n==== ARCHIVE TERMINATED ACCOUNTS =================================" >> $LOGFILE
echo " The following is a log of accounts that have been deleted since " >> $LOGFILE
echo " yesterday. .These files will be stored in the following directory:" >> $LOGFILE
echo -e " $TARGETHOST:$TARGETBUPATH/backup.archive\n" >> $LOGFILE
# Idiot checking
if [ ! -d $LOCBUDIR/$YESTERDAY/accounts ]; then
echo "**** ERROR: $LOCBUDIR/$YESTERDAY/accounts not found. ****" >> $LOGFILE
else
if [ ! -d $LOCBUDIR/$TODAY/accounts ]; then
echo "**** ERROR: $LOCBUDIR/$YESTERDAY/accounts not found ****" >> $LOGFILE
else
# End Idiot checking
# Get lists of files from TODAY and from YESTERDAY.
find $LOCBUDIR/$YESTERDAY/accounts -name "*.tar.gz" -type f -printf "%f\n" > /tmp/YESTERDAYFILELIST
find $LOCBUDIR/$TODAY/accounts -name "*.tar.gz" -type f -printf "%f\n" > /tmp/TODAYFILELIST
# Find the files that were there yesterday, but not there today ....
grep -v -f /tmp/TODAYFILELIST /tmp/YESTERDAYFILELIST > /tmp/DELETEDFILES # = the files that need to be archived from YESTERDAY.
# Check if /tmp/DELETEDFILES has NO DATA (-s) in it. If it has data, log and archive the files
if [ -s /tmp/DELETEDFILES ]; then
cat /tmp/DELETEDFILES >> $LOGFILE
echo -e "(end file list)\n" >> $LOGFILE
# and archive the missing files from yesterday to safe location.
# RSYNC missing files to the remote TARGET $TARGETBUPATH/backup.archive
while read ACCTtoARCHIVE; do
if [ "$TARGETUSERNAME" != "NONE" ]; then
rsync -avHl -e "ssh -i $KEYFILE" $LOCBUDIR/$YESTERDAY/accounts/$ACCTtoARCHIVE $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/backup.archive >> $LOGFILE 2>&1
else
echo "Note: No remote server." >> $LOGFILE
fi
cp -p $LOCBUDIR/$YESTERDAY/accounts/$ACCTtoARCHIVE $LOCBUDIR/backup.archive >> $LOGFILE 2>&1
done < /tmp/DELETEDFILES
else
echo -e "No file need to be archived today." >> $LOGFILE
fi
fi
fi
echo -e "\n==== BACKUP SYSTEM FILES =========================================" >> $LOGFILE
if [ "$TARGETUSERNAME" != "NONE" ]; then
# Backup System Files from TODAY as the system doesn't appear to be doing this.
echo " Comparing source and destination to make sure all files were transferred." >> $LOGFILE
echo " If any files are listed here, there was a problem with the transfer." >> $LOGFILE
rsync -aHlr -e "ssh -i $KEYFILE" $LOCBUDIR/$TODAY/system $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ >> $LOGFILE
rsync -au -progress --itemize-changes --dry-run -e "ssh -i $KEYFILE" $LOCBUDIR/$TODAY/system/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/system/ >> $LOGFILE
else
echo " No remote server defined." >> $LOGFILE
fi
echo -e "\n==== BACKUP ERRORS ===============================================" >> $LOGFILE
echo " The following are problems in the latest backup. This information" >> $LOGFILE
echo -e " derived from the logs in /usr/local/cpanel/logs/cpbackup/ \n" >> $LOGFILE
# Use touch and find to get today's backup log and then the transfer log.
touch --date "$TODAY" /tmp/time.txt
# Get the latest backup log, sift out junk and put ti into the LOGFILE
#find /usr/local/cpanel/logs/cpbackup/ -type f -newer /tmp/time.txt | xargs -i \
#egrep -v "(^\[|\.\.\.|^size is|homesize|Skipping|One or more files|3rdparty|gtar|lib64| 00000000 |=======|^$)" {} >> $LOGFILE
find /usr/local/cpanel/logs/cpbackup/ -type f -newer /tmp/time.txt | xargs -i \
egrep -i -A1 "(Error|Warning)" {} >> $LOGFILE
echo -e "\n==== TRANSFER LOGS ===============================================" >> $LOGFILE
if [ "$TARGETUSERNAME" != "NONE" ]; then
echo " Comparing source and destination to make sure all files were transferred" >> $LOGFILE
echo -e " properly. If any files are listed, there was a problem transferring " >> $LOGFILE
echo -e " that file. " >> $LOGFILE
echo " Codes: f=represents that it is a file. s=represents size changes are there." >> $LOGFILE
echo -e " t=represents timestamp changes are there. o=owner changed. g=group changed. p=permissions. \n" >> $LOGFILE
#echo " $LOCBUDIR/$TODAY/accounts $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY"
rsync -au -progress --itemize-changes --dry-run -e "ssh -i $KEYFILE" $LOCBUDIR/$TODAY/accounts/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ >> $LOGFILE
### rsync -au --progress --itemize-changes --dry-run -e "ssh -i $KEYFILE" $LOCBUDIR/$TODAY/accounts/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/
else
echo " No remote server defined." >> $LOGFILE
fi
echo -e "\n==== DRIVE STATUS ================================================" >> $LOGFILE
df -H >> $LOGFILE
###UNLESS YOU WANT TO EDIT CPANEL FILES YOU'LL NEED TO USE A DIFFERENT METHOD###
###THE FILE TO EDIT IS /usr/local/cpanel/bin/cpbackup_transporter
###THE LINE TO EDIT IT: my $max_log_retention = 10;
###CHANGE 10 TO AMOUNT OF ACCOUNTS ON SERVER + SOME BUFFER
#echo " The following are transfer logs to the backup server derived" >> $LOGFILE
#echo -e " from the logs in /usr/local/cpanel/logs/cpbackup_transporter/ \n" >> $LOGFILE
# Get the latest Transfer log send log entries for upload attempts.
#find /usr/local/cpanel/logs/cpbackup_transporter/ -type f -newer /tmp/time.txt | xargs -i \
#grep -h -A1 "`date +%Y-%m-%d`.* Upload attempt #" {} >> $LOGFILE
### NEW METHOD USES RSYNC TO COMPARE DIRECTORIES
echo -e "\n--\nReport generated by version:$VERSION of $0 on `hostname`" >> $LOGFILE
#SEND REPORT
if [ "$ADMIN" != "" ]; then
cat $LOGFILE | mail -s "BACKUP Report for `hostname` $TODAY" $ADMIN
else
cat $LOGFILE
fi
#Cleanliness is next to godliness.
rm -f /tmp/YESTERDAYFILELIST
rm -f /tmp/TODAYFILELIST
rm -f /tmp/DELETEDFILES
rm -f /tmp/logs.txt
exit
###########################################################################################################
# Documentation:
# - In the examples below where you create a config file or cronjob, remove the # signs
# at the beginning of the lines.
# Requirements prior to running this program:
# - cPanel backups must be backing up 2 or more daily backups.
# Optional: If you use a remote server backup.
# - You have already set up key pair connection to another server for ssh access. No passwords.
# NOTE: If NO remote server is being used, assign the TARGETUSERNAME to "NONE" in the config file
# see below.
# Setup:
# - Put this file on your server and chmod it 700, owned by root.
# - Create a config file. It can be any name you want but for this example we'll call it
# /root/Archive.mgr.cfg. This is a text file with the following variables that you set in it:
# ADMIN="you@email.com" # email address that recieves log reports.
# TARGETUSERNAME="remoteuser" # user name on remote server. Please don't use root.
# TARGETHOST="123.123.123.123" # the backup server's ip addres
# TARGETBUPATH="/home/remoteuser/host.server.com" # Target server path where you want your backups saved.
# LOCBUDIR="/backup" # backup directory on local server.
# KEYFILE="/root/.ssh/id_rsa" # ssh keyfile. default /user/.ssh/id_rsa
# - Create the /home/remoteuser/host.server.com (see above) on the target server.
# This is the directory on the target server where you wnat your backups saved.
# - Create a directory called /home/remoteuser/host.server.com/backup.archive on the target server.
# Here is where the deleted clients tar files will be copied to.
# - Create a directory called /backup/backup.archive on the local server.
# Here is where the deleted clients tar files will be saved.
# Optional: If you've been using the old backup system and want to archive the old backups using the new
# method, copy the files from /backup/cpbackup/daily/* to /backup/2015-00-00 where "2015-00-00" is the directory
# that stores TODAY'S backups. Use the following:
# cp -p /backup/cpbackup/daily/* /backup/2015-00-00/
# NOTE: Don't overwrite any existing files when asked.
# When the programs runs TOMORROW it will copy all the deleted accounts to /backup/backup.archive.
# OR
# You can copy the unique files into /backup/backup.archive manually.
# - Check your disk space. You want enough space for all these backups. After a time you can
# delete the files in /backup/cpbackup/weekly and monthly.
# - Create a directory called /backup/logs on the local server. If you don't the software will.
# Here is where the logs for this program will be saved.
# - Create a crontab for this program. It must run AFTER the cpanel backup is complete. I recommend
# running it 3 hours after the cpanel server backup runs. Don't worry, the program checks to see
# if cpanel backup is running and repeatedly delays 30 minutes before trying again if cpanel backup
# is running. The cron will be something like this. Note that /root/Archive.mgr.cfg is whatever
# you named the config file earlier.
# 27 08 * * * /root/archive.mgr.sh /root/Archive.mgr.cfg
# - As long as you have been running backups for more than 2 days, you can test the program with the above
# command: /root/archive.mgr.sh /root/Archive.mgr.cfg
###########################################################################################################
Verify backup status when moving an account between cpanels
I recently discovered an account did not have any backups.
It was migrated several months ago from another cpanel server - an older server.
With the new backup you can select accounts - it was set to the legacy backup instead of current backup system.
i.e. it retained its setting from the old server.
Luckily, we didn't need a backup, I had just sanity-checked and the counted was off.
I'd have never known - that account was not even listed in the backup report as having been skipped or failed - wasn't mentioned there at all.
It was migrated several months ago from another cpanel server - an older server.
With the new backup you can select accounts - it was set to the legacy backup instead of current backup system.
i.e. it retained its setting from the old server.
Luckily, we didn't need a backup, I had just sanity-checked and the counted was off.
I'd have never known - that account was not even listed in the backup report as having been skipped or failed - wasn't mentioned there at all.
It's a crested auklet
- Arf
- Official Test Penquin
- Posts: 9103
- Joined: Tue Apr 09, 2002 12:00 am
- Location: IDAHO, USA
- Contact:
Re: cPanel (new) Backup doesn't save those deleted accounts
I just updated the code (very minor)
Yeah, it's important to keep an eye on what is being backed up. I recently found out that /root is not backed up.
Yeah, it's important to keep an eye on what is being backed up. I recently found out that /root is not backed up.
Re: cPanel (new) Backup doesn't save those deleted accounts
Do you have any suggestions as to what to look for in the log to indicate issues with the backup?Arf wrote:Yeah, it's important to keep an eye on what is being backed up. I recently found out that /root is not backed up.
Currently, I am running a count on the word "pkgacctfile". If it matches my number of accounts, is that sufficient?
I'm also searching for the strings: fail, error, not, warn
I'm seeing sockets that have ended and an occasional permission error.,
It's a crested auklet
- Arf
- Official Test Penquin
- Posts: 9103
- Joined: Tue Apr 09, 2002 12:00 am
- Location: IDAHO, USA
- Contact:
Re: cPanel (new) Backup doesn't save those deleted accounts
When you say log, I'm assuming you means /usr/local/cpanel/logs/cpbackup/*
Look for "WARN" and "Cannot open file"
I have a script that does several things, among them is a "dry run" of rsync to compare the files themselves. You should be able to glean the info from my variables:
rsync -au -progress --itemize-changes --dry-run -e "ssh -p $PORT -i $KEYFILE" $LOCBUDIR/$TODAY/accounts/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ >> $LOGFILE
Here's the code. If I've left anything in here that is personal, please notify me.
Look for "WARN" and "Cannot open file"
I have a script that does several things, among them is a "dry run" of rsync to compare the files themselves. You should be able to glean the info from my variables:
rsync -au -progress --itemize-changes --dry-run -e "ssh -p $PORT -i $KEYFILE" $LOCBUDIR/$TODAY/accounts/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ >> $LOGFILE
Here's the code. If I've left anything in here that is personal, please notify me.
Code: Select all
# cat archive.mgr.sh
#!/bin/bash
VERSION="1.1"
# v 1.1 2/3/15 moved disk quota to bottom.
# v.0.1.1 12/24/14
# This program prevents the loss of backup files for accounts that have been deleted
# on a cPanel server. It also sends a report of the current backup
# Author: Thomas Leo :)
##################################################################################
# Documentation at the bottom! #
##################################################################################
############### NO NEED TO EDIT BELOW THIS LINE ##################################
# Make sure $1 is the config file and that the file exists.
if [ "$1" = "" ]; then echo "Syntax ERROR. Use: $0 Archive.host.cfg"; exit; fi
if [ ! -f $1 ]; then echo "ERROR: Cannot find config file: $1"; exit; fi
# Read in the config file.
source $1 # Get the config file info
if [ "$ADMIN" = "" -o "$TARGETUSERNAME" = "" -o "$TARGETHOST" = "" -o "$TARGETBUPATH" = "" -o "$LOCBUDIR" = "" -o "$KEYFILE" = "" ]; then
echo "ERROR: One or more of the variables needed in $1 is empty or blank."
exit
fi
TODAY=`date +%Y-%m-%d`;
YESTERDAY=`date +%Y-%m-%d --date="yesterday"`
LOGFILE="$LOCBUDIR/logs/backup-$TODAY.log"
if [ ! -d $LOCBUDIR/logs/ ]; then mkdir $LOCBUDIR/logs/ ; fi
> $LOGFILE # Clear the log file just in case.
# This section delays the backup for 30 minutes until cpbackup stops running.
BACKUPCMD="/usr/local/cpanel/bin/backup"
BUSY=`ps ax | egrep -v "(cpuwatch|grep)" | grep -c "$BACKUPCMD"`
while [ $BUSY -gt 0 ]; do
echo "Notice: backup is waiting for cpanel backups to stop. `date`" >> $LOGFILE
ps ax | egrep -v "(cpuwatch|grep)" | grep $BACKUPCMD >> $LOGFILE
sleep 1800 #sleep for 30 minutes
BUSY=`ps ax | egrep -v "(cpuwatch|grep)" | grep -c "$BACKUPCMD"`
done
#Log Rotation:
TENDAYSAGO=`date +%Y-%m-%d --date="10 days ago"`
touch --date "$TENDAYSAGO" /tmp/logs.txt
find $LOCBUDIR/logs/ -type f -not -newer /tmp/logs.txt | xargs -i rm -f {}
#### ARCHIVE ANY ACCOUNTS THAT HAVE BEEN DELETED SINCE YESTERDAY.##############################
echo -e "\n==== ARCHIVE TERMINATED ACCOUNTS =================================" >> $LOGFILE
echo " The following is a log of accounts that have been deleted since " >> $LOGFILE
echo " yesterday. .These files will be stored in the following directory:" >> $LOGFILE
echo -e " $TARGETHOST:$TARGETBUPATH/backup.archive\n" >> $LOGFILE
# Idiot checking
if [ ! -d $LOCBUDIR/$YESTERDAY/accounts ]; then
echo "**** ERROR: $LOCBUDIR/$YESTERDAY/accounts not found. ****" >> $LOGFILE
else
if [ ! -d $LOCBUDIR/$TODAY/accounts ]; then
echo "**** ERROR: $LOCBUDIR/$YESTERDAY/accounts not found ****" >> $LOGFILE
else
# End Idiot checking
# Get lists of files from TODAY and from YESTERDAY.
find $LOCBUDIR/$YESTERDAY/accounts -name "*.tar.gz" -type f -printf "%f\n" > /tmp/YESTERDAYFILELIST
find $LOCBUDIR/$TODAY/accounts -name "*.tar.gz" -type f -printf "%f\n" > /tmp/TODAYFILELIST
# Find the files that were there yesterday, but not there today ....
grep -v -f /tmp/TODAYFILELIST /tmp/YESTERDAYFILELIST > /tmp/DELETEDFILES # = the files that need to be archived from YESTERDAY.
# Check if /tmp/DELETEDFILES has NO DATA (-s) in it. If it has data, log and archive the files
if [ -s /tmp/DELETEDFILES ]; then
cat /tmp/DELETEDFILES >> $LOGFILE
echo -e "(end file list)\n" >> $LOGFILE
# and archive the missing files from yesterday to safe location.
# RSYNC missing files to the remote TARGET $TARGETBUPATH/backup.archive
while read ACCTtoARCHIVE; do
if [ "$TARGETUSERNAME" != "NONE" ]; then
rsync -avHl -e "ssh -p $PORT -i $KEYFILE" $LOCBUDIR/$YESTERDAY/accounts/$ACCTtoARCHIVE $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/backup.archive >> $LOGFILE 2>&1
else
echo "Note: No remote server." >> $LOGFILE
fi
cp -p $LOCBUDIR/$YESTERDAY/accounts/$ACCTtoARCHIVE $LOCBUDIR/backup.archive >> $LOGFILE 2>&1
done < /tmp/DELETEDFILES
else
echo -e "No file need to be archived today." >> $LOGFILE
fi
fi
fi
echo -e "\n==== BACKUP SYSTEM FILES =========================================" >> $LOGFILE
if [ "$TARGETUSERNAME" != "NONE" ]; then
# Backup System Files from TODAY as the system doesn't appear to be doing this.
echo " Comparing source and destination to make sure all files were transferred." >> $LOGFILE
echo " If any files are listed here, there was a problem with the transfer." >> $LOGFILE
rsync -aHlr -e "ssh -p $PORT -i $KEYFILE" $LOCBUDIR/$TODAY/system $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ >> $LOGFILE
rsync -au -progress --itemize-changes --dry-run -e "ssh -p $PORT -i $KEYFILE" $LOCBUDIR/$TODAY/system/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/system/ >> $LOGFILE
else
echo " No remote server defined." >> $LOGFILE
fi
echo -e "\n==== BACKUP ERRORS ===============================================" >> $LOGFILE
echo " The following are problems in the latest backup. This information" >> $LOGFILE
echo -e " derived from the logs in /usr/local/cpanel/logs/cpbackup/ \n" >> $LOGFILE
# Use touch and find to get today's backup log and then the transfer log.
touch --date "$TODAY" /tmp/time.txt
# Get the latest backup log, sift out junk and put ti into the LOGFILE
#find /usr/local/cpanel/logs/cpbackup/ -type f -newer /tmp/time.txt | xargs -i \
#egrep -v "(^\[|\.\.\.|^size is|homesize|Skipping|One or more files|3rdparty|gtar|lib64| 00000000 |=======|^$)" {} >> $LOGFILE
find /usr/local/cpanel/logs/cpbackup/ -type f -newer /tmp/time.txt | xargs -i \
egrep -i -A1 "(Error|Warning)" {} >> $LOGFILE
echo -e "\n==== TRANSFER LOGS ===============================================" >> $LOGFILE
if [ "$TARGETUSERNAME" != "NONE" ]; then
echo " Comparing source and destination to make sure all files were transferred" >> $LOGFILE
echo -e " properly. If any files are listed, there was a problem transferring " >> $LOGFILE
echo -e " that file. " >> $LOGFILE
echo " Codes: f=represents that it is a file. s=represents size changes are there." >> $LOGFILE
echo -e " t=represents timestamp changes are there. o=owner changed. g=group changed. p=permissions. \n" >> $LOGFILE
#echo " $LOCBUDIR/$TODAY/accounts $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY"
rsync -au -progress --itemize-changes --dry-run -e "ssh -p $PORT -i $KEYFILE" $LOCBUDIR/$TODAY/accounts/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ >> $LOGFILE
### rsync -au --progress --itemize-changes --dry-run -e "ssh -p $PORT -i $KEYFILE" $LOCBUDIR/$TODAY/accounts/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/
else
echo " No remote server defined." >> $LOGFILE
fi
echo -e "\n==== DRIVE STATUS ================================================" >> $LOGFILE
df -H >> $LOGFILE
###UNLESS YOU WANT TO EDIT CPANEL FILES YOU'LL NEED TO USE A DIFFERENT METHOD###
###THE FILE TO EDIT IS /usr/local/cpanel/bin/cpbackup_transporter
###THE LINE TO EDIT IT: my $max_log_retention = 10;
###CHANGE 10 TO AMOUNT OF ACCOUNTS ON SERVER + SOME BUFFER
#echo " The following are transfer logs to the backup server derived" >> $LOGFILE
#echo -e " from the logs in /usr/local/cpanel/logs/cpbackup_transporter/ \n" >> $LOGFILE
# Get the latest Transfer log send log entries for upload attempts.
#find /usr/local/cpanel/logs/cpbackup_transporter/ -type f -newer /tmp/time.txt | xargs -i \
#grep -h -A1 "`date +%Y-%m-%d`.* Upload attempt #" {} >> $LOGFILE
### NEW METHOD USES RSYNC TO COMPARE DIRECTORIES
echo -e "\n--\nReport generated by version:$VERSION of $0 on `hostname`" >> $LOGFILE
#SEND REPORT
if [ "$ADMIN" != "" ]; then
cat $LOGFILE | mail -s "BACKUP Report for `hostname` $TODAY" $ADMIN
else
cat $LOGFILE
fi
#Cleanliness is next to godliness.
rm -f /tmp/YESTERDAYFILELIST
rm -f /tmp/TODAYFILELIST
rm -f /tmp/DELETEDFILES
rm -f /tmp/logs.txt
exit
###########################################################################################################
# Documentation:
# - In the examples below where you create a config file or cronjob, remove the # signs
# at the beginning of the lines.
# Requirements prior to running this program:
# - cPanel backups must be backing up 2 or more daily backups.
# Optional: If you use a remote server backup.
# - You have already set up key pair connection to another server for ssh access. No passwords.
# NOTE: If NO remote server is being used, assign the TARGETUSERNAME to "NONE" in the config file
# see below.
# Setup:
# - Put this file on your server and chmod it 700, owned by root.
# - Create a config file. It can be any name you want but for this example we'll call it
# /root/Archive.mgr.cfg. This is a text file with the following variables that you set in it:
# ADMIN="you@email.com" # email address that recieves log reports.
# TARGETUSERNAME="remoteuser" # user name on remote server. Please don't use root.
# TARGETHOST="123.123.123.123" # the backup server's ip addres
# TARGETBUPATH="/home/remoteuser/host.server.com" # Target server path where you want your backups saved.
# LOCBUDIR="/backup" # backup directory on local server.
# KEYFILE="/root/.ssh/id_rsa" # ssh keyfile. default /user/.ssh/id_rsa
# - Create the /home/remoteuser/host.server.com (see above) on the target server.
# This is the directory on the target server where you wnat your backups saved.
# - Create a directory called /home/remoteuser/host.server.com/backup.archive on the target server.
# Here is where the deleted clients tar files will be copied to.
# - Create a directory called /backup/backup.archive on the local server.
# Here is where the deleted clients tar files will be saved.
# Optional: If you've been using the old backup system and want to archive the old backups using the new
# method, copy the files from /backup/cpbackup/daily/* to /backup/2015-00-00 where "2015-00-00" is the directory
# that stores TODAY'S backups. Use the following:
# cp -p /backup/cpbackup/daily/* /backup/2015-00-00/
# NOTE: Don't overwrite any existing files when asked.
# When the programs runs TOMORROW it will copy all the deleted accounts to /backup/backup.archive.
# OR
# You can copy the unique files into /backup/backup.archive manually.
# - Check your disk space. You want enough space for all these backups. After a time you can
# delete the files in /backup/cpbackup/weekly and monthly.
# - Create a directory called /backup/logs on the local server. If you don't the software will.
# Here is where the logs for this program will be saved.
# - Create a crontab for this program. It must run AFTER the cpanel backup is complete. I recommend
# running it 3 hours after the cpanel server backup runs. Don't worry, the program checks to see
# if cpanel backup is running and repeatedly delays 30 minutes before trying again if cpanel backup
# is running. The cron will be something like this. Note that /root/Archive.mgr.cfg is whatever
# you named the config file earlier.
# 27 08 * * * /root/archive.mgr.sh /root/Archive.mgr.cfg
# - As long as you have been running backups for more than 2 days, you can test the program with the above
# command: /root/archive.mgr.sh /root/Archive.mgr.cfg
###########################################################################################################
Re: cPanel (new) Backup doesn't save those deleted accounts
Got rid of some "warns" and pruned the size of backups by excluding softaculous backups:
Added to the server-wide exclusion list:
/etc/cpbackup-exclude.conf
There are also per account exclusion list files:
/home/OWNER/cpbackup-exclude.conf
Added to the server-wide exclusion list:
/etc/cpbackup-exclude.conf
Code: Select all
.cpanel/live-engine-connector-*.sock
softaculous_backups
/home/OWNER/cpbackup-exclude.conf
It's a crested auklet
- Arf
- Official Test Penquin
- Posts: 9103
- Joined: Tue Apr 09, 2002 12:00 am
- Location: IDAHO, USA
- Contact:
Re: cPanel (new) Backup doesn't save those deleted accounts
That's excellent. If you remind me in a week, I can see if there's an update to that software. I'm in the middle of a migration today.
T
T
Who is online
Users browsing this forum: No registered users and 1 guest