cPanel (new) Backup doesn't save those deleted accounts

If you have a program or script that you'd like to give away or sell, this is the place to do it. All offers should include contact info.

Moderators: BBear, theunknownhost, flaguy

Post Reply
User avatar
Arf
Official Test Penquin
Posts: 9101
Joined: Tue Apr 09, 2002 12:00 am
Location: IDAHO, USA
Contact:

cPanel (new) Backup doesn't save those deleted accounts

Post by Arf » Wed Dec 24, 2014 10:17 pm

I'm finally transitioning from the old backup to the new. I've found a difference that for me is a cause for concern. The old backup system used to not delete any backup files from deleted accounts. This was handy if the client returned and wanted their account restored. The new system deletes all the old back up files.

I've been writing a utility that archives a copy of any deleted accounts by putting them in a safe place both on the server and on a remote backup server if you're using one.

Is this of any concern to anyone else here? Would such a utility be handy?

BTW - the new backup system is great.

User avatar
newcanada
Hosting Superstar
Posts: 1702
Joined: Wed Jan 26, 2005 8:00 pm
Location: Canada

Re: cPanel (new) Backup doesn't save those deleted accounts

Post by newcanada » Sat Dec 27, 2014 10:53 pm

Oh man... this is a concern for me. I have had (and promised) clients who left come back want their old data restored, say a month or two later.

I'd be interested in such an utility but not in a hurry. On a scale of 1-10, with 10 being the most urgent, I say 4.

User avatar
Arf
Official Test Penquin
Posts: 9101
Joined: Tue Apr 09, 2002 12:00 am
Location: IDAHO, USA
Contact:

Re: cPanel (new) Backup doesn't save those deleted accounts

Post by Arf » Wed Dec 31, 2014 4:37 pm

I'm beta testing the software now and it's worked well enough that I'm beginning to roll it out of my second production server. In addition to keeping track of deleted accounts it gives a report on any errors that occurred during backup (there are a lot of things that new/old cpanel backup skipped, mostly do to permissions issues, and people don't realize it) as well as a log of the transfers to the remote backup server if you have one. It also duplicates the back up of deleted accounts on the backup server for safe keeping. I'm a pack rat and have accounts dating back to 2010.

Anyway, so far it's going well. Hopefully there will be more interest by someone. Either way, I'll be happy to post the code.

User avatar
Sapphyre
Official Test Penquin
Posts: 3333
Joined: Tue Nov 19, 2002 2:50 am
Location: Canada

Re: cPanel (new) Backup doesn't save those deleted accounts

Post by Sapphyre » Wed Jan 07, 2015 1:44 pm

Interested.
It's a crested auklet

User avatar
Arf
Official Test Penquin
Posts: 9101
Joined: Tue Apr 09, 2002 12:00 am
Location: IDAHO, USA
Contact:

Re: cPanel (new) Backup doesn't save those deleted accounts

Post by Arf » Tue Jan 13, 2015 8:22 pm

Almost done.

I learned this: If you're not checking for errors in /usr/local/cpanel/logs/cpbackup/* then you probably don't realize that stuff is not getting backed up. Usually permissions issues. The program will send you errors. Chances are about 100% that you have errors and don't know it.

It will also tell you about your disk usage, and if a backup didn't occur the day before.

User avatar
Arf
Official Test Penquin
Posts: 9101
Joined: Tue Apr 09, 2002 12:00 am
Location: IDAHO, USA
Contact:

Re: cPanel (new) Backup doesn't save those deleted accounts

Post by Arf » Sat Jan 17, 2015 7:06 pm

OK, I've been using this for a few weeks on 5 servers. All seems to be working fine. I included as many comments as needed to get a good idea of that the program does and how it works.

Program name: archive.mgr.sh

Code: Select all

#!/bin/bash

VERSION="1.1"
# v 1.1 2/3/15 moved disk quota to bottom. 
# v.0.1.1 12/24/14 
# This program prevents the loss of backup files for accounts that have been deleted
# on a cPanel server. It also sends a report of the current backup
# Author: Thomas Leo :)  

##################################################################################
# Documentation at the bottom!                                                   #
##################################################################################

############### NO NEED TO EDIT BELOW THIS LINE ##################################
# Make sure $1 is the config file and that the file exists.
if [ "$1" = "" ]; then echo "Syntax ERROR. Use: $0 Archive.host.cfg"; exit; fi
if [ ! -f $1 ]; then echo "ERROR: Cannot find config file: $1"; exit; fi
# Read in the config file.
source $1  # Get the config file info
if [ "$ADMIN" = "" -o "$TARGETUSERNAME" = "" -o "$TARGETHOST" = "" -o "$TARGETBUPATH" = "" -o "$LOCBUDIR" = "" -o "$KEYFILE" = "" ]; then
  echo "ERROR: One or more of the variables needed in $1 is empty or blank."
  exit
fi

TODAY=`date +%Y-%m-%d`;
YESTERDAY=`date +%Y-%m-%d --date="yesterday"`
LOGFILE="$LOCBUDIR/logs/backup-$TODAY.log"
if [ ! -d $LOCBUDIR/logs/ ]; then mkdir $LOCBUDIR/logs/ ; fi
> $LOGFILE   # Clear the log file just in case. 

# This section delays the backup for 30 minutes until cpbackup stops running. 
BACKUPCMD="/usr/local/cpanel/bin/backup"
BUSY=`ps ax | egrep -v "(cpuwatch|grep)" | grep -c "$BACKUPCMD"`
while [ $BUSY -gt 0 ]; do
  echo "Notice: backup is waiting for cpanel backups to stop. `date`" >> $LOGFILE
  ps ax | egrep -v "(cpuwatch|grep)" | grep $BACKUPCMD >> $LOGFILE
  sleep 1800 #sleep for 30 minutes
  BUSY=`ps ax | egrep -v "(cpuwatch|grep)" | grep -c "$BACKUPCMD"`
done

#Log Rotation:
TENDAYSAGO=`date +%Y-%m-%d --date="10 days ago"`
touch --date "$TENDAYSAGO" /tmp/logs.txt
find $LOCBUDIR/logs/ -type f -not -newer /tmp/logs.txt | xargs -i rm -f {}

#### ARCHIVE ANY ACCOUNTS THAT HAVE BEEN DELETED SINCE YESTERDAY.##############################
echo -e "\n==== ARCHIVE TERMINATED ACCOUNTS =================================" >> $LOGFILE
echo "  The following is a log of accounts that have been deleted since "  >> $LOGFILE
echo "  yesterday. .These files will be stored in the following directory:" >> $LOGFILE
echo -e "  $TARGETHOST:$TARGETBUPATH/backup.archive\n" >> $LOGFILE

# Idiot checking
if [ ! -d $LOCBUDIR/$YESTERDAY/accounts ]; then 
  echo "**** ERROR: $LOCBUDIR/$YESTERDAY/accounts not found. ****" >> $LOGFILE
else
  if [ ! -d $LOCBUDIR/$TODAY/accounts ]; then 
    echo "**** ERROR: $LOCBUDIR/$YESTERDAY/accounts not found ****" >> $LOGFILE
  else
# End Idiot checking
    # Get lists of files from TODAY and from YESTERDAY. 
    find  $LOCBUDIR/$YESTERDAY/accounts -name "*.tar.gz" -type f  -printf "%f\n" > /tmp/YESTERDAYFILELIST
    find  $LOCBUDIR/$TODAY/accounts     -name "*.tar.gz" -type f  -printf "%f\n" > /tmp/TODAYFILELIST
    # Find the files that were there yesterday, but not there today ....
    grep -v -f /tmp/TODAYFILELIST /tmp/YESTERDAYFILELIST > /tmp/DELETEDFILES      # = the files that need to be archived from YESTERDAY.
    # Check if /tmp/DELETEDFILES has NO DATA (-s) in it. If it has data, log and archive the files
    if [ -s /tmp/DELETEDFILES ]; then 
      cat /tmp/DELETEDFILES  >> $LOGFILE
      echo -e "(end file list)\n" >> $LOGFILE
      # and archive the missing files from yesterday to safe location.
      # RSYNC missing files to the remote TARGET $TARGETBUPATH/backup.archive
      while read ACCTtoARCHIVE; do
        if [ "$TARGETUSERNAME" != "NONE" ]; then
          rsync -avHl -e "ssh -i $KEYFILE" $LOCBUDIR/$YESTERDAY/accounts/$ACCTtoARCHIVE $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/backup.archive >> $LOGFILE 2>&1
        else
          echo "Note: No remote server." >> $LOGFILE
        fi
        cp -p $LOCBUDIR/$YESTERDAY/accounts/$ACCTtoARCHIVE $LOCBUDIR/backup.archive  >> $LOGFILE 2>&1
      done < /tmp/DELETEDFILES
    else
      echo -e "No file need to be archived today."  >> $LOGFILE
    fi
  fi
fi

echo -e "\n==== BACKUP SYSTEM FILES =========================================" >> $LOGFILE
if [ "$TARGETUSERNAME" != "NONE" ]; then
  # Backup System Files from TODAY as the system doesn't appear to be doing this. 
  echo "  Comparing source and destination to make sure all files were transferred." >> $LOGFILE
  echo "  If any files are listed here, there was a problem with the transfer." >> $LOGFILE
  rsync -aHlr -e "ssh -i $KEYFILE" $LOCBUDIR/$TODAY/system $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ >> $LOGFILE
  rsync -au -progress --itemize-changes --dry-run -e "ssh -i $KEYFILE" $LOCBUDIR/$TODAY/system/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/system/ >> $LOGFILE
else
  echo "  No remote server defined." >> $LOGFILE
fi

echo -e "\n==== BACKUP ERRORS ===============================================" >> $LOGFILE
echo "  The following are problems in the latest backup. This information" >> $LOGFILE
echo -e "  derived from the logs in /usr/local/cpanel/logs/cpbackup/ \n" >> $LOGFILE

# Use touch and find to get today's backup log and then the transfer log.
touch --date "$TODAY" /tmp/time.txt

# Get the latest backup log, sift out junk and put ti into the LOGFILE
#find /usr/local/cpanel/logs/cpbackup/ -type f -newer /tmp/time.txt | xargs -i \
#egrep -v "(^\[|\.\.\.|^size is|homesize|Skipping|One or more files|3rdparty|gtar|lib64| 00000000 |=======|^$)" {} >> $LOGFILE

find /usr/local/cpanel/logs/cpbackup/ -type f -newer /tmp/time.txt | xargs -i \
egrep -i -A1 "(Error|Warning)" {} >> $LOGFILE

echo -e "\n==== TRANSFER LOGS ===============================================" >> $LOGFILE
if [ "$TARGETUSERNAME" != "NONE" ]; then
  echo "  Comparing source and destination to make sure all files were transferred" >> $LOGFILE
  echo -e "  properly. If any files are listed, there was a problem transferring " >> $LOGFILE
  echo -e "  that file. " >> $LOGFILE
  echo    "  Codes: f=represents that it is a file. s=represents size changes are there."  >> $LOGFILE
  echo -e "  t=represents timestamp changes are there. o=owner changed. g=group changed. p=permissions. \n"  >> $LOGFILE
  #echo " $LOCBUDIR/$TODAY/accounts $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY"
  rsync -au -progress --itemize-changes --dry-run -e "ssh -i $KEYFILE" $LOCBUDIR/$TODAY/accounts/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ >> $LOGFILE
  ### rsync -au --progress --itemize-changes --dry-run -e "ssh -i $KEYFILE" $LOCBUDIR/$TODAY/accounts/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ 
else  
  echo "  No remote server defined." >> $LOGFILE
fi

echo -e "\n==== DRIVE STATUS ================================================" >> $LOGFILE
df -H >> $LOGFILE

###UNLESS YOU WANT TO EDIT CPANEL FILES YOU'LL NEED TO USE A DIFFERENT METHOD###
###THE FILE TO EDIT IS /usr/local/cpanel/bin/cpbackup_transporter
###THE LINE TO EDIT IT: my $max_log_retention = 10;
###CHANGE 10 TO AMOUNT OF ACCOUNTS ON SERVER + SOME BUFFER
#echo "  The following are transfer logs to the backup server derived" >> $LOGFILE
#echo -e "  from the logs in /usr/local/cpanel/logs/cpbackup_transporter/ \n" >> $LOGFILE
# Get the latest Transfer log send log entries for upload attempts.
#find /usr/local/cpanel/logs/cpbackup_transporter/ -type f -newer /tmp/time.txt | xargs -i \
#grep -h -A1 "`date +%Y-%m-%d`.* Upload attempt #"  {} >> $LOGFILE
### NEW METHOD USES RSYNC TO COMPARE DIRECTORIES

echo -e "\n--\nReport generated by version:$VERSION of $0 on `hostname`" >> $LOGFILE

#SEND REPORT
if [ "$ADMIN" != "" ]; then
  cat $LOGFILE | mail -s "BACKUP Report for `hostname` $TODAY" $ADMIN
else
  cat $LOGFILE
fi

#Cleanliness is next to godliness. 
rm -f /tmp/YESTERDAYFILELIST
rm -f /tmp/TODAYFILELIST
rm -f /tmp/DELETEDFILES
rm -f /tmp/logs.txt

exit
###########################################################################################################
# Documentation: 


# - In the examples below where you create a config file or cronjob, remove the # signs 
#   at the beginning of the lines. 

# Requirements prior to running this program:
# - cPanel backups must be backing up 2 or more daily backups. 

# Optional: If you use a remote server backup. 
# - You have already set up key pair connection to another server for ssh access. No passwords. 
#   NOTE: If NO remote server is being used, assign the TARGETUSERNAME to "NONE" in the config file
#   see below. 

# Setup:
# - Put this file on your server and chmod it 700, owned by root. 

# - Create a config file. It can be any name you want but for this example we'll call it 
#   /root/Archive.mgr.cfg. This is a text file with the following variables that you set in it:
#  ADMIN="you@email.com"        # email address that recieves log reports.
#  TARGETUSERNAME="remoteuser"  # user name on remote server. Please don't use root.
#  TARGETHOST="123.123.123.123" # the backup server's ip addres
#  TARGETBUPATH="/home/remoteuser/host.server.com"  # Target server path where you want your backups saved.
#  LOCBUDIR="/backup"           # backup directory on local server.
#  KEYFILE="/root/.ssh/id_rsa"  # ssh keyfile. default /user/.ssh/id_rsa

# - Create the /home/remoteuser/host.server.com (see above) on the target server.
#   This is the directory on the target server where you wnat your backups saved.

# - Create a directory called /home/remoteuser/host.server.com/backup.archive on the target server. 
#   Here is where the deleted clients tar files will be copied to. 

# - Create a directory called /backup/backup.archive on the local server. 
#   Here is where the deleted clients tar files will be saved. 
#   Optional: If you've been using the old backup system and want to archive the old backups using the new
#   method, copy the files from /backup/cpbackup/daily/* to /backup/2015-00-00 where "2015-00-00" is the directory 
#   that stores TODAY'S backups. Use the following:
#   cp -p /backup/cpbackup/daily/* /backup/2015-00-00/ 
#   NOTE: Don't overwrite any existing files when asked. 
#   When the programs runs TOMORROW it will copy all the deleted accounts to /backup/backup.archive. 
#   OR
#   You can copy the unique files into /backup/backup.archive manually. 

# - Check your disk space. You want enough space for all these backups. After a time you can
#   delete the files in /backup/cpbackup/weekly and monthly. 

# - Create a directory called /backup/logs on the local server. If you don't the software will. 
#   Here is where the logs for this program will be saved.

# - Create a crontab for this program. It must run AFTER the cpanel backup is complete. I recommend 
#   running it 3 hours after the cpanel server backup runs. Don't worry, the program checks to see
#   if cpanel backup is running and repeatedly delays 30 minutes before trying again if cpanel backup 
#   is running. The cron will be something like this. Note that /root/Archive.mgr.cfg is whatever
#   you named the config file earlier. 
#   27 08 * * * /root/archive.mgr.sh /root/Archive.mgr.cfg

# - As long as you have been running backups for more than 2 days, you can test the program with the above
#   command: /root/archive.mgr.sh /root/Archive.mgr.cfg

###########################################################################################################

User avatar
Sapphyre
Official Test Penquin
Posts: 3333
Joined: Tue Nov 19, 2002 2:50 am
Location: Canada

Verify backup status when moving an account between cpanels

Post by Sapphyre » Fri Mar 20, 2015 8:18 am

I recently discovered an account did not have any backups.
It was migrated several months ago from another cpanel server - an older server.
With the new backup you can select accounts - it was set to the legacy backup instead of current backup system.
i.e. it retained its setting from the old server.

Luckily, we didn't need a backup, I had just sanity-checked and the counted was off.
I'd have never known - that account was not even listed in the backup report as having been skipped or failed - wasn't mentioned there at all.
It's a crested auklet

User avatar
Arf
Official Test Penquin
Posts: 9101
Joined: Tue Apr 09, 2002 12:00 am
Location: IDAHO, USA
Contact:

Re: cPanel (new) Backup doesn't save those deleted accounts

Post by Arf » Sat Mar 21, 2015 9:22 am

I just updated the code (very minor)

Yeah, it's important to keep an eye on what is being backed up. I recently found out that /root is not backed up.

User avatar
Sapphyre
Official Test Penquin
Posts: 3333
Joined: Tue Nov 19, 2002 2:50 am
Location: Canada

Re: cPanel (new) Backup doesn't save those deleted accounts

Post by Sapphyre » Wed Sep 30, 2015 3:12 pm

Arf wrote:Yeah, it's important to keep an eye on what is being backed up. I recently found out that /root is not backed up.
Do you have any suggestions as to what to look for in the log to indicate issues with the backup?
Currently, I am running a count on the word "pkgacctfile". If it matches my number of accounts, is that sufficient?
I'm also searching for the strings: fail, error, not, warn

I'm seeing sockets that have ended and an occasional permission error.,
It's a crested auklet

User avatar
Arf
Official Test Penquin
Posts: 9101
Joined: Tue Apr 09, 2002 12:00 am
Location: IDAHO, USA
Contact:

Re: cPanel (new) Backup doesn't save those deleted accounts

Post by Arf » Wed Sep 30, 2015 5:36 pm

When you say log, I'm assuming you means /usr/local/cpanel/logs/cpbackup/*

Look for "WARN" and "Cannot open file"

I have a script that does several things, among them is a "dry run" of rsync to compare the files themselves. You should be able to glean the info from my variables:

rsync -au -progress --itemize-changes --dry-run -e "ssh -p $PORT -i $KEYFILE" $LOCBUDIR/$TODAY/accounts/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ >> $LOGFILE

Here's the code. If I've left anything in here that is personal, please notify me.

Code: Select all

# cat archive.mgr.sh 
#!/bin/bash

VERSION="1.1"
# v 1.1 2/3/15 moved disk quota to bottom. 
# v.0.1.1 12/24/14 
# This program prevents the loss of backup files for accounts that have been deleted
# on a cPanel server. It also sends a report of the current backup
# Author: Thomas Leo :)  

##################################################################################
# Documentation at the bottom!                                                   #
##################################################################################

############### NO NEED TO EDIT BELOW THIS LINE ##################################
# Make sure $1 is the config file and that the file exists.
if [ "$1" = "" ]; then echo "Syntax ERROR. Use: $0 Archive.host.cfg"; exit; fi
if [ ! -f $1 ]; then echo "ERROR: Cannot find config file: $1"; exit; fi
# Read in the config file.
source $1  # Get the config file info
if [ "$ADMIN" = "" -o "$TARGETUSERNAME" = "" -o "$TARGETHOST" = "" -o "$TARGETBUPATH" = "" -o "$LOCBUDIR" = "" -o "$KEYFILE" = "" ]; then
  echo "ERROR: One or more of the variables needed in $1 is empty or blank."
  exit
fi

TODAY=`date +%Y-%m-%d`;
YESTERDAY=`date +%Y-%m-%d --date="yesterday"`
LOGFILE="$LOCBUDIR/logs/backup-$TODAY.log"
if [ ! -d $LOCBUDIR/logs/ ]; then mkdir $LOCBUDIR/logs/ ; fi
> $LOGFILE   # Clear the log file just in case. 

# This section delays the backup for 30 minutes until cpbackup stops running. 
BACKUPCMD="/usr/local/cpanel/bin/backup"
BUSY=`ps ax | egrep -v "(cpuwatch|grep)" | grep -c "$BACKUPCMD"`
while [ $BUSY -gt 0 ]; do
  echo "Notice: backup is waiting for cpanel backups to stop. `date`" >> $LOGFILE
  ps ax | egrep -v "(cpuwatch|grep)" | grep $BACKUPCMD >> $LOGFILE
  sleep 1800 #sleep for 30 minutes
  BUSY=`ps ax | egrep -v "(cpuwatch|grep)" | grep -c "$BACKUPCMD"`
done

#Log Rotation:
TENDAYSAGO=`date +%Y-%m-%d --date="10 days ago"`
touch --date "$TENDAYSAGO" /tmp/logs.txt
find $LOCBUDIR/logs/ -type f -not -newer /tmp/logs.txt | xargs -i rm -f {}

#### ARCHIVE ANY ACCOUNTS THAT HAVE BEEN DELETED SINCE YESTERDAY.##############################
echo -e "\n==== ARCHIVE TERMINATED ACCOUNTS =================================" >> $LOGFILE
echo "  The following is a log of accounts that have been deleted since "  >> $LOGFILE
echo "  yesterday. .These files will be stored in the following directory:" >> $LOGFILE
echo -e "  $TARGETHOST:$TARGETBUPATH/backup.archive\n" >> $LOGFILE

# Idiot checking
if [ ! -d $LOCBUDIR/$YESTERDAY/accounts ]; then 
  echo "**** ERROR: $LOCBUDIR/$YESTERDAY/accounts not found. ****" >> $LOGFILE
else
  if [ ! -d $LOCBUDIR/$TODAY/accounts ]; then 
    echo "**** ERROR: $LOCBUDIR/$YESTERDAY/accounts not found ****" >> $LOGFILE
  else
# End Idiot checking
    # Get lists of files from TODAY and from YESTERDAY. 
    find  $LOCBUDIR/$YESTERDAY/accounts -name "*.tar.gz" -type f  -printf "%f\n" > /tmp/YESTERDAYFILELIST
    find  $LOCBUDIR/$TODAY/accounts     -name "*.tar.gz" -type f  -printf "%f\n" > /tmp/TODAYFILELIST
    # Find the files that were there yesterday, but not there today ....
    grep -v -f /tmp/TODAYFILELIST /tmp/YESTERDAYFILELIST > /tmp/DELETEDFILES      # = the files that need to be archived from YESTERDAY.
    # Check if /tmp/DELETEDFILES has NO DATA (-s) in it. If it has data, log and archive the files
    if [ -s /tmp/DELETEDFILES ]; then 
      cat /tmp/DELETEDFILES  >> $LOGFILE
      echo -e "(end file list)\n" >> $LOGFILE
      # and archive the missing files from yesterday to safe location.
      # RSYNC missing files to the remote TARGET $TARGETBUPATH/backup.archive
      while read ACCTtoARCHIVE; do
        if [ "$TARGETUSERNAME" != "NONE" ]; then
          rsync -avHl -e "ssh -p $PORT  -i $KEYFILE" $LOCBUDIR/$YESTERDAY/accounts/$ACCTtoARCHIVE $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/backup.archive >> $LOGFILE 2>&1
        else
          echo "Note: No remote server." >> $LOGFILE
        fi
        cp -p $LOCBUDIR/$YESTERDAY/accounts/$ACCTtoARCHIVE $LOCBUDIR/backup.archive  >> $LOGFILE 2>&1
      done < /tmp/DELETEDFILES
    else
      echo -e "No file need to be archived today."  >> $LOGFILE
    fi
  fi
fi

echo -e "\n==== BACKUP SYSTEM FILES =========================================" >> $LOGFILE
if [ "$TARGETUSERNAME" != "NONE" ]; then
  # Backup System Files from TODAY as the system doesn't appear to be doing this. 
  echo "  Comparing source and destination to make sure all files were transferred." >> $LOGFILE
  echo "  If any files are listed here, there was a problem with the transfer." >> $LOGFILE
  rsync -aHlr -e "ssh -p $PORT  -i $KEYFILE" $LOCBUDIR/$TODAY/system $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ >> $LOGFILE
  rsync -au -progress --itemize-changes --dry-run -e "ssh -p $PORT  -i $KEYFILE" $LOCBUDIR/$TODAY/system/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/system/ >> $LOGFILE
else
  echo "  No remote server defined." >> $LOGFILE
fi

echo -e "\n==== BACKUP ERRORS ===============================================" >> $LOGFILE
echo "  The following are problems in the latest backup. This information" >> $LOGFILE
echo -e "  derived from the logs in /usr/local/cpanel/logs/cpbackup/ \n" >> $LOGFILE

# Use touch and find to get today's backup log and then the transfer log.
touch --date "$TODAY" /tmp/time.txt

# Get the latest backup log, sift out junk and put ti into the LOGFILE
#find /usr/local/cpanel/logs/cpbackup/ -type f -newer /tmp/time.txt | xargs -i \
#egrep -v "(^\[|\.\.\.|^size is|homesize|Skipping|One or more files|3rdparty|gtar|lib64| 00000000 |=======|^$)" {} >> $LOGFILE

find /usr/local/cpanel/logs/cpbackup/ -type f -newer /tmp/time.txt | xargs -i \
egrep -i -A1 "(Error|Warning)" {} >> $LOGFILE

echo -e "\n==== TRANSFER LOGS ===============================================" >> $LOGFILE
if [ "$TARGETUSERNAME" != "NONE" ]; then
  echo "  Comparing source and destination to make sure all files were transferred" >> $LOGFILE
  echo -e "  properly. If any files are listed, there was a problem transferring " >> $LOGFILE
  echo -e "  that file. " >> $LOGFILE
  echo    "  Codes: f=represents that it is a file. s=represents size changes are there."  >> $LOGFILE
  echo -e "  t=represents timestamp changes are there. o=owner changed. g=group changed. p=permissions. \n"  >> $LOGFILE
  #echo " $LOCBUDIR/$TODAY/accounts $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY"
  rsync -au -progress --itemize-changes --dry-run -e "ssh -p $PORT  -i $KEYFILE" $LOCBUDIR/$TODAY/accounts/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ >> $LOGFILE
  ### rsync -au --progress --itemize-changes --dry-run -e "ssh -p $PORT  -i $KEYFILE" $LOCBUDIR/$TODAY/accounts/ $TARGETUSERNAME@$TARGETHOST:$TARGETBUPATH/$TODAY/ 
else  
  echo "  No remote server defined." >> $LOGFILE
fi

echo -e "\n==== DRIVE STATUS ================================================" >> $LOGFILE
df -H >> $LOGFILE

###UNLESS YOU WANT TO EDIT CPANEL FILES YOU'LL NEED TO USE A DIFFERENT METHOD###
###THE FILE TO EDIT IS /usr/local/cpanel/bin/cpbackup_transporter
###THE LINE TO EDIT IT: my $max_log_retention = 10;
###CHANGE 10 TO AMOUNT OF ACCOUNTS ON SERVER + SOME BUFFER
#echo "  The following are transfer logs to the backup server derived" >> $LOGFILE
#echo -e "  from the logs in /usr/local/cpanel/logs/cpbackup_transporter/ \n" >> $LOGFILE
# Get the latest Transfer log send log entries for upload attempts.
#find /usr/local/cpanel/logs/cpbackup_transporter/ -type f -newer /tmp/time.txt | xargs -i \
#grep -h -A1 "`date +%Y-%m-%d`.* Upload attempt #"  {} >> $LOGFILE
### NEW METHOD USES RSYNC TO COMPARE DIRECTORIES

echo -e "\n--\nReport generated by version:$VERSION of $0 on `hostname`" >> $LOGFILE

#SEND REPORT
if [ "$ADMIN" != "" ]; then
  cat $LOGFILE | mail -s "BACKUP Report for `hostname` $TODAY" $ADMIN
else
  cat $LOGFILE
fi

#Cleanliness is next to godliness. 
rm -f /tmp/YESTERDAYFILELIST
rm -f /tmp/TODAYFILELIST
rm -f /tmp/DELETEDFILES
rm -f /tmp/logs.txt

exit
###########################################################################################################
# Documentation: 


# - In the examples below where you create a config file or cronjob, remove the # signs 
#   at the beginning of the lines. 

# Requirements prior to running this program:
# - cPanel backups must be backing up 2 or more daily backups. 

# Optional: If you use a remote server backup. 
# - You have already set up key pair connection to another server for ssh access. No passwords. 
#   NOTE: If NO remote server is being used, assign the TARGETUSERNAME to "NONE" in the config file
#   see below. 

# Setup:
# - Put this file on your server and chmod it 700, owned by root. 

# - Create a config file. It can be any name you want but for this example we'll call it 
#   /root/Archive.mgr.cfg. This is a text file with the following variables that you set in it:
#  ADMIN="you@email.com"        # email address that recieves log reports.
#  TARGETUSERNAME="remoteuser"  # user name on remote server. Please don't use root.
#  TARGETHOST="123.123.123.123" # the backup server's ip addres
#  TARGETBUPATH="/home/remoteuser/host.server.com"  # Target server path where you want your backups saved.
#  LOCBUDIR="/backup"           # backup directory on local server.
#  KEYFILE="/root/.ssh/id_rsa"  # ssh keyfile. default /user/.ssh/id_rsa

# - Create the /home/remoteuser/host.server.com (see above) on the target server.
#   This is the directory on the target server where you wnat your backups saved.

# - Create a directory called /home/remoteuser/host.server.com/backup.archive on the target server. 
#   Here is where the deleted clients tar files will be copied to. 

# - Create a directory called /backup/backup.archive on the local server. 
#   Here is where the deleted clients tar files will be saved. 
#   Optional: If you've been using the old backup system and want to archive the old backups using the new
#   method, copy the files from /backup/cpbackup/daily/* to /backup/2015-00-00 where "2015-00-00" is the directory 
#   that stores TODAY'S backups. Use the following:
#   cp -p /backup/cpbackup/daily/* /backup/2015-00-00/ 
#   NOTE: Don't overwrite any existing files when asked. 
#   When the programs runs TOMORROW it will copy all the deleted accounts to /backup/backup.archive. 
#   OR
#   You can copy the unique files into /backup/backup.archive manually. 

# - Check your disk space. You want enough space for all these backups. After a time you can
#   delete the files in /backup/cpbackup/weekly and monthly. 

# - Create a directory called /backup/logs on the local server. If you don't the software will. 
#   Here is where the logs for this program will be saved.

# - Create a crontab for this program. It must run AFTER the cpanel backup is complete. I recommend 
#   running it 3 hours after the cpanel server backup runs. Don't worry, the program checks to see
#   if cpanel backup is running and repeatedly delays 30 minutes before trying again if cpanel backup 
#   is running. The cron will be something like this. Note that /root/Archive.mgr.cfg is whatever
#   you named the config file earlier. 
#   27 08 * * * /root/archive.mgr.sh /root/Archive.mgr.cfg

# - As long as you have been running backups for more than 2 days, you can test the program with the above
#   command: /root/archive.mgr.sh /root/Archive.mgr.cfg

###########################################################################################################

User avatar
Sapphyre
Official Test Penquin
Posts: 3333
Joined: Tue Nov 19, 2002 2:50 am
Location: Canada

Re: cPanel (new) Backup doesn't save those deleted accounts

Post by Sapphyre » Sun Jan 31, 2016 2:45 pm

Got rid of some "warns" and pruned the size of backups by excluding softaculous backups:

Added to the server-wide exclusion list:
/etc/cpbackup-exclude.conf

Code: Select all

.cpanel/live-engine-connector-*.sock
softaculous_backups
There are also per account exclusion list files:
/home/OWNER/cpbackup-exclude.conf
It's a crested auklet

User avatar
Arf
Official Test Penquin
Posts: 9101
Joined: Tue Apr 09, 2002 12:00 am
Location: IDAHO, USA
Contact:

Re: cPanel (new) Backup doesn't save those deleted accounts

Post by Arf » Mon Feb 01, 2016 6:49 pm

That's excellent. If you remind me in a week, I can see if there's an update to that software. I'm in the middle of a migration today.
T

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest