Editing Introduction to a Self Managed Life: a 13 hour & 28 minute presentation by FUTO software (section)

==== 7.1 Backup script creation: ====

This script below will allow you to have your virtual machines backed up automatically. It does the following:

* Shuts down the virtual machine
* Copies its disk image qcow2 file to the <code>/mediapool/vmbackups</code> zfs dataset
* Copies its configuration so it can be set up again
* Keeps five backups but deletes the oldest ones after you have five.

This means the following:

* You can mess things up by deleting files you weren’t supposed to, mess up configurations and programs, and restore everything to where it was last week with one or two kindergarten level GNU/Linux commands.
* You can migrate this to another computer entirely &amp; start the virtual machine up there.

shuts each virtual machine down one by one, backs up the virtual

Save this as <code>/root/vm_backup.sh</code>:

<pre># Open the text editor
sudo nano -w /root/vm_backup.sh</pre>
<pre>#!/bin/bash

# thank you to stack overflow for giving me the courage to wade through 100s of posts and hack together something that looks like it works. 

# config for backups
BACKUP_DIR=&quot;/mediapool/vmbackups&quot;
LOG_FILE=&quot;/var/log/vm_backups.log&quot;
RETENTION_DAYS=56  # how long to keep backups

# Function to write messages to our log file`
log_message() {
    # Get the current timestamp and message
    echo &quot;[$(date '+%Y-%m-%d %H:%M:%S')] $1&quot; | tee -a &quot;$LOG_FILE&quot;
}

# Function to find the actual disk path for a VM when the default path doesn't exist
# Uses virsh dumpxml to get the disk source path from the VM's XML configuration
find_vm_disk_path() {
    local vm_name=$1
    # Get the VM's XML configuration and extract the first disk source path
    # Using grep with -o to only output the matched portion
    # Using sed to extract just the path part from the source attribute
    local disk_path=$(virsh dumpxml &quot;$vm_name&quot; | grep -o &quot;source file='[^']*'&quot; | head -n1 | sed &quot;s/source file='\(.*\)'/\1/&quot;)
    
    # Check if we found a path and if it exists
    if [ -n &quot;$disk_path&quot; ] &amp;&amp; [ -f &quot;$disk_path&quot; ]; then
        echo &quot;$disk_path&quot;
        return 0
    else
        return 1
    fi
}

# main backup function 
backup_vm() {
    local virtual_machine_name=$1  # The name of the virtual machine we're backing up
    local date_stamp=$(date +%Y%m%d)  # Today's date for the backup file name
    local source_file=&quot;/var/lib/libvirt/images/${virtual_machine_name}.qcow2&quot;  # Where the virtual machine is
    
    # If the default path doesn't exist, try to find the actual disk path
    if [ ! -f &quot;$source_file&quot; ]; then
        log_message &quot;Default disk path not found for ${virtual_machine_name}, searching XML configuration...&quot;
        local found_path=$(find_vm_disk_path &quot;$virtual_machine_name&quot;)
        
        # If we found a valid path, use it instead
        if [ -n &quot;$found_path&quot; ]; then
            log_message &quot;Found alternate disk path: ${found_path}&quot;
            source_file=&quot;$found_path&quot;
        fi
    fi
    
    local backup_file=&quot;${BACKUP_DIR}/${virtual_machine_name}-${date_stamp}.qcow2&quot;  # Where we're putting the backup of it
    local config_file=&quot;${BACKUP_DIR}/${virtual_machine_name}-${date_stamp}.xml&quot;  # Where it saves the virtual machine config
    
    # Check if source file exists before attempting backup
    if [ ! -f &quot;$source_file&quot; ]; then
        log_message &quot;ERROR: Source file $source_file does not exist for ${virtual_machine_name}&quot;
        return 1
    fi
    
    # Announce backup is starting
    log_message &quot;Starting backup process for ${virtual_machine_name}&quot;
    
    # Save virtual machine's config
    virsh dumpxml &quot;$virtual_machine_name&quot; &gt; &quot;$config_file&quot;
    
    # Set ownership and permissions for config file
    chown libvirt-qemu:kvm &quot;$config_file&quot;
    chmod 644 &quot;$config_file&quot;

    # Try to shut down the virtual machine nicely 
    log_message &quot;Shutting down ${virtual_machine_name}&quot;
    virsh shutdown &quot;$virtual_machine_name&quot;
    
    # Wait patiently for the virtual machine to shut down 
    local count=0
    while [ &quot;$(virsh domstate $virtual_machine_name)&quot; != &quot;shut off&quot; ] &amp;&amp; [ $count -lt 30 ]; do
        sleep 10
        count=$((count + 1))
    done
    
    # If it doesn't turn off, make it turn off(like holding the power button)
    if [ &quot;$(virsh domstate $virtual_machine_name)&quot; != &quot;shut off&quot; ]; then
        log_message &quot;WARNING: Force shutting down ${virtual_machine_name}&quot;
        virsh destroy &quot;$virtual_machine_name&quot;
        sleep 10
    fi
    
    # Make sure it's actually off - trust but verify
    if [ &quot;$(virsh domstate $virtual_machine_name)&quot; != &quot;shut off&quot; ]; then
        log_message &quot;ERROR: Failed to shut down ${virtual_machine_name}&quot;
        return 1
    fi
    
    # Create the backup - doesn't use compression since qemu-img convert compression is single threaded and insanely slow
    log_message &quot;Creating backup of ${virtual_machine_name}&quot;
    if ! qemu-img convert -p -f qcow2 -O qcow2 &quot;$source_file&quot; &quot;$backup_file&quot;; then
        log_message &quot;ERROR: Backup failed for ${virtual_machine_name}&quot;
        virsh start &quot;$virtual_machine_name&quot;
        return 1
    fi
    
    # Set ownership and permissions for backup file
    chown libvirt-qemu:kvm &quot;$backup_file&quot;
    chmod 644 &quot;$backup_file&quot;

    # Make sure the backup isn't insanely small since that means this didn't work
    # Fixed stat command for Linux systems
    local source_size=$(stat -c%s &quot;$source_file&quot;)
    local backup_size=$(stat -c%s &quot;$backup_file&quot;)
    if [ &quot;$backup_size&quot; -lt 1048576 ]; then  # Less than 1MB is suspicious - like a $5 &quot;genuine&quot; Rolex
        log_message &quot;ERROR: Backup file suspiciously small for ${virtual_machine_name}&quot;
        rm -f &quot;$backup_file&quot; &quot;$config_file&quot;
        virsh start &quot;$virtual_machine_name&quot;
        return 1
    fi
    
    # Turn virtual machine back on when backup is done. 
    log_message &quot;Starting ${virtual_machine_name}&quot;
    virsh start &quot;$virtual_machine_name&quot;
    
    # Wait for it to come back online 
    count=0
    while [ &quot;$(virsh domstate $virtual_machine_name)&quot; != &quot;running&quot; ] &amp;&amp; [ $count -lt 12 ]; do
        sleep 5
        count=$((count + 1))
    done
    
    # Make sure it actually started(inspect what you expect)
    if [ &quot;$(virsh domstate $virtual_machine_name)&quot; != &quot;running&quot; ]; then
        log_message &quot;ERROR: Failed to start ${virtual_machine_name}&quot;
        return 1
    fi
    
    # announce that it worked
    log_message &quot;Backup of ${virtual_machine_name} completed!&quot;
    
    # Clean up old backups - because nobody likes a full hard drive
    log_message &quot;Cleaning up old backups for ${virtual_machine_name}&quot;
    find &quot;$BACKUP_DIR&quot; -name &quot;${virtual_machine_name}-*.qcow2&quot; -mtime +${RETENTION_DAYS} -exec rm -f {} \;  # Delete old qcow2 files
    find &quot;$BACKUP_DIR&quot; -name &quot;${virtual_machine_name}-*.xml&quot; -mtime +${RETENTION_DAYS} -exec rm -f {} \;   # Delete old xml files
}

# Start of the main backup process
log_message &quot;Starting backup process&quot;

# Make sure we're running as root 
if [ &quot;$EUID&quot; -ne 0 ]; then
    log_message &quot;ERROR: Must run as root&quot;
    exit 1
fi

# Check if the backup directory exists 
if [ ! -d &quot;$BACKUP_DIR&quot; ]; then
    log_message &quot;ERROR: Backup directory $BACKUP_DIR does not exist&quot;
    exit 1
fi

# Get list of ALL virtual machines, not just running ones
# Changed to list all VMs instead of just running ones
VMS=($(virsh list --all --name))

# Check if we have enough disk space to back up
available_space=$(df -B1 &quot;$BACKUP_DIR&quot; | awk 'NR==2 {print $4}')
required_space=0

# Calculate how much space we need
for virtual_machine in &quot;${VMS[@]}&quot;; do
    if [ -n &quot;$virtual_machine&quot; ]; then
        # Try the default path first
        local_path=&quot;/var/lib/libvirt/images/${virtual_machine}.qcow2&quot;
        
        # If default path doesn't exist, try to find actual path
        if [ ! -f &quot;$local_path&quot; ]; then
            local_path=$(find_vm_disk_path &quot;$virtual_machine&quot;) || continue
        fi
        
        if [ -f &quot;$local_path&quot; ]; then
            virtual_machine_size=$(du -b &quot;$local_path&quot; 2&gt;/dev/null | cut -f1)
            required_space=$((required_space + virtual_machine_size))
        fi
    fi
done

# Make sure we have enough space 
if [ &quot;$available_space&quot; -lt &quot;$required_space&quot; ]; then
    log_message &quot;ERROR: Insufficient space in backup directory&quot;
    exit 1
fi

# loop for backing up every virtual machine
for virtual_machine in &quot;${VMS[@]}&quot;; do
    if [ -n &quot;$virtual_machine&quot; ]; then
        backup_vm &quot;$virtual_machine&quot;
    fi
done

# announce it's all done
log_message &quot;Backup process completed!&quot;
</pre>
<blockquote>'''Nerd note: This script would be laughed out of the room for use in production environments for major web companies &amp; datacenters.''' This script turns off the virtual machine to back it up.

This means that at 1 AM, the service goes down. This would be unacceptable in a production environment where people expect the service to be available 24/7.

There are ways to do live backups where you flush mysql tables and lock them, make redis background save, pause call processing in asterisk, pause io, create atomic snapshots, coordinate with databases of all the different programs…. the audience of this guide is a person running a home server in his closet. Do you really want to subject a beginner to docker volumes that may not be in a consistent state, email delivery/receipt being interrupted, database transactionst hat are messed with in the middle fo a write, corrupt call recordings, partially written large files, all so someone can get live backups of a server in their closet, you monster?

If you need that level of uptime, you’re not a newbie reading this guide. or you are, and you need to hire a consultant to set you up with something like veeam.

To subject a newbie to the risk of error/corruption/screwups that comes with doing live backups for these things when they’re at the level of this guide being helpful to them is cruel.
</blockquote>
<span id="set-permissions-so-script-works"></span>