Editing Introduction to a Self Managed Life: a 13 hour & 28 minute presentation by FUTO software (section)

== Step 2: Setting up Syncthing for android backups ==

<span id="step-1-install-syncthing"></span>
=== Step 1: Install syncthing ===

<span id="add-the-syncthing-repository"></span>
==== 1.1 Add the Syncthing Repository ====

First, we need to add the Syncthing repository and its PGP key for package verification.

<ol style="list-style-type: decimal;">
<li><p>Create a directory for the keyring:</p>
<pre>sudo mkdir -p /etc/apt/keyrings</pre></li>
<li><p>Download the Syncthing release PGP key:</p>
<pre>sudo curl -L -o /etc/apt/keyrings/syncthing-archive-keyring.gpg https://syncthing.net/release-key.gpg</pre></li>
<li><p>Add the Syncthing stable repository to your APT sources:</p>
<pre>echo &quot;deb [signed-by=/etc/apt/keyrings/syncthing-archive-keyring.gpg] https://apt.syncthing.net/ syncthing stable&quot; | sudo tee /etc/apt/sources.list.d/syncthing.list</pre></li></ol>

<span id="make-sure-syncthing-repository-takes-priority"></span>
==== 1.2 Make Sure Syncthing Repository Takes Priority ====

To make sure the system packages don’t take preference over the ones in the Syncthing repository:

<ul>
<li><p>Create a preferences file for APT:</p>
<pre>sudo nano /etc/apt/preferences.d/syncthing</pre></li>
<li><p>Add the following content to the file:</p>
<pre>Package: *
Pin: origin apt.syncthing.net
Pin-Priority: 990</pre></li>
<li><p>Save &amp; exit the editor (in nano, press <code>Ctrl+X</code>, then <code>Y</code>, then <code>Enter</code>).</p></li></ul>

<span id="install-syncthing"></span>
==== 1.3 Install Syncthing ====

Now that we’ve added the repository and made sure its priority, let’s install Syncthing:

<ul>
<li><p>Update the package lists and make sure your system is up to date:</p>
<pre>sudo apt-get update
sudo apt-get upgrade -y</pre></li>
<li><p>Install Syncthing:</p>
<pre>sudo apt-get install syncthing -y</pre></li></ul>

<span id="step-2-setting-up-syncthing-as-a-system-service"></span>
=== Step 2: Setting Up Syncthing as a System Service ===

To have Syncthing start automatically on system boot, even without user login, we’ll set it up as a <code>systemd</code> service that runs as our user, even if we haven’t logged in yet.

<span id="create-a-systemd-service-file"></span>
==== 2.1 Create a Systemd Service File ====

<ol style="list-style-type: decimal;">
<li><p>Create a new service file:</p>
<pre>sudo nano /etc/systemd/system/syncthing@$USER.service</pre></li>
<li><p>Add the following content to the file:</p>
<pre>[Unit]
Description=Syncthing
Documentation=man:syncthing
After=network.target

[Service]
User=%i
ExecStart=/usr/bin/syncthing -no-browser -gui-address=0.0.0.0:8384
Restart=on-failure
RestartSec=5
SuccessExitStatus=3 4
RestartForceExitStatus=3 4
# Harder
ProtectSystem=full
PrivateTmp=true
SystemCallArchitectures=native
MemoryDenyWriteExecute=true
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target</pre></li>
<li><p>Save and exit the editor, hit <code>Ctrl+X</code> then <code>Y</code> to save.</p></li></ol>

<span id="configure-the-service"></span>
==== 2.2 Configure the Service ====

<ol style="list-style-type: decimal;">
<li><p>Enable the service:</p>
<pre>sudo systemctl enable syncthing@$USER.service</pre></li>
<li><p>Start the service:</p>
<pre>sudo systemctl start syncthing@$USER.service</pre></li></ol>

<span id="step-3-securing-syncthings-web-interface"></span>
=== Step 3: Securing Syncthing’s Web Interface ===

By default, Syncthing’s web interface is accessible from any device that can reach your server. This makes it very important to secure the interface with a strong password.

<gallery mode="packed-hover" heights=250 widths=400 perrow=2>
File:lu55028jxef6_tmp_a1d1627a.png
File:lu55028jxef6_tmp_7548e599.png
File:lu55028jxef6_tmp_79b6bf05.png
File:lu55028jxef6_tmp_e4e01235.png
File:lu55028jxef6_tmp_3e71e99.png
</gallery>

<span id="access-the-web-interface-1"></span>
==== 3.1 Access the Web Interface ====

# Open a web browser and navigate to <code>http://192.168.5.5:8384</code> or <code>http://androidstuff.home.arpa</code>.
# You should see the Syncthing web interface.

<span id="add-a-gui-password"></span>
==== 3.2 Add a GUI Password ====

# In the web interface, click on the “Actions” button (gear icon) in the top right corner.
# Select '''“Settings”''' from the dropdown menu.
# In the Settings page, scroll down to the '''“GUI”''' section.
# Find the '''“GUI Authentication User”''' field and enter a username.
# In the '''“GUI Authentication Password”''' field, enter a strong password.
# Check '''“Use HTTPS for GUI”''' so we can visit the server using https://androidstuff.home.arpa:8384 instead. It’s a good habit. :)

<blockquote>'''Note:''' Choose a complex password so some random person who attaches to your home wifi if you forget to set up a guest network that has no LAN access can’t mess with your Syncthing configuration.
</blockquote>
<ol start="7" style="list-style-type: decimal;">
<li>Click '''“Save”''' at the bottom of the page.</li>
<li>Syncthing will prompt you to confirm the changes. Click '''“Yes”''' to apply the new settings.</li>
<li>You’ll be logged out and prompted to log in with your new credentials.</li>
<li>Attempt to access the interface again. You should be prompted for the username and password you set. If not, you messed something up. Do not pass go, do not collect $200, until this asks you for a password to log in.</li></ol>

<span id="step-4-configuring-syncthing-discovery-settings"></span>

=== Step 4: Configuring Syncthing Discovery Settings ===

<span id="understanding-discovery-methods-why-we-dont-use-them."></span>
==== 4.1 Understanding Discovery Methods &amp; why we DON’T USE THEM. ====

Discovery methods are how the syncthing app on your phone will “find” the server you set up as your backup server.

<blockquote>'''NOTE:''' Our server has a static IP: <code>192.168.5.5</code>. We went through the trouble to make sure it always lives at <code>192.168.5.5</code> via static mappings in '''pfSense''' and configuring a static IP in the server’s networking settings. Our server will '''always''' be present at <code>192.168.5.5</code> or <code>androidstuff.home.arpa</code> while we are connected via VPN. All Syncthing “discovery” is doing is trying to find our machine, but why use a find feature when we already know where it is? This adds another point of failure for no good reason! Think of it like making your iPhone invisible &amp; then enabling '''“find my iPhone.”'''
</blockquote>
This setup we are installing syncthing onto has the following:

# A static IP configured, so that it is '''always''' <code>192.168.5.5</code>
# A static IP mapping configured in our router, so that no other device on our network can ever steal <code>192.168.5.5</code> from the computer running syncthing.
# A static hostname of <code>androidstuff</code> that does not change.
# Dynamic DNS for our main internet connection, so when we are outside our network our '''pfSense''' router &amp; <code>FreeDNS</code> will make sure that <code>louishomeserver.chickenkiller.com</code> always points to our home network IP address.

'''I will showcase local discovery failing on video.''' It ''“works”'' when I initially connect to my server via QR code &amp; visiting it in the browser, but fails when I try to connect again. This is because my VPN is on network <code>192.168.6.0/24</code> and my Syncthing is on <code>192.168.5.0/24</code>. I was hoping local discovery would be “smart” enough to remember the last IP address my server was on since it had not changed, but it did not.

'''NEVER RELY ON SOMETHING ELSE TO BE “SMART” IN SOLVING A PROBLEM THAT DOES NOT HAVE TO EXIST IN THE FIRST PLACE!'''

<span id="local-discovery-do-not-trust"></span>

==== 4.2 Local Discovery – DO NOT TRUST! ====

Local discovery allows Syncthing to find other devices on your local network automatically. Key word, ''local'' – meaning your subnet of <code>192.168.5.0/24</code>. What if you connect via your VPN, which is on <code>192.168.6.0/24</code>?

When we first add the QR code of our Syncthing instance to our Android phone Syncthing app, Syncthing will connect to our desktop server running Syncthing. HOWEVER: our Android application will NOT find the Syncthing server the NEXT time we connect. THIS IS BAD!!

This is even worse than it not working at all, as it will give the false impression that it works. This is how people who have set up “backup solutions” end up as customers of Rossmann Repair Group paying $2000 to recover a hard drive that fell off a balcony.

<span id="connecting-reliably-to-syncthing-without-discovery-hassles"></span>
===== Connecting Reliably to Syncthing without Discovery Hassles =====

This situation is actually '''worse''' than if Syncthing had no Local Discovery feature at all. If it didn’t work from the start, you’d know you couldn’t rely on it and would just hardcode the IP of your Syncthing server right into your Android app, using the server’s local IP to connect directly.

What’s dangerous is that Syncthing’s Android app connects the first time by scanning the QR code on the server, making it seem like it’s actually discovering your computer. But it’s not. Next time you try to connect—especially if you’re on a different subnet via VPN—it’ll fail to find the server.

'''Syncthing doesn’t even remember the last IP address it used, so it ends up trying to rediscover it, failing again.'''

I get it. If it can’t find the server on a different subnet when you’re using a VPN, fine, but it’s dangerous that Syncthing doesn’t try the last known IP to see if it still works.

'''TL;DR – to avoid becoming a data recovery customer, don’t trust local or global discovery. Just use the IP address of the server, which in our case is <code>192.168.5.5</code>, and check that it works three separate times under three separate conditions before ever assuming that it is working, as you should with ANY backup solution!'''

<span id="global-discovery"></span>
==== 4.3 Global Discovery ====

Global discovery helps Syncthing find your devices over the internet. It works by periodically announcing your device’s presence to global discovery servers.

* '''Privacy Implications:''' Higher risk, as it involves sharing your device’s information with external servers. This could potentially expose:
** Your IP address
** The fact that you’re using Syncthing
** When your device is online

The bigger issue with this is not privacy, ''it’s that it is unnecessary'' and adds another point of failure over entering the hostname manually.

<span id="configuring-discovery-settings"></span>
==== 4.2 Configuring Discovery Settings ====

# '''Access Syncthing Settings'''
## Open the Syncthing web interface (typically <code>https://192.168.5.5</code> or <code>https://androidstuff.home.arpa:8384</code>).
## Click on the “Actions” button (gear icon) in the top right corner.
## Select “Settings” from the dropdown menu.
# '''Adjust Discovery Settings'''
## In the Settings page, scroll to the “Connections” section.
## Find the following options:
##* '''Enable Local Discovery:''' Keep this checked.
##* '''Enable Global Discovery:''' Uncheck this box.
## Click “Save” at the bottom of the page.
## Syncthing will prompt you to confirm the changes. Click “Yes” to apply the new settings.

<span id="step-5-connecting-server-syncthing-to-android-syncthing"></span>
=== Step 5: Connecting server syncthing to android syncthing ===

<gallery mode="packed-hover" heights=250 widths=400 perrow=2>
File:lu55028jxef6_tmp_90112fd.png
File:lu55028jxef6_tmp_5f13fe67.png
File:lu55028jxef6_tmp_98846264.png
File:lu55028jxef6_tmp_7cd671c.png
File:lu55028jxef6_tmp_9884f00d.png
File:lu55028jxef6_tmp_1527f750.png
File:lu55028jxef6_tmp_27000b93.png
File:lu55028jxef6_tmp_2a5ef23.png
File:lu55028jxef6_tmp_66c6b48d.png
</gallery>

<span id="connect-to-your-vpn."></span>
==== 5.0 – Connect to your VPN. ====

Your android phone must be connected to your VPN for you to connect to your server if your phone is not on the same wifi network as the virtual machine running the syncthing server.

<span id="install-syncthing-from-the-f-droid-store."></span>
==== 5.1 Install syncthing from the f-droid store. ====

* Go to the ''[https://f-droid.org/en/packages/com.github.catfriend1.syncthingandroid/ F-Droid Store to install syncthing-fork]''
* Upon starting syncthing, provide it permissions for notifications.
* Permissions for location are not necessary.

<span id="avoid-becoming-a-data-recovery-customer"></span>

==== 5.2 Avoid becoming a data recovery customer ====

'''Delete the Camera Folder''': Not from the device, just from the sync list, within syncthing. Tap on the camera folder &amp; hit the trash bin in the upper right.

There’s a good reason for that. You might think, “Why? I WANT to sync and back up my photos and videos!!” Here’s the thing: sometimes, camera apps switch folders without you knowing. I’ve seen cases where photos were saved in a different folder INSIDE the DCIM folder, and the gallery app only showed one specific folder.

I’m not a predatory technician that ''[https://www.youtube.com/watch?v=OVZTBhVV5tI&pp=ygUVZHJpdmVzYXZlcnMgIHJvc3NtYW5u bills people $3000 for a bad iPhone screen or charge port]''. But they are out there, and someone was close to paying $500 to a different scam artist data recovery company because their gallery app wasn’t checking a 2nd folder inside of the DCIM folder where another program was saving photos to.

We are not going to back up the camera folder ''within'' the DCIM folder. We are going to back up the '''entire DCIM folder.'''

For those who don’t know, on 99% of Android devices, '''DCIM''' is a folder in the root directory of the ''“visible”'' filesystem within which the subfolders storing your recorded videos &amp; pictures reside.

Next, I am going to do something different. I wanted to show you what happens when you use local discovery/dynamic rather than inserting your actual server IP address into the server field. This meant including screenshots from a LATER step, after I had already added folders that we are going to sync, to show you how syncthing fails with local discovery. '''It’s important to me that you understand how this fails with images for yourself, so you don’t create a setup that makes you a data recovery customer.'''

<gallery mode="packed-hover" heights=250 widths=400 perrow=2>
File:lu55028jxef6_tmp_621c170c.png
File:lu55028jxef6_tmp_fb6bf453.png
File:lu55028jxef6_tmp_364a837e.png
File:lu55028jxef6_tmp_d9c45480.png
File:lu55028jxef6_tmp_1948d4b.png
File:lu55028jxef6_tmp_2bfff860.png
File:lu55028jxef6_tmp_2a5c001f.png
</gallery> hEREEEEEEEEEEEEEE<gallery mode="packed-hover" heights=250 widths=400 perrow=2>
File:lu55028jxef6_tmp_cc3f5925.png
File:lu55028jxef6_tmp_62d8fd43.png
File:lu55028jxef6_tmp_f74f4252.png
File:lu55028jxef6_tmp_cb876287.png
File:lu55028jxef6_tmp_123b314.png
File:lu55028jxef6_tmp_4e264fbe.png
</gallery>

'''Here is what will happen if you set this up with dynamic, disconnect, and then reconnect. Note how it shows up as “idle” for syncing and “disconnected” on the android phone; it is transferring NOTHING, even though the desktop syncthing server GUI shows that we are out of sync.'''

<gallery mode="packed-hover" heights=250 widths=400 perrow=2>
File:lu55028jxef6_tmp_b52ee6c4.png
File:lu55028jxef6_tmp_30c1edaf.png
</gallery>

<span id="add-a-device-to-syncthing-android-app"></span>
==== 5.3 Add a device to syncthing android app ====

# On the top, you’ll see '''Folders''' and '''Devices'''.
# Tap '''Devices'''.
# Tap the plus in the upper right corner to add a device.
# Tap the QR code next to '''Device ID''' in the upper right.
# Go back to the '''Ubuntu Server Syncthing Web Interface'''.
#* Open a web browser and navigate to <code>http://192.168.5.5:8384</code> or [http://androidstuff.home.arpa:8384/ http://androidstuff.home.arpa:8384].
# Obtain Device ID and QR Code
#* In the web interface, click on the blue gobbledygook of numbers &amp; letters next to '''“Identification”''' under '''“This Device”''' (gear icon) in the top right.
#* Select '''“Show ID”'''.
#* You’ll see a QR code and the device ID. ''SCAN YOURS. DO NOT SCAN MINE. I SHOWED A PICTURE OF MINE SO YOU CAN SEE WHAT IT LOOKS LIKE.''
# Configure Device Settings on Android
#* '''Device Name:''' Enter a recognizable name (e.g., “Ubuntu Server”).
#* '''Addresses:''' ''DO NOT CHOOSE DYNAMIC. USING DYNAMIC WILL CAUSE IT TO NOT SYNC WHEN YOU DISCONNECT &amp; RECONNECT FROM YOUR NETWORK. IT WILL WORK THE FIRST TIME, AND THEN NEVER SYNC AGAIN, AND YOU WILL BE PAYING DATA RECOVERY DOUCHEBAGS TO RECOVER YOUR PHONE.''

<blockquote>'''How dynamic failed:''' I used “dynamic” as an example of why it doesn’t make sense to use autodiscovery when you KNOW where your server is. I chose dynamic, and it connected &amp; worked. When I disconnected from my network &amp; reconnected, the ''Devices'' tab in the Syncthing Android app showed me to be ''disconnected'' and the ''Folders'' tab showed the folders to be ''idle'' even though the web GUI for Syncthing said that my folder was ''Out of sync'' and ''Remote Devices'' showed my phone as ''Disconnected''.
</blockquote>
<ul>
<li><p>FILL IN '''“Address”''' when adding a device as follows, if you used the setup I was using within this guide to Syncthing.</p>
<pre>tcp://192.168.5.5:22000</pre></li>
<li><p>OR</p>
<pre>tcp://androidstuff.home.arpa:22000</pre></li>
<li><p>The format is <code>tcp://</code>, then your IP address, then <code>:22000</code> for the port.</p></li>
<li><p>No need to check “Introduce new devices”.</p></li>
<li><p>'''Did you include the <code>tcp://</code> at the beginning, and the <code>:22000</code> at the end for the port? You’d better have!'''</p></li>
<li><p>Save and continue.</p></li></ul>

<ol start="8" style="list-style-type: decimal;">
<li><p>'''Approve the Connection on Ubuntu Server'''</p>
<ul>
<li><p>Return to the Ubuntu Server web interface.</p></li>
<li><p>You should see a prompt to add a new device.</p></li>
<li><p>Verify the Device ID matches your Android device.</p></li>
<li><p>Click '''“Add Device”'''.</p></li>
<li><p>Set a name for the Android device (e.g., “Android Phone”).</p></li>
<li><p>Click '''“Save”'''.</p></li></ul>
</li>
<li><p>'''Check the Connection'''</p>
<ul>
<li>On both devices, check that the other device appears as connected. The connection might take a few moments to establish.</li></ul>
</li></ol>

<blockquote>'''Note:''' Make sure that port <code>22000</code> (or your configured Syncthing port) is open in your Ubuntu Server’s firewall for incoming connections from your local network. B'''y default <code>ufw</code> is not running and blocking things when you first boot Ubuntu Server''' but that may change at a later date, same way they snuck in the suggestion of pre-installing a snap version of Docker.
</blockquote>
Now you’ve added your Ubuntu Server Syncthing instance to your phone; no open ports, will sync whenever you are on wifi with your VPN on, and continuously back up your phone. Beautiful. :)

<blockquote>'''REMEMBER – DO NOT SET “ADDRESSES” TO “DYNAMIC” – TAP “DYNAMIC” AND REPLACE IT WITH''' <code>tcp://youripaddress:22000</code> '''REPLACING “youripaddress” WITH THE IP ADDRESS OF THE VIRTUAL MACHINE THAT IS RUNNING SYNCTHING.'''
</blockquote>
<gallery mode="packed-hover" heights=250 widths=400 perrow=2>
File:lu55028jxef6_tmp_65626c83.png
File:lu55028jxef6_tmp_135ac5ce.png
File:lu55028jxef6_tmp_ffc850bf.png
File:lu55028jxef6_tmp_23122602.png
File:lu55028jxef6_tmp_4468da5a.png
File:lu55028jxef6_tmp_a83ae883.png
</gallery>

<span id="step-6-configuring-syncthing-for-organized-android-backups"></span>
=== Step 6: Configuring Syncthing for Organized Android Backups ===

<span id="configure-android-syncthing-app"></span>
==== 6.1 Configure Android Syncthing App ====

# Open Syncthing on your Android device.
# For each folder you want to sync:
#* Tap the plus icon in the upper right in the folders part of the app.
#* Tap folder label and label it.
#* Tap the directory and choose your directory you want to sync (it’ll let you choose everything besides the download folder on android).
# '''MAKE SURE TO TOGGLE THE SERVER SWITCH UNDER WHERE YOU TAPPED TO CHOOSE THE DIRECTORY YOU WANTED TO SYNC SO THAT IT ACTUALLY BACKS UP.'''
#* Choose ''send &amp; receive'' if you want two-way folder sync.
#* Choose ''send'' if you want it to only send files to your server.
#* Choose ''receive'' if you only want it to receive files from your server

A good rule of thumb: For smaller folders and stuff you transfer to your phone to read on a trip, audiobooks, etc., I choose ''SEND &amp; RECEIVE'' so I can transfer both ways. For stuff like videos I record and photos I take (the DCIM folder), I choose ''SEND ONLY''. I have a 256 GB phone, and over 1.3 terabytes of videos I have recorded… I can’t sync all of that to my phone or it will fill up. But I have less than 1 GB of audiobooks, books, and max 20 GB of movies I am watching at any given time on my phone.

<ol start="7" style="list-style-type: decimal;">
<li>'''Tap checkbox in upper right corner when done.'''</li></ol>

<span id="syncing-on-wifi-only-yes-or-no"></span>
==== 6.2 Syncing on wifi only – yes or no? ====

Your Android device can connect to Syncthing, and you can configure Syncthing while you’re on the go. But by default, your Android device must be on wifi in order for file transfer and backup to occur. Even if you are connected to your VPN, your Android device is not going to transfer files if you are not on wifi.

The way you change this is by editing the folder settings in the Syncthing Android app, and disabling the “sync on wifi only” option. I would suggest doing this for folders with SMALL files like documents, audiobooks, and not for folders with LARGE files like the DCIM folder with your recorded videos and camera pictures.

Unlimited plans have data caps; try using 200 GB in 10 days on any ''“unlimited”'' wireless data plan in the United States and watch your ''“unlimited 5G”'' turn into a 56k modem. The only reason they can market using this wankery is because consumer protection law in the United States is a joke.

<gallery mode="packed-hover" heights=250 widths=400 perrow=2>
File:lu55028jxef6_tmp_32088dd0.png
File:lu55028jxef6_tmp_40872f6d.png
File:lu55028jxef6_tmp_5f537918.png
File:lu55028jxef6_tmp_5b05dae0.png
File:lu55028jxef6_tmp_a7b2093d.png
File:lu55028jxef6_tmp_16c65783.png
File:lu55028jxef6_tmp_19a75a7a.png
File:lu55028jxef6_tmp_ceaf4d3d.png
File:lu55028jxef6_tmp_9bd9aead.png
</gallery>

<span id="accept-folders-on-ubuntu-server"></span>
==== 6.3 Accept Folders on Ubuntu Server ====

# On the Syncthing web interface of your Ubuntu server, you’ll see notifications for new folders.
# For each folder: Click '''“Add”'''.
# CHANGE THE BASE DIRECTORY FROM <code>~/(foldernamehere)</code> '''to''' <code>~/androidbackup/(foldernamehere)</code> so you don’t clog up your base directory. This makes it easy to see in one click what everything we’re backing up from the android phone is.
# Click '''Save'''.

<span id="creating-new-folders-on-ubuntu-server"></span>
==== 6.4 Creating New Folders on Ubuntu Server ====

* It does it for you. What a beautiful program, right? :)

<span id="step-7-verify-and-test-inspect-what-you-expect"></span>
=== Step 7: Verify and Test – INSPECT WHAT YOU EXPECT! ===

'''Don’t become a data recovery customer. Syncthing is used for backing up your phone - arguably the most important part of this entire process.'''

# 99% of the people who show up for data recovery at a data recovery business thought their data was backing up.
# '''It was not.'''
# Use common sense, look through the folders on your server, look at the web interface, make sure things open.

'''You now have working Android backups!'''

* All folders from your Android device will be organized within the <code>~/androidbackup</code> directory.
* Each Android folder will have its own subdirectory for better organization.

<span id="step-3-installing-onlyoffice-workspace-and-wsgidav-to-replace-google-docs"></span>