External Storage for Longhorn Backup

mrgionsi

I have a Talos cluster with two VMs and longhorn installed as block storage. I want to take longhorn backup, on another machine ( Ubuntu) with Docker installed.

https://longhorn.io/docs/1.10.0/snapshots-and-backups/backup-and-restore/set-backup-target/

Longhorn docs report

A backupstore is a server that stores the backups of Longhorn volumes. You can use NFS, SMB/CIFS, Azure Blob Storage, and S3-compatible servers.

I was thinking to use MinIO as S3 in order to be able to move to any other S3 in the future, but Community edition is not provided already built, i.e. you have to build it yourself and then I can move ahead :)

I found some alternative:

Please share your view. I can go also with NFS basic storage.

@imgios @fonzdm @lucaam @fabius

fonzdm

Hi @mrgionsi
that is a great feature to implement in your self-hosted environment!
Before going into details, I would like to know your hardware setup.
You said that the remote machine has Docker installed, which gives good options if we want to go for something beyond NFS; but how the storage is organized? Do you have an external HDD attached to this machine? Is it equipped with internal disks (maybe in RAID)?

I'm asking this because, just giving my two cents:

Using S3 or other complex solutions may need something more resilient than a simple external HDD, since in case of fault of the machine (not the drive) you would need to find a similar build (with docker and so on, so you need to take into account the storage of docker-compose.yml file in some git and so on).
Usually we should have more than one backup, so going to S3 means go with S3 for any other backup solution, and this may be not easy (so that's why it should be useful to have an array, so you can take care of the PVC rebuilt without worrying to much on HDD failures)
With NFS, an external HDD is more than enough: in case of faults, you simply attach it to another machine and you are good to go (the nfs setup is quite "out-of-the-box". And any other backup is as simple as a clone

What do you think?

mrgionsi

Hello mate,
Thanks a lot.
I have an external HDD connected to the remote machine. As of now I don't have a RAID or other backup, so it seems you're suggesting NFS.

And if I told you I want to use an S3 to store SaaS file as well ?

fonzdm

mrgionsi sorry, which SaaS files? Can you be more specific?

mrgionsi

Image files about personal document. Remote check-in

fonzdm

I'm glad we are the only one using this community because people wouldn't otherwise understand what are you saying.
Anyway, as I understand, you would like to have a backup storage that you can restore also on other cloud providers, right?
Meaning you are now self hosting your remote check-in app which saves image files locally.
The goal is? Because if you setup an S3 based backup system, it's just for backup. If you restore the backup on, say, Amazon S3, you still need to implement the S3 integration on your app (since the app access the file locally as simple directories/files).
So let's try to understand the end goal is and find the best solution.