Talk: File system of choice / with encryption / with compression / deduplication

[_roman_]

This is not real a support topic. More a talk topic

I'm curious on the file system the userbase here uses for their desktop boxes / ...
If possible please provide how you set it up if you want to share.

I use:
Main box:

• GPT -> LVM2 -> LUKS -> BTRFS + ZSTD compression factor3
• tmpfs for lots of temporary junk files / lint files
• swap = none, but 64GiB DRAM
• ext4 - old unreliable 1.5TB SATAII-HDD downloads = source code / junk (package manager checks checksum anyway for those downloads)
• vfat - boot: efistub-kernel

Backup:

• GPT -> LVM2 -> LUKS -> EXT4

Does anyone uses compression, encryption or deduplication?

edit: reformat - forgot about my hdd.
edit2: Deduplication

 

[Solaris17]

no compression here but looks like you and I match otherwise. Left ext4 a bit ago

GPT -> LVM2 -> LUKS -> BTRFS

 

[tpa-pr]

For all of my systems I use GPT layout disks in the following layout:
[GPT]
-> [BOOT] - FAT32
-> [ROOT] - XFS
-> [SWAP] 1:1 RAM

(The only exception is my media server which is the same setup on MBR, since it's an old box).

No compression or encryption. I used to use EXT4 but found it unreliable compared to XFS. I have used BTRFS in the past but it's a bit overkill for my needs so I swapped back to XFS. I have multiple onsite and offsite backups, onsite drives are also formatted as XFS.

 

[eidairaman1]

I can understand compression for archives as long as it doesn't make files become lossy.

 

[R-T-B]

XFS root. I use SEDs and encrypted containers for my encryption needs. Compression I keep to dedicated archived, no in FS stuff.

 

[A Computer Guy]

Windows user here so ...

• NTFS all day every day. No compression on any volumes. Typically use the 4K default cluster size.
• Boot/OS volume 500-ish GB drive C. Whatever is leftover on the device as Data volume drive D.
• NVMe SSD's: Typically I leave some extra free space for overprovisioning.
• Optanes: Pairs striped in Windows Disk Management (become dynamic volumes)
• RAM Drive for temporary files, downloads, etc...
• Even with 64GB to 128GB RAM ... OS managed SWAP file ( sure why not )

Backup: Various levels of NAS, USB, DVD

 

[Timbaloo]

Ext4 because well proven and fast. Will likely switch to Btrfs in the future. Might aswell consider encryption then. Since I don‘t have large amounts of compressible data, I don’t see much point in FS level compression.

 

[_roman_]

I forgot to ask about deduplication feature of the file system.

I have not yet looked into that: https://btrfs.readthedocs.io/en/latest/Deduplication.html

EXT4 is stable. I see the risk of using the same file system for full backups and root.

 

[W1zzard]

For backup, just use restic + rclone to S3 (or any other of its gazillion supported storages) and rclone to duplicate the storage bucket to another continent at any other provider, maybe a third copy on something local. Encrypts data by default before it leaves your machine, incremental, deduplication, compressed, you can mount and/or restore any backup partially/individually

On Windows use Duplicati to S3

 

[tpa-pr]

For backup, just use restic + rclone to S3 (or any other of its gazillion supported storages) and rclone to duplicate the storage bucket to another continent at any other provider, maybe a third copy on something local. Encrypts data by default before it leaves your machine, incremental, deduplication, compressed, you can mount and/or restore any backup partially/individually

On Windows use Duplicati to S3

For BSD/Linux I highly recommend Tarsnap as well: https://www.tarsnap.com/

Relatively cheap, easy to set up and automate and very secure. The guy behind it is also extremely helpful, I've emailed him a couple of times for account changes and he's done it without delay. I've been using this for...6 years now? No issues, even when I had my very jank setup for it under WSL2 to back up my Windows box.

 

[W1zzard]

Relatively cheap

I had to lol .. $0.25 / GB-month

I'm backing up 10 TB, just paid my storage provider $0.0017 per GB per month for it

 

[windwhirl]

Windows here mostly, so that means NTFS. No compressed files or encryption.

I've had my Linux time but then I usually settled for whatever was default (usually ext4), without any peculiar partition arrangements (so, everything on /, no LVM, etc.). I'm mildly interested in atomic COW, deduplication and such but I find it annoying that most of it is just command line only, with one single GUI tool existing as far I saw back in the day, and already dead development-wise when I heard of it.

 

[tpa-pr]

I had to lol .. $0.25 / GB-month

I'm backing up 10 TB, just paid my storage provider $0.0017 per GB per month for it

To my knowledge the initial upload cost will be expensive but then the maintenance after that will be quite cheap. Still, yeah, you're definitely getting the FAR cheaper option there. It seems my knowledge of online storage costs is not that comprehensive!

 

[W1zzard]

To my knowledge the initial upload cost will be expensive but then the maintenance after that will be quite cheap. Still, yeah, you're definitely getting the FAR cheaper option there. It seems my knowledge of online storage costs is not that comprehensive!

The software looks like a solid solution, and if you want to backup only a few gigs, then it's definitely an option. otoh there's free cloud storage options for this size, and with client-side encryption you don't have to worry about them indexing your personal data

 

[Selaya]

what's s3's business model? they charge for in-/egress? egress only? so you're only paying them when you actually have to access your backup?

 

[W1zzard]

what's s3's business model? they charge for in-/egress? egress only? so you're only paying them when you actually have to access your backup?

Depends on the plan you pick. With "S3" I meant S3-compatible storage, of which there's dozens of offerings

 

[Selaya]

oh, ic
you're not using amazon itself tho? last i've heard their business model can be quite succinctly summarised as scam but i've not delved too deep into it, there might be plans that are priced acceptably depending on the use-case
(i know you mentioned oracle cloud having some decent-priced plans for some of tpus use-cases)

 

[igormp]

oh, ic
you're not using amazon itself tho? last i've heard their business model can be quite succinctly summarised as scam but i've not delved too deep into it, there might be plans that are priced acceptably depending on the use-case
(i know you mentioned oracle cloud having some decent-priced plans for some of tpus use-cases)

S3 is expensive, yeah, there are storage, ingress and egress costs, but it's one of the best options for large, performant, distributed object storage platforms out there.

For a regular individual its pricing and complexity is indeed non-sense. You can have a cheaper experience with the likes of Backblaze or Wasabi, which also offer S3-compatible storage.