Proposed File Hierarchy system

The following is a proposal for my home network's main NAS filesystem.

Contents

Priority levels

Priority levels are assigned to each subdirectory in /. They represent how critical each directory is for offsite backups and online mirroring. Priority levels are as follows:

Priority Level Description Example types of data Minimum RAID Redundancy Level Mirror Redundancy Level
Priority 1 (P1) Unique data. Irreplacable if lost. Protect at all costs. Original works, precious photos, etc. Preferrably RAIDZ3 but RAIDZ2 is okay if there are many tested backups. Offsite mirror and frequent online synchronizations.
Priority 2 (P2) Unique data that is (not easily) replaceable. Kind of a pain in the ass actually. Things like VM configurations, config files, etc. RAIDZ1 Preferrably on and offsite mirror so the data is both redundant in the event of disaster but also quickly replaceable without having to transfer from the off-site mirror.
Priority 3 (P3) Non-unique data that would be very difficult to replace. Includes collections of things that were randomly scavenged from around the internet, and are probably still out there. Items that cannot be programmatically replaced. So for example, images randomly saved from the internet, parts downloaded from Thingiverse. Programmatically replaceable data is things like Movies and TV. RAIDZ1 Offsite mirror is sufficient.
Priority 4 (P4) Non-unique data that can be replaced somewhat easily, but would take a long time and still involve some effort. Note that some items in my plex library are actually kind of rare and were difficult to find/download on the internet. So some of this data is probably P3. Example: plex library. RAIDZ1 Offsite mirror if space allows.
Priority 5 (P5) Non-unique data that can be lost without consequence. Octoprint log files, freenas scrub reports, unintentionally but knowingly duplicated data RAIDZ0 stripe (though I don't have a pool at RAIDZ0 yet anyway so thiswill be RAIDZ1 just because that's all I have for now. No mirrors are required.

Backup and Restore Strategy

Onsite backups will be to the co-located TrueNAS VM running in my XCP-ng server. Disks are local pass-through and not iSCSI (of course). Sufficient restore tests will be required to make that machine eligible to be an on-site backup. On-site backups can be used as high availibility when both datasets are available or as failover when one fails. Used for recovering from accidental deletions where snapshots can't work for some reason or RAID configuration mistakes.

For priority levels where offsite backup level is required, datasets will sync via TrueNAS replication where possible with the off-site TrueNAS server at [undisclosed location]. [Undisclosed location] has a 1000Mbit down, 10Mbit up connection. Backing up to the server would be fine, but restoring from backup would be very slow for large/multitudinous files. I would be better off restoring large backups via sneakernet.

Internet backups will occur via cloud sync tasks in TrueNAS. They will backup to my "unlimited" cloud storage provider [REDACTED] and if I'm feeling extra ambitions I can try to figure out how to backup to [REDACTED] as well.

Backup tasks should occur automatially. Restore audits should take place regularly. (Every few months?)

Tree

├── archive
│   ├── company-data
│   ├── contents.md
│   ├── in-place-archives
│   └── school-data
├── archivist-activities
│   ├── archive-team
│   ├── contents.md
│   ├── local-caching
│   ├── website-mirroring
│   └── youtube
├── audio
│   ├── audio-books
│   ├── contents.md
│   ├── music
│   ├── podcasts
│   └── recordings
├── backups
│   ├── contents.md
│   ├── smart-phone-backups
│   └── website-takeouts
├── binaries
│   ├── bios-images
│   ├── contents.md
│   ├── cracked-software
│   ├── drivers
│   ├── installers
│   ├── keygens
│   └── portable-apps
├── business
│   ├── contents.md
│   └── it-company
├── downloads
│   ├── contents.md
│   └── torrent
├── images
│   ├── backgrounds
│   ├── contents.md
│   ├── headshots
│   ├── internet-photos
│   ├── scans
│   ├── screenshots
│   ├── timeline
│   └── upcoming-instagram
├── incoming
│   └── contents.md
├── it
│   ├── contents.md
│   ├── documentation
│   ├── dotfiles
│   ├── iso-images
│   ├── keys
│   ├── server-configs
│   ├── software-config-files
│   ├── software-database-backups
│   └── system-images
├── other
│   ├── contents.md
│   └── video-game-roms
├── personal-records
│   ├── contents.md
│   ├── financial
│   ├── forms
│   ├── legal-documents
│   ├── support-call-transcripts
│   └── warranty-cards
├── projects
│   ├── contents.md
│   ├── creative
│   └── technical
├── readme.md
├── temp
│   └── contents.md
├── text
│   ├── books
│   ├── charts
│   ├── cheatsheets
│   ├── contents.md
│   ├── how-to-guides
│   ├── maps
│   ├── product-manuals
│   ├── schematics
│   ├── textbooks
│   └── textfiles
├── unsorted
│   └── contents.md
└── video
    ├── contents.md
    ├── lynda
    ├── plex-library
    ├── purpose-video
    └── zoneminder

74 directories, 18 files

Description of directories

archive

In place archives, dumpster HDD images

Backup strategy

Files in this directory are priority P1

archivist-activities

Somewhat temporary holding place for my archivist activities until the things I want to keep get sorted into their proper respective locations elsewhere in the filesystem.

Other items may stay here for long term storage.

audio

Music collection, audiobooks, podcasts, recordings.

Backup Strategy

Priority 3. This data is replaceable, but it would be a pain and I'm pretty sure my collection is small enough that I can just take the storage space hit.

backups

Periodic system images, vm images, phone backups, and website takeout exports.

Backup strategy

Priority 3. I can lose this data and get it all back, but those Google takeout exports are like 700GB how bout let's not?

binaries

Installers, drivers, bios images, portable apps, perfectly legally acquired softwares

Backup strategy

Priority 4.

business

Data related to running my business (which I don't have)

Backup strategy

Priority 1

downloads

One of the names I'm using for the directories that I vomit files in to. Any data in this directory is expendable. If it isn't, then that is my fault for not moving data to where it should be.

Backup strategy

Priority 5

projects

Okay, there's a lot of shit in here. This has everything related to my free time livelihood. There's also a lot of unique important data in here.

Backup strategy

Priority 1

images

do_i_look_like_i_know_what_a_jpeg_is.jpg

Backup strategy

Priority 1

incoming

Another alias for expendable data. Data shouldn't live in this directory for very long anyway.

Backup strategy

Priority 5

it

Infrastructre related data. Everything that runs the homelab. I think this directory is small enough to get backed up everywhere.

Backup strategy

Priority 1

other

The only thing I have in here right now is video game roms.

Backup strategy

Priority 2

personal-records

Financial data, taxes, account statements, legal documents, receipts, invoices, warranty cards, invoices.

Backup strategy

Priority 1

temp

Alias for an expendable data. Data shouldn't live in here long.

Backup strategy

Priority 5

text

Books, manuals, schematics, maps, charts, cheatsheets, posters, textbooks, how-to guides

Backup strategy

Small enough to be priority 1

unsorted

I'll deal with this later

video

Plex library, purpose video recordings, lynda videos, zoneminder recordings

Backup strategy

Priority 2. This is a large dataset, I might make this P3. Or just buy more hard drives :)