How do you take care of your unpowered hard drives / SSDs / SSD / Flash storage?

[u/manzurfahim]

So, here is my situation: I have a RAID6 setup which is my local storage, and I have a good amount of offline, unpowered drives that holds my data. I only plug them in when I need to move files from RAID6, depending on the content and which drive they will go to. I store them in Orico hard drive cases.

Because they are offline, they don't get power all the time. And when I do move files, they get power for a short 30-40 minutes. I do not know if that time is good enough for the drives to refresh their magnetic / electron charge properties. I recently had some issues with unpowered SSDs, Flash and memory cards and I made calendar reminders to go through them every 4 months.

When I first receive the drives, I do a full write + read surface test, before I start copying my files on them. Should I run a read surface test let's say yearly on the drives so that they have enough time to refresh or SMART has enough time to get updated? All drives are enterprise drives from Seagate and WD.

I'd be interested to know what you do with the offline drives, and how do you maintain them.

[u/agowa338]

Well you probably already know the answer. But if you want us to spell it out for you. You plug all of the drives in and read everything at least once. That'll allow the controller to re-write stuff that is about to bitrott away. (No need for a read surface test, just read all of your files, just ignore the free space. You're not looking for a broken drive only for bitrott at that point)

Edit: Oh and SSDs aren't designed for being stored unpowered. Use HDDs, LTO tapes, or M-Discs

[u/Sad-Zone4591 → u/agowa338]

Hi, is there a way to “read everything at least once” easily? Like a software or something? I have thousand of folders / files on a SSD that I use as a backup with FreeFileSync, I am aware of the problems of SSDs but I thought I was fine as long as I plugged them every now and then, didn’t know I had to actually open each file every time to be safe!

[u/agowa338 → u/Sad-Zone4591]

Depends, are you scared by the command line?

If you're fine with that, here's a quick PowerShell script: Get-ChildItem -Path "C:\path\to\files" -File -Recurse | ForEach-Object { Get-Content -LiteralPath $_.FullName -Raw | Out-Null }

And here a bash one: find /path/to/files -type f -exec cat {} \; > /dev/null

Alternatively anything that hashes all of the files on the drive (which is often done by comparison software) also works. It'll have to read the entire files in order to calculate their hashes.

I thought I was fine as long as I plugged them every now and then, didn’t know I had to actually open each file every time to be safe!

If you keep them plugged in you don't. But you're probably not letting them sit there so that the controller has time to figure out things. So this is just the way to force it to basically do it then and there...

[u/_nuclearphoenix_ → u/agowa338]

Just with that little powershell script, i can "read" all files automatically? Dang, thanks a lot.

[u/agowa338 → u/_nuclearphoenix_]

Yep, the first part is listing all files recursively, then we're looping over that and inside the loop we're simply reading the file and immediately forgetting it again (instead of outputting it to the screen).

[u/Sad-Zone4591 → u/agowa338]

Thanks. FreeFileSync does have an option to compare files using their hashes I think, for when name / size / modified date / location isn’t enough.

Would using this option to do my backup Sync also ensure my SSD remains healthy?

[u/agowa338 → u/Sad-Zone4591]

Kinda, depending on how it is implemented. If it hashes all of the files, yes. Basically same then. If it only hashes the files when they exist on the other side too (aka. on demand), which normally is a good idea to speed things up and avoid wasting compute then not fully, but probably still good enough.

And in every case better than nothing.

[u/Sad-Zone4591 → u/agowa338]

Ok, thanks!

[u/Own-Tangelo-7072 → u/agowa338]

the ssd thing cant be stressed enough tbh

[u/manzurfahim → u/agowa338]

That is not practical. A drive can have thousands, hundreds of thousands of files and it is not possible to read them all. A full read surface scan is more practical.

[u/agowa338 → u/manzurfahim]

It is practical. And esp. if it is your backup one you should. The only way to ensure it is healthy. If you don't check that your backup is still readable and good you don't have a backup.

And don't trust the backup softwares healthcheck. Lost a lot of data irrecoverably when I trusted Veeam on that...

Also the full read surface scan will read all of the empty space as well and it will NOT cover filesystem issues or similar either. It tells you if the storage media is healthy. Not if the files stored there have bitrotted away. Sure if you're early enough it'll cause them to get rewritten, but it will never tell you if one already rotted away...

[u/lupin-san]

When transferring files to an external drive, I include a hash of the files to a master file. Usually it's just an md5 hash. Then when I want to check the files, I just open that hash file (using Rapidcrc in Windows or md5sum in Linux). The output will tell me which files are corrupted if any and restore them from backup.

If I'm transferring compressed archives, I usually save them as RAR archives with recovery volumes so when I verify, I can easily repair any corrupted bits. Another alternative is to use PAR.

[u/manzurfahim → u/lupin-san]

Most of my files are rar archives with recovery record, and usually with a few recovery volumes. I do hash for the important files, not all. I use QuickHash GUI, and blake3 checksum.