Storage for 8+ TB?

Hi to the Forum

I have a bunch of sample libraries stored on 2 external hard drives (2TB and 4 TB). I am runnning out of space. I think I should consolidate them on a new external hard drive and alos get a drive with more space. In online searches I see a G-technology 10 TB 7200 rpm drive on various sites but very few other high-capacity (8TB+) drives that run at 7200 RPM. Can anyone recommend a good drive or other solution for safely storing all this data?
Thanks,
Manny

 

you could do a JBODs in a case. If you want to live on edge, set them as RAID0 stripes. if you want to be save. Setup as RAID 5 or 6.

Something like this and stuff with large SATA drives.

https://www.rosewill.com/product/ro...ultiplier-tray-design-controller-card-bundle/

 

I have four drives setup as RAID 10, 4x 6tb drives. Speed, space, and safety.

 

Does it behave like a normal single big drive? Can it be partitioned?

 

That's the point of RAID, to behave as a single drive. Partitioning is the devil... I never partition any drives anymore.

 

depends how its setup. if its raid storage setup on controller on the hardware, then it will show up in the OS as one big chunk of an unpartitioned disk and you can partition it any way you want to in the OS and create whatever file system you need on them.

Most RAID setups are for redundancy in case of disk failure. usually RAID 6 or 5. Sometimes RAID 10. The issue is cost. RAID 10 is a 50% penalty. So if you buy 4 8tb disks, you dont get 4 x 8 TB storage. you get 2 x 8. Do you blew 50% of the cost of the disks in redundancy. Its not a big deal when its cheap SATA drives but becomes an issue if you use high end disks which are $2000 per disk.

It really depends what you want it for and how you want to handle failure. If its data that mostly rests and does not change and is not accessed all the time. you just need it on hand attached to your system in some way, .one option is RAID 0 stripe. You get the full capacity of the disks you invested in and you get speed. You could use something like Glacier to backup.Where you will get bit in the ass is, if one of the disks fails, you have to buy a new disk and rebuild the volume and restore all 32TB from backup or Glacier.

https://aws.amazon.com/glacier/

 

I understand, my question was aimed towards if they function exactly the same as I have no experience. It's been quite a few years since I needed to partition a drive, is it dangerous specifically for a raid or just generally?

My interest in Raid is a safe but consolidated backup + data archiving so I stop dealing with 1 ton of different external hd's.

I have 2 internal SSD's with different OS's + 2 internal data drives, backing up the OS ones safely requires cloning so they're bootable in case something happens, to be bootable they need to be seen by the system as a separate drive, afaik partitioning is the only way to do that.
So far i have been using one external HD per OS drive to backup +1 for data... I was thinking a single multi-partition raid enclosure could simplify my setup but if partitions can cause issues then it's not an option.

Half the capacity for safety is fine by me as long as it's ok to partition it and use different filesystems.

 

In general, with platter drives, the more partitions you do, the more each successive partition becomes slower and slower, due to the geometrical nature of the spinning drive. For SSDs, partitioning drives just makes you lose valuable hard drive space to formatting tables...

 

Oh... I see, makes sense, the outer edge takes more rotation cycles to read than the center.. I guess. For my purposes a bit of performance loss is not a problem. I guess i'll do some proper research and probably invest on Raid.

 

It's the other way around. Partitions start from the outside in. Outer edge is faster than inner edge at the same rotation speed.

 

Yes, thats usually what you want to use them for. So then the decision comes down to do you want to survive 1 or 2 drive failures.

RAID 5 - 1 drive can only fail = must replace it ASAP. More usage space.
RAID 6 - 2 drives can fail == you still should replace ASAP the failed one. Less usable space.

There are various performance gains with various ones but it really depends on application and file system and block size e.t.c. In most case its more about how many drives are you willing to let fail.

As far as partitioning them. Really no point unless you are using different ACLs or you want to do weird volume based encryption. Just make folders and share on home network or direct connect via ether cable. I only split out partitions on *nix based server systems so that a /var/log folder does not bring down the entire root volume and OS and applications.

 

I have the four drives set up so that it makes "one" 12TB drive; and is seen as such. Over 6GB/eSATA it looks like internal drive, and the box auto on and auto off with the Mac which is nice. Using "striped" it runs faster than normal single drive as it's using two 'lanes' to access the spinning drives; so two 6GB drives (this was the sweet spot on price couple of years back, might not be now). In my case (pun intended), since I don't want to ever reinstall all these libraries ever again -- I also it up where the other 2 drives are mirroring the first two (RAID 1+0 = '10'); so speed of 'striped' with mechanical drives, safety of backup drives in case one fails. I don't 'partition' it as works better to have everything in one big volume. Also, since I installed everything fresh on the new RAID; everything is very clean and not very fragmented, aside from the minor stuff one gets from updating existing libraries. It's very fast -- nowhere near SSD or NVME/PCI stuff --- (or SSD on PCI card); but way faster than a single 12TB drive.

 

Please be aware that RAIDs aren't a substitution for backups: if you delete something by accident or if you get hit by a virus, the data is lost regardless of how many discs you have in your RAID. RAIDs only minimize the failure rate or speed up access times of disc space but they are not a backup strategy. You still need to copy your data over to another disc and best keep one copy out of home (e.g. a HDD copy at you work place or a friend's/family's home; cloud space like AWS Glacier or Gcloud Coldline storage).

 

Yep. That's why RAID 1+0 is such an awesome thing; you have complete spare set of drives for your main drives. We have three such setups here; two desktop and one rack mount. Much better than old days of RAID 5; or simple 'striped' volume. With RAID 1+0 you have perfect mirror copies of everything running on two additional lanes so no slowdown of the main striped RAID; speed + safety.

[ 6TB ][ 6TB ] = 12TB 'voilume'
[ 6TB ][ 6TB ] = 12TB 'realtime backup'

And - duh - yes, you should be keeping safety backups of your *work* (songs, recordings, demos) and libraries and installers in more than one place. We use Google's cloud backup here since it's browser based and don't have to use Cloudberry app with AWS. We keep local backup compressed as RAR encrypted files on local disk in fireproof safe; and cloud backup of encrypted RAR files (with updated RAR software without the security flaw, obviously).

 

I dont plan on replacing my system drives with raid, only the backups+archieve-data would be stored in it, so it would be the backup of all my internal ssds.

 

EvilDragon... in reference to my question (and as an inexperienced storage buyer) is RAID the way to go for larger capacities? And...(anyone) I would be grateful for a specific walk thru of what to buy to arrive at approx 10-12 TB of storage that "looks" like a hard drive. Any non-RAID solutions would also be welcome.

A related question. Many lower RPM drives seem to have very high Data transfer rates. For instance, if a Data transfer rates is "up to 5 GB/sec, do I need to still demand 7200 RPM???). For instance:
https://www.bestbuy.com/site/wd-eas...YnjWCSowExrXN5ZTj9G2EaAgwh8P8HAQ&gclsrc=aw.ds

 

IMHO no, you don't go RAID for larger capacities. That means striping the drives, which is gambling with safety (half data gets written on one drive, half on second, if one craps out, you're out of luck unless you do a deeper RAID that does striping and mirroring simultaneously...)

HDDs are absolutely not able to transfer up to 5 GB/sec. 5 GB/s is just maximum transfer rate for USB 3.0 protocol. HDDs cannot dream of ever achieving those speeds.

 

What he said . Definitely important for anyone new to the world of RAID not to think of it as a backup solution. It certainly has its uses as outlined above but can leave you in dire straits when least expected. Silent data corruption is real and there’s nothing that can be done about it if detected too late. Copies away from home are always a good idea. Fortunately, online storage is getting cheaper. Amazon recently released Deep Glacier which is 1/4 the cost of Glacier. The caveat is longer recovery times but I guess the stuff that ends up here isn’t stuff one plans to use frequently anyway, so that may not matter.

 

FWIW, I would get an external enclose and setup RAID 6 or 10 be done with it. There are other options than USB 3 if you need faster speed.