Jump to content

Project Hard Drive / Storage System


Thomas S

Recommended Posts

Dear cinematography community,

I have build my self a good pc system for various tasks such as: editing, color grading etc. My main fast hard drive for projects was a 1Tb M.2 SSD. It has been working really good (maybe a bit overkill).

Now I ran in to a problem that most of the projects I work on are bigger than 1Tb and now I need a new solution for a fast storage system / drive to do editing and color grading with big RAW files. I have been looking into the Samsung 4Tb SSD (2.5") which would work but if a project is just a little over 4Tb than that would not work.

So does anybody have a good solution to this problem? It needs to be fast and not too expensive. Maybe something like a Hard Drive Raid System where I can stick 4 hard drives with a total of 8Tb and connect that to my pc via thunderbolt or usb-c? My pc does not have thunderbolt connection but I think there are some expansion cards out there?

After a projects is done, it gets archived on a other drive.

Would love to hear from you guys!

 

Best Thomas.

Edited by Thomas S
Link to comment
Share on other sites

Are you expecting to use it for 1 project at a time?  Your RAID idea is what I would be looking into, especially if you're going to put SSDs in it.

As for Thunderbolt, if your motherboard doesn't have thunderbolt already, you can't just add it via PCIe, you have to have internal TB headers as well.  The expansion cards simply add the ports, but still need the underlying Thunderbolt architecture on the motherboard.  Chances are if you don't have any ports on the IO board, you won't be able to add it.

Link to comment
Share on other sites

  • Premium Member

Unless you need it to be portable, I would just build the raid inside the computer. 

https://www.amazon.com/Adapter-Transmission-Technology-Support-2242、2260、2280、22110/dp/B07K52TKF9/ref=sr_1_9?keywords=nvme+pcie+adapter+4+port&qid=1571949792&s=electronics&sr=1-9

Is an 4 bank NVME solution. Put 4x2tb NVME's in there and raid ZERO them. This will give you 6400Mbps throughput with PCI 3 and 8tb worth of storage. If that is too much, there are some PCI based raid cards for Sata and all you do is buy the cable harness and plugin as many sata drives inside the machine as you want. It's a bit slower to do it that way, but it does work well. External boxes suck sadly, they're nowhere near as fast and easy to fail. 

Link to comment
Share on other sites

  • Premium Member

I'm with Tyler on the make-it-internal.

If you want bulk storage, it's still hard to beat spinning metal. If you have the spare power supply capacity and the space, rack up four, six or eight SATA drives and use RAID10. Works like a charm and is absolutely the cheapest, most reliable way to buy storage.

  • Upvote 1
Link to comment
Share on other sites

17 hours ago, Adam Froehlich said:

Are you expecting to use it for 1 project at a time?  Your RAID idea is what I would be looking into, especially if you're going to put SSDs in it.

As for Thunderbolt, if your motherboard doesn't have thunderbolt already, you can't just add it via PCIe, you have to have internal TB headers as well.  The expansion cards simply add the ports, but still need the underlying Thunderbolt architecture on the motherboard.  Chances are if you don't have any ports on the IO board, you won't be able to add it.

Thanks for the information. Will check if my motherboard has TB headers ?

Link to comment
Share on other sites

14 hours ago, Tyler Purcell said:

Unless you need it to be portable, I would just build the raid inside the computer. 

https://www.amazon.com/Adapter-Transmission-Technology-Support-2242、2260、2280、22110/dp/B07K52TKF9/ref=sr_1_9?keywords=nvme+pcie+adapter+4+port&qid=1571949792&s=electronics&sr=1-9

Is an 4 bank NVME solution. Put 4x2tb NVME's in there and raid ZERO them. This will give you 6400Mbps throughput with PCI 3 and 8tb worth of storage. If that is too much, there are some PCI based raid cards for Sata and all you do is buy the cable harness and plugin as many sata drives inside the machine as you want. It's a bit slower to do it that way, but it does work well. External boxes suck sadly, they're nowhere near as fast and easy to fail. 

Ah thats really cool! But I think M.2 SSD are a bit to expensive. 

The PCI raid cards sound intersting. Do you know a good tutorial on that? Can I mix 2.5" SSDs and 3.5" HDs ?

Link to comment
Share on other sites

13 hours ago, Phil Rhodes said:

I'm with Tyler on the make-it-internal.

If you want bulk storage, it's still hard to beat spinning metal. If you have the spare power supply capacity and the space, rack up four, six or eight SATA drives and use RAID10. Works like a charm and is absolutely the cheapest, most reliable way to buy storage.

I have an 850W Powersupply. CPU is a Intel i9 9900k @ 5.0Ghz and GPU is a RTX2080 Ti.

 

Do you have this internal Raid setup? Do you know a good tutorial on that?

Link to comment
Share on other sites

  • Premium Member

There are various calculators available online which allow you to enter all of your system components and determine the total power consumption; I'd use one of those before committing to a purchase.

I set up an 8-disk RAID10 using a Highpoint 2720 disk controller. You can probably find as much information as you need online. If your motherboard has enough SATA ports to accommodate the amount of disks you need, you might not even need that.

Physical installation is straightforward; how you set it up software-wise depends on specifics.

P

Link to comment
Share on other sites

  • Premium Member
5 hours ago, Thomas S said:

Ah thats really cool! But I think M.2 SSD are a bit to expensive. 

The PCI raid cards sound intersting. Do you know a good tutorial on that? Can I mix 2.5" SSDs and 3.5" HDs ?

You can use spinning disc's, but they aren't very fast. SSD's can get 500Mbps each, through a decent raid controller. Hard drives are capped at 150 - 200Mbps. So even if you have 4 HD's, your throughput isn't going to be anywhere near SSD level. No you can't mix SSD and Spinning disc's, that would be a bad idea. 

M.2's are the same price as standard SSD's. In fact, in some cases, M.2's are CHEAPER because they have no packaging. You may not be able to see/get those lower prices in your country tho, that could be why you aren't founding them online. Here state side, I can save at least $15 - $20 on buying M.2's over standard SSD's. 

Here is the pretty standard 8 port card: https://www.newegg.com/supermicro-aoc-sas2lp-mv8-sata-sas/p/N82E16816101792?Description=8 port raid controller&cm_re=8_port_raid_controller-_-16-101-792-_-Product

The special cables you need, you'll see when you scroll down a bit on the page. Each of those big ports on the card is 4 sata ports. So this card will run 8 drives. You can start with 4 and then add 4 later. You'd probably use a bios based raid configuration. I sadly haven't done it in windows, just mac OS. However, I'm certain there is software out there that can do it. 

Link to comment
Share on other sites

On my windows HP z820, I use an internal RAID 0 with 4 drives (16TB total).  If you can't do this in your machine, you will need a RAID controller card and external drive bay to build the RAID.

Since I have not redundancy in my RAID, if one drive fails, all is lost.  So, I keep a back up on large USB drives, just in case of emergency, for all the camera originals.

These days, if I want to spend the money, I could build a RAID of SSD drives, or just place 4 large SSD drives instead of the RAID.  And, if I start to grade 4k+ RAW material, I might just need to do this.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...