Jump to content
Thomas S

Project Hard Drive / Storage System

Recommended Posts

Dear cinematography community,

I have build my self a good pc system for various tasks such as: editing, color grading etc. My main fast hard drive for projects was a 1Tb M.2 SSD. It has been working really good (maybe a bit overkill).

Now I ran in to a problem that most of the projects I work on are bigger than 1Tb and now I need a new solution for a fast storage system / drive to do editing and color grading with big RAW files. I have been looking into the Samsung 4Tb SSD (2.5") which would work but if a project is just a little over 4Tb than that would not work.

So does anybody have a good solution to this problem? It needs to be fast and not too expensive. Maybe something like a Hard Drive Raid System where I can stick 4 hard drives with a total of 8Tb and connect that to my pc via thunderbolt or usb-c? My pc does not have thunderbolt connection but I think there are some expansion cards out there?

After a projects is done, it gets archived on a other drive.

Would love to hear from you guys!

 

Best Thomas.

Edited by Thomas S

Share this post


Link to post
Share on other sites

Are you expecting to use it for 1 project at a time?  Your RAID idea is what I would be looking into, especially if you're going to put SSDs in it.

As for Thunderbolt, if your motherboard doesn't have thunderbolt already, you can't just add it via PCIe, you have to have internal TB headers as well.  The expansion cards simply add the ports, but still need the underlying Thunderbolt architecture on the motherboard.  Chances are if you don't have any ports on the IO board, you won't be able to add it.

Share this post


Link to post
Share on other sites

Unless you need it to be portable, I would just build the raid inside the computer. 

https://www.amazon.com/Adapter-Transmission-Technology-Support-2242、2260、2280、22110/dp/B07K52TKF9/ref=sr_1_9?keywords=nvme+pcie+adapter+4+port&qid=1571949792&s=electronics&sr=1-9

Is an 4 bank NVME solution. Put 4x2tb NVME's in there and raid ZERO them. This will give you 6400Mbps throughput with PCI 3 and 8tb worth of storage. If that is too much, there are some PCI based raid cards for Sata and all you do is buy the cable harness and plugin as many sata drives inside the machine as you want. It's a bit slower to do it that way, but it does work well. External boxes suck sadly, they're nowhere near as fast and easy to fail. 

Share this post


Link to post
Share on other sites

I'm with Tyler on the make-it-internal.

If you want bulk storage, it's still hard to beat spinning metal. If you have the spare power supply capacity and the space, rack up four, six or eight SATA drives and use RAID10. Works like a charm and is absolutely the cheapest, most reliable way to buy storage.

  • Upvote 1

Share this post


Link to post
Share on other sites
17 hours ago, Adam Froehlich said:

Are you expecting to use it for 1 project at a time?  Your RAID idea is what I would be looking into, especially if you're going to put SSDs in it.

As for Thunderbolt, if your motherboard doesn't have thunderbolt already, you can't just add it via PCIe, you have to have internal TB headers as well.  The expansion cards simply add the ports, but still need the underlying Thunderbolt architecture on the motherboard.  Chances are if you don't have any ports on the IO board, you won't be able to add it.

Thanks for the information. Will check if my motherboard has TB headers 😉

Share this post


Link to post
Share on other sites
14 hours ago, Tyler Purcell said:

Unless you need it to be portable, I would just build the raid inside the computer. 

https://www.amazon.com/Adapter-Transmission-Technology-Support-2242、2260、2280、22110/dp/B07K52TKF9/ref=sr_1_9?keywords=nvme+pcie+adapter+4+port&qid=1571949792&s=electronics&sr=1-9

Is an 4 bank NVME solution. Put 4x2tb NVME's in there and raid ZERO them. This will give you 6400Mbps throughput with PCI 3 and 8tb worth of storage. If that is too much, there are some PCI based raid cards for Sata and all you do is buy the cable harness and plugin as many sata drives inside the machine as you want. It's a bit slower to do it that way, but it does work well. External boxes suck sadly, they're nowhere near as fast and easy to fail. 

Ah thats really cool! But I think M.2 SSD are a bit to expensive. 

The PCI raid cards sound intersting. Do you know a good tutorial on that? Can I mix 2.5" SSDs and 3.5" HDs ?

Share this post


Link to post
Share on other sites
13 hours ago, Phil Rhodes said:

I'm with Tyler on the make-it-internal.

If you want bulk storage, it's still hard to beat spinning metal. If you have the spare power supply capacity and the space, rack up four, six or eight SATA drives and use RAID10. Works like a charm and is absolutely the cheapest, most reliable way to buy storage.

I have an 850W Powersupply. CPU is a Intel i9 9900k @ 5.0Ghz and GPU is a RTX2080 Ti.

 

Do you have this internal Raid setup? Do you know a good tutorial on that?

Share this post


Link to post
Share on other sites

There are various calculators available online which allow you to enter all of your system components and determine the total power consumption; I'd use one of those before committing to a purchase.

I set up an 8-disk RAID10 using a Highpoint 2720 disk controller. You can probably find as much information as you need online. If your motherboard has enough SATA ports to accommodate the amount of disks you need, you might not even need that.

Physical installation is straightforward; how you set it up software-wise depends on specifics.

P

Share this post


Link to post
Share on other sites
5 hours ago, Thomas S said:

Ah thats really cool! But I think M.2 SSD are a bit to expensive. 

The PCI raid cards sound intersting. Do you know a good tutorial on that? Can I mix 2.5" SSDs and 3.5" HDs ?

You can use spinning disc's, but they aren't very fast. SSD's can get 500Mbps each, through a decent raid controller. Hard drives are capped at 150 - 200Mbps. So even if you have 4 HD's, your throughput isn't going to be anywhere near SSD level. No you can't mix SSD and Spinning disc's, that would be a bad idea. 

M.2's are the same price as standard SSD's. In fact, in some cases, M.2's are CHEAPER because they have no packaging. You may not be able to see/get those lower prices in your country tho, that could be why you aren't founding them online. Here state side, I can save at least $15 - $20 on buying M.2's over standard SSD's. 

Here is the pretty standard 8 port card: https://www.newegg.com/supermicro-aoc-sas2lp-mv8-sata-sas/p/N82E16816101792?Description=8 port raid controller&cm_re=8_port_raid_controller-_-16-101-792-_-Product

The special cables you need, you'll see when you scroll down a bit on the page. Each of those big ports on the card is 4 sata ports. So this card will run 8 drives. You can start with 4 and then add 4 later. You'd probably use a bios based raid configuration. I sadly haven't done it in windows, just mac OS. However, I'm certain there is software out there that can do it. 

Share this post


Link to post
Share on other sites

On my windows HP z820, I use an internal RAID 0 with 4 drives (16TB total).  If you can't do this in your machine, you will need a RAID controller card and external drive bay to build the RAID.

Since I have not redundancy in my RAID, if one drive fails, all is lost.  So, I keep a back up on large USB drives, just in case of emergency, for all the camera originals.

These days, if I want to spend the money, I could build a RAID of SSD drives, or just place 4 large SSD drives instead of the RAID.  And, if I start to grade 4k+ RAW material, I might just need to do this.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.



  • Visual Products



    Tai Audio



    Rig Wheels Passport



    Wooden Camera



    Metropolis Post



    FJS International



    New Pro Video - New and Used Equipment



    Media Blackout - Custom Cables and AKS



    CineLab



    Broadcast Solutions Inc



    Ritter Battery



    Just Cinema Gear



    Glidecam



    Paralinx LLC



    Gamma Ray Digital Inc



    The Original Slider



    Abel Cine



    Serious Gear



    G-Force Grips


    Cinematography Books and Gear
×
×
  • Create New...