“We have Amazon and CJ affiliate links in this article and we earn commission from qualifying purchases at no cost to you. We only recommend products that we use ourselves and would never compromise the integrity of your build. This helps us bring you quality content and keep this site running.” 

It is tempting to buy old used hard drives that are very low cost. 4TB and 8TB drives can be found for as little as $10/TB. Great right? There are two problems with this approach. (If this is you, do not feel bad. I have banished all the 4TB from my farm, and working on the 8TB next)

TL:DR buy big drives, 16TB and 18TB are the sweet spots today. You will use 4-4.5x less space, have 4-16x fewer drive failures, and consume 4-4.5x less electricity (and power bill). Internal drives with 5 year warranty will be less of a hassle, but generally, the same models can be found in high-capacity external HDDs if you are willing to “shuck” them and remove them from the enclosure. 16TB drives are 4x more space and power-efficient than 4TB drives (this is obvious, but is insanely important)

Hard Drive Recommendations

Western Digital:

First Choice: Ultrastar DC HC550 18 TB Hard Drive

Second Choice: 18TB WD Gold

Back up plan: WD Elements 18TB Drives have an enterprise grade drive inside, which is an amazing deal but remember the warranty is void after you shuck the drive!!

Seagate:

Seagate Exos X18 or Seagate 16TB Exos .

Disk failures

Hard drives have moving parts, and over time the motors, heads, and actuators fail. SMART is useful for finding predictive failures and removing a device before the data is lost, but this is tedious for novice users to understand (although I will show you how later). HDDs exhibit a bathtub curve for failures, with device failures increasing as time increases, rapidly accelerating after the 5 year warranty on most models. Device time is measured in power on hours, not physical time (as time with the disk powered off for extended periods of time does not count towards device wear). To make it even more complicated, the higher the workload and temperature the drive was used at, the faster it will fail.

Source: Architecting IT

Backblaze has established themselves as the industry experts on this topic, as they have published their HDD failure rates for individual models for years. For one, HDD vendors have cleaned up their act in the last 5 years. New models are much more reliable than some of the old models were. This doesn’t change the fact that if you are buying a drive with 6 years power on that was ran at a high workload (you don’t know if you bought on eBay) you are putting a ticking time bomb into your farmer. Doing the math, four 4TB drives have 4x higher probability of failing vs one 16TB drive, and then on top of that add the old model higher failure rate, and end of life…you may have up to 8-16x more failures with an old 4TB drive vs a new 16TB drive. WOW.

Tip – how to measure drive lifetime with SMART. You can install smart

sudo apt install smartmontools
sudo smartctl -a /dev/sda

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
 1 Raw_Read_Error_Rate     0x000f   081   064   044    Pre-fail  Always       -       122541488
 3 Spin_Up_Time            0x0003   099   099   000    Pre-fail  Always       -       0
 4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       1
 5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
 7 Seek_Error_Rate         0x000f   064   060   045    Pre-fail  Always       -       2780049
 9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       72
10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       1
18 Unknown_Attribute       0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   077   049   040    Old_age   Always       -       23 (Min/Max 14/28)
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       0
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       174
194 Temperature_Celsius     0x0022   023   040   000    Old_age   Always       -       23 (0 14 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       7 (199 103 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       11364333296
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       99512

Although it is a work in progress project, Scrutiny is a cool idea to have a simple docker container be able to run an monitor the SMART of all the drives on a system through a web interface

Power

In the farming workload, drives do not generally have time to spin down between sub blocks, so the idle power of the drive is close to what the idle power estimate is for the drive model. Chiapower.org details exactly why for Chia, you would want to use the largest drive models. This is what the largest consumers of HDDs in the world, hyperscale data centers like Google, Facebook, Microsoft, and Amazon, use for the most power efficient storage and lowest total cost of ownership. Small drives don’t just consume a little more power per TB than large drives, it scales linearly as the idle power is very similar for drive models.

The highest capacity drives are the most power efficient, in terms of W/TB. This will reduce the total energy consumption for the Chia network and lower the electricity cost for farming.

Let’s take an example of running 180 drives in two 90 bay JBODs. With 90 drives, the total power consumption is around 1000W. With 8TB drives that is 1.4PB, but with 18TB drives that is 3.2PB. The operational expenditure of power cost will be 2x lower per TB!! ($3.41/TB vs $7.66). I cannot overstate the consolidation of increased TB / rack unit, especially in a large hard drive deployment. Using 25% (compared to 4TB) or 50% (compared to 8TB) of the total 3.5in HDD slots to store the same amount of data is a considerable operational improvement, ease of management, and better for the environment by reducing power consumption.

Shucking

If you are looking for what is inside various external hard drives, there is pretty much a YouTube video for every single model. Beware though!! Although the spec changed in 2013…power supply manufacturers route 3.3V to pin 3, which on modern drives is used for device sleep (near-zero idle power for SATA SSDs) or power disable (to turn off drive power to a slot). An easy hack is to use a SATA to Molex power cable, or you can tape off the 3.3v pin on the drive. From what I have heard, this is only an issue on WD external drives when shucked (slightly tweaked models from their 3.5in counterparts)

Source: Serial ATA International Organization Serial ATA Revision 3.4, June 13 2018 Gold Revision