JM with Optane
|

Best NVMe for chia plotting on budget build

Best Data Center SSD for Budget Chia Plotting

Summary, TL;DR

Buy a 1.6 or 2TB data center NVMe SSD for plotting, which can be found used for around $200. These will perform as well or better in plotting as a high-end consumer M.2 NVMe, and have 3-10x the endurance (which we need for plotting). Some models to look for on eBay, Craigslist, or other used venues are Intel P4610, P4510, Samsung PM983. With madMAx chia plotter we are targeting 4TB/day out of this budget build with plot times of 38 minutes.

There is a pretty large difference between the fastest SSD in the world and a cheap consumer NVMe for plot times. One year after the original budget build, the SSD picks are still relevant for data center SSD for plotting. Intel and Samsung are still the largest data center SSD vendors (although Intel NAND SSDs now are part of Solidigm), but you can find awesome picks from SK hynix, Kioxia, Micron, and others. Selecting the SSD will depend on the number of cores for the plotting system, and of course, the budget. We now have the madMAx Chia plotter integrated into the Chia application for CLI and GUI, and plotting is much, much easier than it was one year ago. In the budget build, we try to recommend an SSD for around $200. Today, this will buy you a mainstream 2TB consumer NVMe, or, what I recommend instead, a used 1.6-2TB enterprise or data center SSD. 

If you are looking for something to max out the performance of a 16 or 32 core machine, you will need something more than a $200 SSD! I’ve done other videos of super high-end plotters, like the beast, and tested the Chia optimized plotting SSDs, which I still recommend the Intel/Solidigm D7-P5510 just because it is such a good all-around drive, and of course, Optane SSDs if you can afford them.

Results

These are not the right capacity that I’ll be recommending this is just what I had laying around handy that I could test. Remember, we are targeting 1.6-2TB and $200 for a budget build, but we want to know the cheapest drive that gives us reasonable plot times.

driveCapacityTotal time k=32Time (min)TB/daypriceTBW (spec)TB of plots
P5800X16001963.4932.74.8$ 3,498.9929200020633
LX303020002059.3934.34.6$ 1,169.00540006868
PM98338402105.6335.14.5$ 375.0054661545
P551038402113.4135.24.4$60070001979
P4800X3752172.1336.24.3$ 500.006843.75484
P451020002245.1637.44.2$ 219.002610738
P461032002303.2538.44.1$ 500.00218503860
P460016002720.1245.33.5$ 280.0090001590
P315002911.0248.53.2$ 57.0030042
P37004002993.0249.93.1$ 149.0072001119

Test Setup

This test is pretty straightforward. Run one instance of madMAx chia plotter with 8 threads (-r 8) to each SSD, one at a time. Use the same mount point for -t , -2, -d (we aren’t saving these plots, so this is just quicker to test)

Compile madMAx. This is much easier now that it is integrated into Chia!
Format NVMe SSD
Make Filesystem
Mount with discard
Linux filesystem permissions
Open screen, run script that runs each drive and waits 60 seconds between

cd chia-blockchain
. ./activate
chia plotters install madmax
sudo cp ~/.chia/mainnet/plotters/madmax-plotter/build/chia_plot /usr/local/bin

sudo su
for i in 0 1 2 3 4 5; do nvme format /dev/nvme"$i"n1 -f; done
for i in 0 1 2 3 4 5; do mdir /mnt/ssd"$i"; done
for i in 0 1 2 3 4 5; do mkfs.xfs -m crc=0 /dev/nvme"$i"n1 -f; done
for i in 0 1 2 3 4 5; do mount -t xfs -o discard /dev/nvme"$i"n1 /mnt/ssd"$i"; done
chown jm:jm /mnt/ssd*
chmod 766 /mnt/ssd*

and my script does this for each SSD

chia_plot -k 32 -n 1 -r 8 -f <farmer key> -c <contract address> -t /mnt/ssd0/ -2 /mnt/ssd0/ -d /mnt/ssd0/ > ssd0.log

Desktop

Intel® Core™ i7-10700 CPU @ 4.60GHz
32GB DDR4 @ 3200 MT/s, Crucial (2×16)
MSI Z590-A PRO

Just for making my testing easier
PCIe Switch Card – U.2 / NVMe
U.2 Cable

SSDs

Intel SSD DC P3700, 400GB
Intel SSD DC P5800X, 1.6TB
Intel SSD D7-P5510, 3.84TB
Intel SSD DC, P4600
Phison Maker Pro / LX3030, 2TB
SK Hynix P31, 500GB
P4800X, 375GB
P4610, 3.2TB
P4510, 2TB
PM983, 3.84TB

Graphs

First 5 drives, in order

Intel SSD DC P3700, 400GB
Intel SSD DC P5800X, 1.6TB
Intel SSD D7-P5510, 3.84TB
Intel SSD DC, P4600
Phison Maker Pro / LX3030, 2TB
SK Hynix P31, 500GB
Busy System 
1.50GB,'s 
1 GB/s 
500 MB's 
-500 MB's 
-1 GB/s 
dm-O - Read bytes 
Busy User 
Busy Iowait — Busy IRQs 
CPU Basic 
09:10 
Busy Other — Idle 
Disk R,'W Data 
09:10 
09:15 
09:15 
— nvmeOn1 - Read bytes 
nvmelnl - Read bytes 
— sda - Read bytes 
dm-O - Written bytes 
nvmeOn1 - Written bytes 
min 
2.37 kB,'s 
444 B/S 
4.00 MB's 
712 MB's 
367 B/S 
4.00 MB's 
3.26 MB's 
1 u GB/s 
avg 
143 kB/s 
434 MB's 
7.06 B/S 
143 kB/s 
183 kB/s 
476 MB's 
current 
31.0 kB,'s 
367 B/S 
31.0 0/s 
5.28 kB,'s 
1.21 kB/s
100% 
75% 
25% 
0% 
Busy System 
1.50 GB/s 
1 GB/s 
500 MB's 
-500 MB's 
-1 GB,'s 
-1.5GB/s 
-2 GB,'s 
— Busy User — 
Busy Iowait Busy IRQS — 
CPU Basic 
Busy Other Idle 
Disk RIW Data 
09-40 
09-42 
09-46 
09-48 
09:50 
2.37 kB/s 
2.09 kB/s 
09:52 
09:54 
09:56 
nl - Read bytes 
Sda Read bytes 
dm-O - Written bytes 
nvmeOnI - Written bytes 
nvmelnl - Written bytes 
nvme2nI - Written bytes 
Sda - Written bytes 
1.53 GB/s 
BBI kB/s 
160 MB's 
119 MB's 
1.24 GB/s 
295 MB,'s 
MB,'s 
645 MB's 
77.8 kB,'s 
262 kB,'s 
863 kB/s 
707 MB's 
2.14 MB's 
261 kB,'s 
268 MB's 
220 kB/s 
3.19 kg's 
kB/s 
295 MB's 
2.82 kg's
CPU Basic 
100% 
10:00 
10:02 
Busy Systern 
— Busy User — 
1.50 
1 GB,'s 
500 MB's 
-500 MB's 
-1 GB/s 
-1.5 GB/s 
09:58 
10:00 
— dm-O- Read bytes 
nvmelnl - Read bytes 
— nvme2n1 - Read bytes 
— sda- Read bytes 
— dm-O - Written bytes 
— nvmelnl - Written bytes 
— nvme2n1 - Written bvtes 
10:04 
10:06 
10:08 
10:10 
— Busy Other 
10:12 
Idle 
10:12 
10:14 
10:16 
10:18 
101B 
10:20 
1 C20 
10:22 
1 C22 
10:24 
Busy Iowait Busy IRQS 
10:02 
10:04 
10:06 
10:08 
10:10 
Disk R/W Data 
10:16 
10:26 
1 C26 
2.37 kB/s 
1 C30 
10:32 10:34 
1.89 MB's 
268 MB's 
1.15 
1.89 MB's 
2.63 MB's 
101 MB's 
1.18 GB/s 
avg 
171 kB,'s 
7.04 MB,'s 
592 MB,'s 
171 kB/s 
128 kB,'s 
680 kB/s 
660 MB's 
current 
597 kB,'s 
7.06 MB's 
597 kB/s' 
23.2 kB,'s 
1.34 kB/s
100% 
10:35 
— Busy Systern 
1.50 GB/s 
1 GB,'s 
MB's 
OBIS 
-500 MB's 
10:40 
Busy User 
Busy Iowait 
10:40 
10:45 
— Busy IRQs 
10:45 
10:50 
Busy Other 
10:50 
Idle 
10:35 
CPU Basic 
10:55 
Disk R/W Data 
10:55 
11:00 
11:00 
11:05 
11:05 
• Read bytes 
nvme3nl • Read bytes 
• Read bytes 
Sda - Read bytes 
drn•O - Written bytes 
nvme2n1 Written bytes 
nvme3nl - Written bytes 
-1 GB/s 
nvme2n 1 
nvme4n 1 
11:10 
11:10 
2.46 kB/S 
11:15 
11:15 
680 MB's 
821 MB's 
21.3 
597 kB/s 
890 kB/s 
50.1 MB's 
1.08 GB/s 
14.0 MB's 
461 MB's 
435 kg's 
kg's 
60.4 kg's 
263 kB/s 
514 MB's 
11:20 
11:20 
21.3 MB/ 
29.1 
5.64 kg,' 
466 B/S
CPU Basic 
12:15 
Disk R/W Data 
11:55 
Busy System 
1 _50GB/s 
1 GB/s 
500 MB's 
-500 MB's 
-1 GB/s 
-1.5 GB,'s 
11:55 
12:00 
Busy User 
12•00 
12:05 
Busy Iowait Busy IRQs 
12:05 
12:10 
Busy Other 
12:10 
12:15 
12:20 
12:25 
12:30 
12:35 
nvmeanl - Read bytes 
nvme5nl - Read bytes 
Sda - Read bytes 
drn-O - Written bytes 
nvme4nl - Written bytes 
- nvme5nl - Written bytes 
Sda Written bytes 
kB/s 
1.37 kB/s 
386 MB's 
1.12 
889 kg's 
gag kg's 
MB's 
1.29 GB/s 
847 kg's 
12:40 
12:40 
7.56 MB's 
443 MB's 
86.9 kg's 
Bl _6 kg's 
306 kg's 
490 MB's 
Bl _2 kg's 
10.4 MB's 
142 kB/ 
2.82 kg,'s 
398 B/S 
2.46 kB/

The last 4 drives

P4800X, 375GB
P4610, 3.2TB
P4510, 2TB
PM983, 3.84TB

As you can see, the bandwidth is messed up on the P4800X that I had formatted to 4k sector size (fixed now!)

Here is the last 3 with the correct bandwidth

CPU Basic 
08:30 08:32 08:34 
Busy System — Busy User 
1.50GB,'s 
1 GB/s 
500MB,'s 
-500 MB/s 
08:36 08:38 
Busy Iowait 
— Busy IRQs 
08-42 08-44 
BusyOther — Idle 
08:50 
Disk R/W Data 
-1 GB/s 
-1.5GB,'s 
08:30 
08:32 
08:34 
08:36 
08:38 
08-40 
0842 
08-44 
08-46 
0848 
08:50 
08:52 
08:52 
08:54 
08:54 
08:56 
08:56 
08:58 
08:58 
09:00 
09:00 
O Bis 
O Bis 
68.3 B/s 
2.00 kB/s 
O Bis 
09:02 
09:02 
1-02 GB/s 
6_86 MB/s 
2.17 MB/s 
3.08 MB/s 
6_52 MB/s 
129 GB/s 
09:04 
09:04 
09:06 
09:06 
09:08 
09:08 
O Bis 
nvme2n1 - Read bytes 
nvme3n1 - Read bytes 
Sda - Read bytes 
dm-O - Written bytes 
nvmelnl - Written bytes 
nvme2n1 - Written bytes 
564 MB/s 
171 kB/s 
114kB/s 
167 kB/s 
41_5kB/s 
622 MB/s 
6.86 MB/s 
22.5 kB/s 
5_73 kB/ 
O Bis 
603 Bis
Disk R,'W Data 
1.50GB,'s 
1 GB/s 
500 MB's 
-500 MB's 
-1 GB/s 
-1.5GB/s 
09:08 
09:10 
09:12 
09:14 
09:16 
09:18 
09:20 
09:22 
09:24 
09:26 
09:28 
09:30 
09:32 
09:34 
nvme3nl - Read bytes 
Sda - Read bytes 
drn-O - Written bytes 
nvme2nl - Written bytes 
n vme3nl - Written bytes 
09:36 
209 kg's 
09:38 
1.09 GB/s 
781 kB/s 
836 kB/s 
158 MB's 
1.17 GB/s 
09:42 
592 MB's 
98.6 kB/s 
71.3 kg's 
1.07 MB's 
659 MB's 
09:44 
781 kB,'s 
17.3 kg,'s 
785 B/S

Similar Posts