PostgreSQL expertise from specialists with a source code level understanding of RDBMS
Intel SSDs: Lifetime and the 320 vs. 710 Series
This week I’ve been digging deep into PostgreSQL storage hardware again. Since I’m giving a conference talk on database storage in Austin and in the DC area next week, it seems like a good time for me to actually know the material. One of the most common questions here is “what’s the cheapest SSD I can put my database on?”, with the implied hope “…without losing it all the time“. Last year the first inexpensive answer to that appeared on the market, and I suggested people take a look at Intel’s 320 series drives. With 217 days of runtime on my first 320 drive here, and Intel’s 3rd generation storage line filled out with the more enterprise oriented 710 Series now, it’s worth reviewing how that turned out.
It wasn’t long after the 320 series drives were introduced that people started reporting a firmware problem with the drive, where it did things like report a capacity of 8MB after a restart along with “BAD_CTX 0000013x” errors. A firmware update to fix that was released. There’s still some claims of continued problems floating around. You have to expect some percentage of any product are going to be bad, and the later production of this drive (after the big bug was fixed) don’t seem above the usual risk level in hard drives to me. With the warranty here extended to 5 years (unless you’re using it at ‘enterprise usage levels’), I think that Intel would be getting killed if the reliability on these was as bad as some people claim.
The reason behind the usage level caveat is the main thing worth talking about here. The long version of the warranty suggests “The media wear-out indicator reports a normalized value of 100 (when the SSD is brand new out of the factory) and declines to a minimum value of 1. When the value reads 1, this indicates that the SSD is reaching the wear-out limit”. Here’s what my first 320 looks like so far:
[root@toy ~]# smartctl -a /dev/sdc
=== START OF INFORMATION SECTION ===
Device Model: INTEL SSDSA2CW120G3
...
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
3 Spin_Up_Time 0x0020 100 100 000 Old_age Offline - 0
4 Start_Stop_Count 0x0030 100 100 000 Old_age Offline - 0
5 Reallocated_Sector_Ct 0x0032 100 100 000 Old_age Always - 0
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 5225
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 58
170 Unknown_Attribute 0x0033 100 100 010 Pre-fail Always - 0
171 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 0
172 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 0
184 End-to-End_Error 0x0033 100 100 090 Pre-fail Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 34
225 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 201389
226 Load-in_Time 0x0032 100 100 000 Old_age Always - 2687040
227 Torq-amp_Count 0x0032 100 100 000 Old_age Always - 0
228 Power-off_Retract_Count 0x0032 100 100 000 Old_age Always - 314526
232 Available_Reservd_Space 0x0033 100 100 010 Pre-fail Always - 0
233 Media_Wearout_Indicator 0x0032 099 099 000 Old_age Always - 0
241 Total_LBAs_Written 0x0032 100 100 000 Old_age Always - 201389
242 Total_LBAs_Read 0x0032 100 100 000 Old_age Always - 133021
Not all of these attributes are labeled correctly, and some are “Unknown”; all the gory details are in the product specifications. You can see the Media Wearout above; here’s the raw values for other interesting ones, formatted so they’re more blog-friendly:
ID# ATTRIBUTE_NAME RAW_VALUE
9 Power_On_Hours 5225
12 Power_Cycle_Count 58
192 Power-Off_Retract_Count 34
225 Load_Cycle_Count 201389
241 Total_LBAs_Written 201389
242 Total_LBAs_Read 133021
192 (hex C0) is “Power-Off Retract Count”. That’s how many unsafe shutdowns the drive has been through, which are the situations where the battery backed cache in the drive has been triggered. With 34 of them here, you can see I’ve tried to get this drive to die that way.
The first interesting wear figure is 225 (hex code E1) which Intel’s documentation describes as “Host Writes”. The units for that are 32MB. If you look carefully, you’ll see that’s the same value given in 241 “Total LBAs Written”. That suggests the LBA unit for the drive is also 32MB, which I double-checked last year. At 32MB each, my write value of 201389 means I’ve written 6.15TB to this drive.
Now, computing the true lifetime of an SSD depends on a couple of magic values, like the “write amplification” of your workload. That suggests how often your workload forces small bits of data out to flash, using up some of the NAND cell lifetime faster than it might otherwise last. These numbers are really hard to estimate. The most realistic way is figure this out is to run a workload simulation after resetting the drive’s internal counters, then see just how much you burned through. The process is walked through with an example at “Measuring How Long Your Intel SSD Will Last”, and it’s not too hard to translate that example (which uses Intel’s SSD Toolbox software) into a set of of smartctl commands if you’re on Linux–the article even uses smartctl for the counter reset part.
The official documentation is this is Intel’s Enterprise Server addendum, and here we finally find some hard numbers about the expected life of these drives. My 120GB drive is said to have a “write endurance” of 15TB. A pessimistic look at my sample drive here would check total writes and say that, having written over 6TB, I’ve gone through 40% of the drive lifetime. But write endurance doesn’t work that way; the firmware is constantly doing tricks to extend the life of the drive. Intel’s official number they sometimes tie the warranty to, the Media Wearout, is showing 99% left! If that’s true–I’ve only used 1% of the drive’s lifespan–then I might manage 600TB of writes before this one really dies on me.
So what’s the story with the true Enterprise lifetime 710 Series drives? Those almost the same drives as the 320 series ones, with three significant changes. First, they’re said to use higher quality flash, probably with the same sort of “put the best tested chips first in the expensive models” approach Intel is said to use on their CPU production–what’s sometimes called binning. Second, the drives are overprovisioned with a lot more unused flash compared to the 320 series models, and unused flash really helps extend longevity. Finally, they don’t claim the capabilities to be quite as good. Random write IOPS numbers on the 710 series drives are lower; my 120GB 320 series drive is specified at 14K write IOPS, while the 100GB 710 series only aims for 2700. The drive doesn’t claim to support lots of tiny writes and still last for years, which means it’s aimed at a different set of write amplification expectations. Similarly, the 710 series drives don’t refresh the stored cells in the same way. The downside there is that 710 models are only specified to retain their data for 3 months. That’s probably fine for data center use, but that wouldn’t be very acceptable to the more consumer oriented market the 320 series is sold to.
The end result of that, and how the 710 compares to the 320 series drives, is nicely summarized in the “Write Endurance” table in the Tom’s Hardware Review. Instead of the 15TB endurance number my 320 drive specifies, the similar 100GB 710 series model aims for 500TB. That’s just over 30X as long. In the real world, there may not be that big of a difference, as shown by the projected 600TB figure I’m seeing out of my 320 drive so far. But Intel’s aiming at conservative engineering lifetimes on the specification sheets, and by that measure the storage cells 710 will last longer; the 320 models only may last longer. And an expected lifetime 30X as long is something some people are surely willing to pay the 710′s price premium for.
This Post Has 1 Comment
Leave A Reply
© Copyright2013 2ndQuadrant
design by WPExplorer
Thanks for nice and helpful post.
I have been reading a lot about SSD’s recently for possisble usage on our MySQL and still it seem putting SATA SSD into production server is a magical decision, as absence of capacitors on most drives seems a huge problem.