SSD - HDD Comparison

[dnk 13]

Got a new SSD and figured why not do some benchmarking. I created a simple fairly script-less mission on Altis with 120 AI. To specifically test the SDD, I made the first part of the mission have the player setpos across the map to 8 different locations every 7 seconds, while I panned left and right after each move (the first pan right would load everything, the return pan left would look back over what was loaded). Then I had the player flown over Kavala for a couple minutes as I kept my view squarely on the hospital, with a couple quick pans out at specific locations. It should be easy to see on the graphs when the mission switched from the setpos script to the flyover.

Rig:

Intel i5 3350P 4c @3.3GHz (stock) 32-32-256-6M caches

Gigabyte B75M mATX board (current BIOS)

BFG GTS 250 1GB GDDR3 256-bit 738MHz/1100MHz core/memory (current drivers) (yes, this is a horrid GPU that's 12hours away from retirement)

2x2 GB OCZ 1066MHz 7-7-7-16 DC DDR3 (moderate latency, low bandwidth)

WD 7200RPM 1TB HDD SATA3

vs

Corsair Force LS 120GB SSD SATA3 (current firmware)

Expectations:

I expected minimal differences in FPS overall, with far fewer and lower stutters on the SSD (which would be the reason for whatever average FPS differences were seen). I expected the SSD would have far lower max frametimes than the HDD, and the SSD would do far better on the scripted setpos part of the mission with little to no pop-in.

Results:

Basically what was expected.

In each graph, the green is the SSD and the red is the old magnospinner.

The first graph is the frametimes, the inverse of FPS - higher is bad. 100 = 10FPS, 500 = 2FPS. The second graph is all the frames in order from fastest to slowest, again high = bad.

FIRST TEST

Settings:

LOW + 2000VD

Texture: low

Objects: low

Terrain: low

Shadow: ultra

Particles: low

Cloud: low

PIP: off

HDR: low

Dyn Lights: low

VD: 2000/2000/200

1600x900

Nothing but full AF for the last tab

In overall FPS neither is the clear winner; actually, the spinner looks a bit faster, but that's almost certainly variance at work. Already, though, the SSD is showing fewer stutters, especially in Kavala, but also far faster load-in for the setpos sudden moves. Still, overall they're basically the same.

SECOND TEST

Settings:

LOW + 3500VD

Texture: low

Objects: low

Terrain: low

Shadow: ultra

Particles: low

Cloud: low

PIP: off

HDR: low

Dyn Lights: low

VD: 3500/3500/200

1600x900

Nothing but full AF for the last tab

Now, here's a stark difference when it comes to the setpos sudden mass loads. The SSD does a much better job of streaming in all the data quickly, with far less stuttering. There's still quite visible pop-in, but the SSD clears it within 1-2 frames each time, while the WD struggles frame after frame to stream in all that crap. In Kavala during the flyover, though, both drives are basically the same, with the SSD getting hit hard by a sudden turn in the middle.

THIRD TEST

Settings:

High + 2000VD

Texture: high

Objects: high

Terrain: high

Shadow: ultra

Particles: low

Cloud: high

PIP: off

HDR: standard

Dyn Lights: low

VD: 2000/2000/200

1600x900

Nothing but full AF for the last tab

Here, there's clearly a bigger difference in average FPS in Kavala (and the WD just fails at the setpos portion), though this may be random luck. While the SSD still loses a few frames occasionally, the WD is clearly constantly struggling to keep up, and it should show up in lowered average FPS by a smidge as well.

FINAL TEST

Settings:

High + 3500VD

Texture: high

Objects: high

Terrain: high

Shadow: ultra

Particles: low

Cloud: high

PIP: off

HDR: standard

Dyn Lights: low

VD: 3500/3500/200

1600x900

Nothing but full AF for the last tab

Here, there's not a clear overall FPS winner, but the SSD crushes the WD in the fast loading part. I'm going to go a bit more indepth, since this is where we should see the biggest difference. The longest frame for the WD is 540ms (less than 2FPS), with 15 being under 5FPS, very noticeable stutters, if not breaks in the game. By comparison, the SSD has only 3 frames under 5FPS (all just under that), and the 15th longest frame is 120ms (which is the same as the 40th longest for the WD).

Looking solely at the Kavala flyover, ignoring the quickload tests, there's a clear favorite in the SSD:

Time under FPS for SSD, for WD, ratio

10FPS - 0.03% - 0.3% :: 10:1

15FPS - 0.6% - 1.9% :: 3:1

20FPS - 17% - 21% :: 1.25:1

33FPS - 91% - 95% :: ~1:1

AVG SSD = 42.9FPS

AVG WD = 44.7FPS

Conclusion

Well, there's random luck at play. It's doubtful such a long, AI-heavy (for me) scenario would yield perfect results, and with just 1 run-through we're likely to get a noisy result. But it's clear that if there's any overall FPS gain from an SSD, it's miniscule. The real difference is in reducing stutters and stutter amplitude and in quick loading (the latter isn't very important usually, though). Going from having serious stutters 2% of the time to just a quarter as much is going to feel better, and having massive game-stopping stutters all but disappear is pretty huge since those happen all too frequently on the WD. However, I doubt it's worth the pricetag of an SSD just for that since the $100+ could be better spent on faster RAM (see this thread), saving up for a better CPU or GPU, or you know, hookers and blow.

Still, if you were considering an SSD for other uses or just general computing, it probably will be worth it to put Arma on the drive.

Edited February 18, 2014 by DNK

[papy.rabbit.08 0]

Interesting, so if I understand well, the SSD seems worse for 2 FPS than the normal HDD? Seems quite strange! :S

[UltimateBawb 1]

Interesting, so if I understand well, the SSD seems worse for 2 FPS than the normal HDD? Seems quite strange! :S

Variance.

[dnk 13]

^ exactly. This is why I never fully trust FPS numbers from Arma benchmarks, especially when done just once. There's always like a +/-10%. It might actually be like 4% in the SSD's favor, hopefully further testing will get it more accurate.

I have noticed that with all the reduced stuttering, I can increase my texture quality from low to high without a significant performance impact (due to said stutters), which is a huge IQ improvement since I would get huge hangs and long frames before on the WD with anything other than low (very high still hangs too much, but with another 1GB in the GPU it probably wouldn't, but right now there's too much memory management going on with just the 1GB).

MP seems a lot smoother now, but that might just be my brain playing tricks, trying to justify the thing. Unfortunately with MP, there are so many more variables to control that it's damn-near impossible to do a benchmark.

[neodammerung 8]

You might want to try a RAM Disk (if you have enough RAM) , it's a stutter free experience for me (well when there's no AI)

[MavericK96 0]

In my experience with ArmA 2 and 3 with SSD vs HDD, there is virtually no difference in framerate but a HUGE difference in the amount of stutter/pop-in you get, especially while flying a jet at high speed.

[Valken 622]

Your system is fairly anemic by today's standards. 4 GB of ram with a slower GPU. I believe you have bottlenecks holding you back from maximizing SSD benefits.

16+ GB of ram with 8+ GB ramdisk shows significant improvements for A2 and A3. Stutter free and while not maxed out FPS, it is much smoother experience.

There is another thread that shows fast cpu (3.8 - 4 + GHZ Intel CPU) with high speed ram 1600 DDR3 yields more FPS, provided you have a strong GPU setup.

It all boils down to having a well suited system for ARMA games.

[dnk 13]

4GB is all I need for this game, and before people jump up and say, "no, it can utilize blahblahGB of RAM!" recognize that I watch my RAM usage after finishing, and I've yet to see total physical usage go past 3GB out of 3.95 available. The RAM is low bandwidth, but I'm fairly sure the VRAM streams direct from disk, so that shouldn't matter. The CPU is middle-of-the-road (but limited for Arma, yes) - for any other FPS it'd be more than adequate for 60FPS. The GPU has now been replaced by a 650 Ti Boost (yay!).

I wouldn't expect many players to build a system just around this game, when so many other games prioritize the GPU over the CPU and more RAM over a faster hard drive. Now, I'm totally undoubtedly CPU limited with just a mediocre 650 Ti Boost, like 60-70% core usage. Given the lack of graphical polish, there's just nothing for the GPU to do but dawdle and wait for a better graphical game to get played. This isn't really news, though, and I wholly expected it.

Anyway, I've started testing with the new GPU and much higher settings. Report coming soon. First thing I've noticed is that the GPU doesn't use more than like 1.2GB of RAM (it's 2GB total), meaning I have almost a full GB there too that's not being utilized, so instead the VRAM keeps going up and down as memory management does minor purges and loads. Ancient engine is ancient.

I'll also do another test once I've moved the Windows and pagefile over to the SSD to see if that does anything.

[Sniperwolf572 758]

Can you describe the process for getting and processing the data? I have the exact same HDD and the SSD is Samsung 840 Pro with a bit better rig I believe.

My friend has the almost exact same PC specs as I do. The difference being no SSD and a bit weaker hard drive and the performance difference is much greater (double digits) between the two rigs at the same settings, which we chalked up to the SSD providing a performance boost. I've never tried to run Arma 3 on my machine from the hard drive due to that.

I'd really like to check if we can replicate your results on my machine. If what you concluded is true, maybe there's some hardware issues on my friends system.

[dnk 13]

@Sniperwolf572 RE: Scenario and Processing

For the original tests, I ran the following scenario;

http://i482.photobucket.com/albums/rr181/davidk594/arma32014-02-2020-04-40-65_zps7dc9bfe1.png (365 kB)

For the previous tests, I would run through this mission one time before benchmarking at the lowest IQ settings, then ran each after that in sequence without performing any flushes or program restarts. For the newest tests below, I ran through one time to preload everything, then one time again to benchmark (this is why the newest comparison test has a smaller difference, since both the HDD and SDD had already loaded almost all the textures to VRAM on the first run-through, so there wasn't much to do on the second run-through).

The markers are the 7 different locations that the script setpos'd the player's chopper to every 7 seconds. The player enters the chopper as a passenger, then switches to 3rd person view, then hits zoom in. Then the chopper flies around Kavala. After each setpos, I would quickly pan 180-degrees and back. In Kavala, I kept the camera centered on the hospital, with a couple specifically timed occasions when I would do a quick 180-degree pan.

For benchmarking, I used FRAPS' benchmark with frametimes selected. After finishing, I opened it in a spreadsheet and took the delta time for each frame (FRAPS sets them as cumulative, which is not very useful for a graph). Then I put the HDD and SSD data together and made a chart from them. For the "ordered" charts, I just copied the values, pasted them as numbers (not formulas), and used data->sort on each, then graphed them.

Really, it must be another hardware issue, because the SSD offers no real FPS advantage I've seen.

NEW TESTS

Rig:

(this has changed!)

Intel i5 3350P 4c @3.3GHz (stock) 32-32-256-6M caches

Gigabyte B75M mATX board (current BIOS)

Gigabyte 650 Ti BOOST 2GB GDDR5 192-bit 900/1500MHz core/memory

2x2 GB OCZ 1066MHz 7-7-7-16 DC DDR3 (moderate latency, low bandwidth)

WD 7200RPM 1TB HDD SATA3

vs

Corsair Force LS 120GB SSD SATA3 (current firmware)

FIFTH TEST

This was a copy of the old scenario with the new GPU and new settings.

Settings:

HIGH + 2500VD

Texture: very high

Objects: high

Terrain: high

Shadow: ultra

Particles: high

Cloud: high

PIP: off

HDR: standard

Dyn Lights: ultra

VD: 2500/2500/200

1600x900

16xAF 8xAA "Low" HDAO ATOC all trees+grass, caustics on

Black is SSD, pink is HDD.

Here, I preloaded all the textures/objects by running through the mission once before testing. As you can see, this reduces the SSD's edge over the HDD considerably, especially with a new 2GB VRAM video card (which never goes past 1.2-1.4GB of usage but whatever, it's more) to store it all in. Still, there's considerably less hanging after each sudden setpos movement for the SSD.

In Kavala, the two are fairly similar, though the SSD has fewer lag spikes overall and what it does have are half to a quarter as bad as the HDD.

Average FPS of SSD vs HDD: 31.6 v 31.2

SIXTH TEST

Settings:

HIGH + 2500VD (same as above)

Texture: very high

Objects: high

Terrain: high

Shadow: ultra

Particles: high

Cloud: high

PIP: off

HDR: standard

Dyn Lights: ultra

VD: 2500/2500/200

1600x900

16xAF 8xAA "Low" HDAO ATOC all trees+grass, caustics on

Scenario: player takes a 2x time speed chopper ride from Pyrgos, around the island, and drops in to Kavala, where he runs around a set course, quickly panning and zooming in constantly. This is meant to approximate a plane ride and typical infantry combat movements in a major city. This was done right after the prior test, so most textures/models should have been preloaded already. If that was not the case, I would expect the HDD to perform significantly worse as it did on tests 1-4.

Black is SSD, pink is HDD

Black is HDD, Blue is SSD

First, other than a very few major lag spikes, neither drive has a clear "low end performance" problem here. Additionally, the HDD and SSD have very similar stutter performance while flying over the island - no clear winner there, though most textures/objects had been preloaded in both cases. However, the HDD clearly struggles a LOT more in terms of stutters in the last half, when the chopper gets to Kavala and the player gets out and runs around on foot. Here, the SSD clearly offers far smoother gameplay.

Total frames above 40ms for SSD and HDD: 0.3% vs 0.6%.

Worst frames for SSD (in ms): 104, 103, 83, 81, 74, 71, 63, 62, 61, 56...

Worst frames for HDD (in ms): 456, 179, 104, 103, 102, 96, 90, 89, 89...

The SSD's frequence of stutters is perhaps half as much as the HDD, and the severity also is roughly half. And this was in a test that was as biased in favor of the HDD as possible, with all textures being preloaded. We've seen how badly the HDD does when the bias is shifted in the SSD's favor, so it's only fair.

Avg FPS of SSD vs HDD: 47.7 v 47.6

Edited February 20, 2014 by DNK

[lao fei mao 21]

A SSD is absolutely needed for enhancing game experience. It's worth of your money just for the smooth fps, not mention the load time.

[dnk 13]

The FPS are definitely smoother IF you're using high quality textures/models. If not, it doesn't do much. It depends heavily on your IQ settings and actual system if it's a must-have or not. Someone with a GTS 250 like I had before... not a must-have. I'm still not convinced it was worth $100 (only talking about Arma and not general computing) compared to how much the rest of my system cost - the $130 I spent on the 650 Ti-B yielded vastly superior improvements than the SSD, and I'd imagine a new i7 4xxx would likewise yield better than 3x the improvements of the SSD (at 3x the price). Even more RAM probably would, though that's a bit more of a gray area.

[Sneakson 1]

A SSD is absolutely needed for enhancing game experience. It's worth of your money just for the smooth fps, not mention the load time.

Do you have any benchmark to back up that statement? How is the fps improved and how/when is the load time improved?

I'm thinking SSDs are primarily good for boot times... they are definitely over-hyped for other uses. Never heard of SSDs making much of a difference in games. I'm not quite sure what to have on my SSD actually. Currently I have the OS, all docs, programs and a few games which is basically everything I have except for downloads.

[dnk 13]

Frankly, if you're willing to spend $100+ to shave 5sec off your load times, you've got more money than you need, and you should give me some to thank me for my report :P

[lightspeed_aust 681]

Thanks for this testing, I was debating about a switch to SSD or new card. You just answered the question. :)

[arotished 10]

This game must be the only game that you really need to dig to the center of earth to find any FPS above 30.

I have:

4770K @ 4,5GHz

Samsong 840 Pro SSD in RAID0

Nvidia GTX780Ti in SLi.

32GB, 2400mhz CL9 RAM

aaaaaannnnndddddd the FPS is this game is still the worse FPS I have EVER played....I cant even turn my mouse before I get irritated because of the lag. I need to disable every settings in the AA section and go all the way down to 1000meters viewdistance along with Standard setting on many options in order to get close to 60FPS ingame (and still I get severe FPS drops in town) which say is a minimum to play a game without getting sick.

Bohemia, we are in 2014.....people actually have multi CPU/GPU hardware now....

[Sneakson 1]

Frankly, if you're willing to spend $100+ to shave 5sec off your load times, you've got more money than you need, and you should give me some to thank me for my report :P

Actually a small SSD, 60 or 120GB is highly recommendable. I've heard it cuts your boot time quite a lot and makes the OS quicker though I've never really seen any concrete benchmarks about it really.

Think about this though: if you can cut down 1 second on every load time on everything you do on your computer, every day, all year that eventually adds up not to mention a computer even being marginally quicker when it comes to things like opening documents makes for a big difference in user experience.

When I upgraded to Windows 8 and bought an SSD at the same time my boot time went from 32 to 16 seconds and when I upgraded to Windows 8.1 it increased to 17 seconds because there’s a new log-on screen that takes a moment to wait through… and that’s a massive difference. The days of going to have snacks while your computer is starting are over.

But I’ve never really compared an HDD to SSD directly… if I open a folder with lots of images on my SSD compared to my HDD the thumbnails seem to load nearly instantaneously, that’s about all the testing I’ve done.

[Janxy 10]

I have a 64GB SSD for the system ( I've moved the entire user dir, including hidden dirs to a larger HDD ) and my Arma/TOH installations on another SSD. The PC boots very quickly indeed and everything seems snappy as opposed to sluggish. As far as Arma is concerned, stuttering has more or less disappeared and although I didn't notice an overall increase in FPS, perceived performance is significantly higher.

[dnk 13]

I was referencing specifically Arma's boot times as "got more money than you need", didn't realize we'd expanded out into general computing. If you mean Windows and general computing, then yes it's worthwhile. For a long, long time hard drives have been the slowest component and increasing their speed would show the best increase in "responsivity", program loading, etc, assuming RAM and CPU were up to spec already. I'm looking forward to my full reinstall tomorrow quite eagerly. That was the primary reason for the purchase. Adding a lot of perceived smoothness to Arma is the cherry on top (and it does seem to be quite better subjectively, even if it's not so great an improvement objectively).

[frag85 10]

I had done something similar going back to A1/A2 with 7200RPM vs 10k RPM drives, later using an SSD for A2, and then updated with some A3 stuff.

Here is an A3 comparison I recently re-did for 1.14 because that patch fixed some performance issues I was having when running higher texture details.

The difference of a ramdisk is there, kind of.

Its not worth the cost if you ask me. Going back to 2007 when arma 1 came out, I spent $300 for a 150GB 10k RPM hard drive, on sale. That was cheap back then. I used that 10k drive through A1 and into A2 and didn't get an SSD untill 2012 when the 10k RPM drive started getting some errors after a PSU issue.

SSD is where its at. When you look at the fact that a ramdisk run on your average consumer level PC is 20-50x faster than your average SATA SSD of today, you aren't really gaining much of an improvement. The bottleneck is shifted elsewhere. Just look at some of the video comparisons I've made of HDD vs SSD vs RD. My ramdisk has a sequential read/write of about 10GB/s, and a random 4k read/write of >1GB/s. That blows any consumer level SSD or raid array out of the water.

If I didn't get my ram when it was cheap, I wouldn't be running 32GB. I paid $110 for 32 GB (2x $55 16GB kits on black friday in 2012 before production was cut back). I generally spend about $1000 on a new PC, and budget around $100-150 for ram. That same ram today would run you $370 ($185x2). Even 16GB would be pushing my usual budget. Its a shame that manufacturers cut back production to increase demand and prices, but that's business.

However, I do use some programs that will use as much ram as they can so having 32GB is a nice bump over 16GB. It also allows me to keep many programs open, and the way windows 7 caches data in the ram is pretty nice when I have gobs of free ram. Right now I'd say 8GB is the absolute bare minimum, 16 is recommended (and a 'gaming' PC would be able to get away with disabling the pagefile). 32GB+ is overkill unless you are running a lot of VM or rendering software.

Edited April 14, 2014 by frag85