This article doesn't make any sense. The author is doing generic phone benchmarks. But the Spectre fix is a webkit fix only, not an OS-wide fix. If they're seeing performance regressions across the whole OS because of fixing Spectre, something's seriously wrong with their benchmark methodology.
Edit: The author upgraded from iOS 11.1.2 to iOS 11.2.2. This isn't just a test of the Spectre fix. The most likely explanation here is upgrading to iOS 11.2 caused their iPhone 6 to start throttling due to battery wear (11.2 added throttling to iPhone 7, and it's plausible that it changed the conditions for throttling on iPhone 6). It's also possible that this is instead caused by the Meltdown patch, but these numbers are still way out of line with what was expected for Meltdown on iOS, whereas they're very much in line with what we've been seeing with battery throttling.
The article should be flagged off the front page or edited by the mods to show that it's questionable.
This is a single person's test results, across versions that are known to have touched the battery/CPU slowdown logic, showing insane performance degradation across almost every aspect of the phone... it's super questionable. And right now it's #1 on HN despite all of that, just spreading misinformation.
I did this same test with an iPhone X and iPad Pro 10.5" as well as a several year old Mini 2 and didn't see a difference. In fact, I saw what you did, my scores got better in most categories.
Yeah there is. Get your battery replaced. Apple will replace your battery for $29 (as long as there's sufficient wear, but if you're seeing throttling, you're well past the cutoff).
> Apple is reducing the price of an out-of-warranty iPhone battery replacement by $50 — from $79 to $29 — for anyone with an iPhone 6 or later whose battery needs to be replaced, available worldwide through December 2018.
The "whose battery needs to be replaced" sure implies they're still going to actually test it.
I have an iPhone 6, and didn’t notice any further performance issues after upgrading to 11.2.2. I did have pretty serious issues (which have mostly remained) since upgrading to iOS 11. So I don’t think the latest version did anything more, at least on my phone.
>> a significant decrease in performance on the iPhone 6 up to 50%
Something's up. I updated my 6S yesterday and have noticed zero in performance changes or battery loss. Still plenty of bugs though, I was reading an email and the 'flag/file/trash/reply/new' bar totally went away. At least the touchscreen hasn't gone unresponsive, causing me to have to hit sleep/wake to toggle it back on. Maybe they finally fixed that.
On an iPhone 6 - I actually gained performance from 11.2 to 11.2.2 (I neglected to run Geekbench on 11.2.1). Previously my best single-core score on iPhone 6 was 1536 (and only when battery was above 93% would I get in the 1500s), now today I got 1540 single core on only 66% battery - previously anything below about 70% battery would give me a score down in the 1200s.
There really seems to be some performance issues with either the network stack or the threading/background processing in iOS11 on older devices.
I've had periodic slowdowns and unresponsiveness with my iPhone 6 since upgrading to 11, but a few weeks ago I was downloading a bunch of music for a trip and my phone was virtually unusable until the downloads were done. Since then I've noticed unresponsiveness around the same time I see notifications for podcast updates or after exiting an app that was downloading tons of content (e.g. The Weather Channel app).
Ok, I'm sceptical about the results. The reason is that there doesn't seem to be a massive difference between the tests. Since this fix is about speculative exec, why would it affect crypto code which is very register based and branchless as much as sqlite which is full of branches and memory/storage based? Why would it affect AES which is hardware accelerated as much as integer processing which is not?
I'm not saying this is impossible - maybe there's something that I'm missing. But it just doesn't add up at the moment. I'd love a more detailed / repeatable test.
I can't speak about these benchmarks (the site is down). But Meltdown is not about speculative execution, it's about out-of-order execution. The fixes for Meltdown involve getting kernel data out of user virtual memory, and flushing the TLB if the processor does not have the pcid instruction. So context switches - including system calls - will tend to be more expensive, and if the processor does not implement pcid, way more expensive.
See https://meltdownattack.com/. I haven't yet read the Spectre paper, but I can say that the Meltdown paper is very readable and walks readers through everything. I anticipate the Spectre paper is as well.
Is it possible that this benchmark has coincided with a battery that's just crossing the threshold of battery-throttling induced slowdown?
The low scores in the article are in a similar ballpark to my (slow) iphone 6 on ios 10.3.3 with a 3yr old battery - i.e. about half what they should be with a fresh battery.
The A8 doesn't have a dedicated crypto engine like Intel chips do (uncore I think they call it). The A8 does have a couple of AES specific instructions so each core can speed up the work, but they were executed speculatively and presumably are no longer done so.
I've went through Denial, Anger, Bargaining, and now in an Acceptance stage after taking up to 31% performance hit on some of services managed by my team. Worst case has been Elasticsearch so far with our load pattern, taking that 31% hit.
What about using dedicated hardware and not running the patches? At attacker has to run malicious code on your server to exploit the vulns anyhow, right?
I usually don't trust people. More than one have we found things improperly secured and/or exposed to the outer internet, or otherwise at risk.
We also run black-box software from third party vendors that provide compiled JAR files which is directly related to this fuss. Last thing I'd like to have is another attack vector at home.
Sure, but I think that would (potentially, with the proper exploit) turn any non-root-level code-execution-able breach of your machine of a machine into a root level one, no? Maybe that's okay?
We're actually using those new 24-core Epyc hosts for our data science pipeline (HDP/Spark cluster) and they work much better than our previous generation Xeon-based builds.
Unfortunately, with those things it takes plenty of time to re-evaluate hardware + in our case we're mostly in the cloud where Skylakes on GCP is the most we can get.
I wonder what the difference is between the 6 used in the benchmark in the article and the iPhone X that would cause these different results. They both support speculative execution.
Either way, encouraging results to a fellow iPhone X user.
I'm curious what the delta is between perf hit on an iPhone 6 and an iPhone 8. I'm guessing that, given the deadline apple had to get this out, most of the attention & optimization went to current devices versus near-EOL devices like iPhone 6.
It looks like the Exynos 9 Octa 8895's 4 main (big) cores are a custom Samsung architecture which is almost certainly out-of-order, and is probably affected. The "little" cores are Cortex-A53, which is in-order and not affected.
Assume that [EDIT: high-end - dogma1138 is right] smartphone-class processors and above are affected. Raspberry Pi and microcontroller-class ARMs are fine, though (but much slower!)
This might sound dumb but.. with these Meltdown + Spectre bugs would an attacker be able to penetrate to gain full access only if a user downloads a native app or would it be possible through the browser/js?
Spectre can give you read-access to the browser process via Javascript, yes, as they described in the original article: https://spectreattack.com/spectre.pdf (section 4.3 on page 6). No write access, or access to the OS internals.
Haven't seen much coverage on how this affects game consoles like the Xbox and Playstation (which use AMD CPUs). Does anyone know if they've talked about patching it and how this would affect game performance (which is a pretty big deal for gamers who expect a consistent experience)?
I will repeat these benchmarks on my iPhone 6 when I get home tonight...but in the meantime, the benchmark numbers in this article look almost exactly like the ones I had as a result of battery throttling. I bet the actual patch-related performance hit is pretty minor.
I’ve more than a little skeptical of these results, I too am very disappointed in major CPU vendors (especially their PR and management teams), but these results seem more than a little smelly.
I am what I’d consider a heavy iOS user on multiple devices and I don’t believe I’ve honestly noticed any difference at all. If these results were correct, I believe I’d notice at least a 10-15% decrease in performance - but no. What I haven’t looked at is battery life so I cannot comment on that.
All major cloud providers are buying renewables for their operations. Yay?
At least the additional renewable generation capacity built out will exist for decades (while processors should become more efficient as these security issues are addressed and new hardware rolls out).
>All major cloud providers are buying renewables for their operations. Yay?
Not yay. This is an overnight massive bump so they are going to consume from the same existing pool of renewables, forcing utilities to use more non renewables to make up for the demand difference.
Also, even when they do build out the renewable generation to make up the difference, it's still environmentally a problem due to the manufacturing of the renewable generation (solar panels, windmills, transformers, etc).
We have no surplus of renewable energy to absorb this kinda stuff so more energy usage is always bad at this point. It's just slightly less bad when they commit to paying to fund renewable.
I've read a few articles saying that there is so much Solar adoption in the U.S. that there is a surplus at some times (summer days I believe). It use to be that there was too much use from home AC systems so power was expensive during the day. It seems to be switching to night now as so much is generated during the day (with just 3% adoption of rooftop Solar).
Here's an example of one article on the matter, which I just skimmed a little.
I'm guessing that CPU's use a whole lot less electricity than things like AC systems, so they probably don't affect total electricity used by that much. I was able to surmise that a 3 ton central air conditioner uses around 3500 watts per hour while an Intel I7 (whole system) uses around 150 watts per hour.
My theory, then, is that this won't affect total power use in the U.S. all that much.
I'm guessing more than 1/2 of all AWS instances are idle. Probably 10% are systems nobody knows anything about, whoever started them left the company..
Add the 5% that are nothing but public garbage bins for personal information collected by various companies and intelligence agencies, just waiting to be discovered!
My iPhone X has been on 11.2.5 since the day it came out, and I haven't perceived any slowdowns.
Have not done any benchmarking, and I'm not saying performance is the same as before, but anecdotally I haven't seen a difference. Just one guy's opinion.
I just updated my 6+ and ran Geekbench before and after. No major difference in scores and the single and multi-core benchmarks are consistent with the comparison numbers for other 6+ phones. Battery is in good health.
this leads me to a couple questions, hopefully you smart folk can answer:
1. I thought Spectre was "Intel-only", and Meltdown was the general case, which is less severe but effectively nearly everywhere? If so, how is an iPhone susceptible to Spectre?
2. Beyond that, I thought meltdown/spectre was an x86 problem. So why all this trouble on phones, with ARM?
3. I've read the first, simplest variant of meltdown, and it is so beautifully simple. Is this "speculative execution + cache timing" thing an entirely novel exploit, or have we seen incarnations of this before?
1: Spectre is the general case, Meltdown is the Intel-only evil cousin.
2: Spectre applies to all modern processors with speculative execution, which includes smartphone-class ARM processors. Raspberry Pi's and microprocessors ("toasters") are not affected.
3: There's some prior work - Spectre didn't fall from thin air - but using speculative execution as the basis of an exploit makes Spectre the first bugs in, I believe, a new class. (Meltdown, on the other hand, is just a silly mistake that shouldn't be repeated.)
1. is reversed: meltdown is intel-only. Spectre is a more generic speculation attack.
2. Apple's custom ARM chips have extensive speculative execution, and clearly their fixes for it are extremely expensive.
3. It's novel, which is why security researchers are so excited! Branching based side-channel attacks were previously only known to be able to track execution (so branch-based AES encryption was vulnerable to having its keys stolen)-- but reading arbitrary data is much stronger.
Except Cloudflare is saying this: "However, because the site uses Cloudflare's Always Online™ technology you can continue to surf a snapshot of the site"
It shouldn't be telling you that the host is failing.
Is that how is's supposed to work? Because any time this happened in the past to a site where it is overcrowded and the Cloudflare banner is showing, the 502 page is showing as well, so I thought it's part of the Cloudflare error page?
I've never known Cloudflare to do things well. It's the last CDN I'd want to use.
Edit: For example, the 502 page is showing, but supposedly, you are supposed to see the site based on this: "However, because the site uses Cloudflare's Always Online™ technology you can continue to surf a snapshot of the site."
Yeah, this is the first time I've ever seen the "you can continue to surf a snapshot of the site" message along with the 502 though. Or maybe I've just never looked that closely.
Is this a "bug" we can recover from (in future hardware or software) or are we going to collectively take a lifetime -18 hit to our Moore's Law progress and lose the gains from this technique forever?
At least for Meltdown there are CPU microcode updates out there that should mitigate it, which means the OS level mitigations can be rolled back. And I haven't measured it yet, but I suspect the microcode mitigates are faster.
Regarding Spectre, I think it's too early to tell.
I'd like a new computer architecture, where communication between different parts of software systems was more explicitly treated as communication. Instead of an exception to pass control to the kernel, how about sending data? Instead of multiple CPU communication through memory (and therefore the cache) how about sending data?
Wow. After ios 10.2.1 halved performance on my iphone 6 due to the battery issues, I next avoided 11.0 as I accepted responsiveness would be slower on the older iphone6 model.
Now if we want these security fixes that's another -40%... Yikes!
However, it looks like ios 11.2.2 and a new battery is still slightly faster than ios 10.3.3 and an old battery! (geekbench single/multi core score from article of 924/1616 vs my 844/1379)
I don't understand this logic at all. The fixes are being made for a reason... avoiding the update just means you are getting more performance for a less secure system. Are you betting on a law of averages to break in your favor? Hoping you're not a target of hackers? what am I missing with this strategy?
The OP doesn't seem to suggest that they won't update, just that there is a considerable price to be paid in doing so.
> Are you betting on a law of averages to break in your favor? Hoping you're not a target of hackers?
Aside from anything else, I wouldn't be at all surprised if some people do take this gamble. They shouldn't, but if they're informed about it then it's in their own hands.
All of these issues require local execution. It’s perfectly fine for many single user, performance critical use cases to make this type of security trade off.
It wouldn't be the first time that a buffer overflow somewhere led to arbitrary code execution. What we're patching is a mechanism that allows escalating from unprivileged execution to reading kernel memory, and a mechanism that defeats KASLR
Just that I could be looking at a > 75% performance reduction on the same phone compared to when it was new due to the combo of battery degradation, security fixes and ios9 to ios11 general slow down.
I think that if they've disabled some cpu core features that might reduce the maximum power loading anyway and make the double whammy impact of security and battery related slowdown unnecessary.
Edit: The author upgraded from iOS 11.1.2 to iOS 11.2.2. This isn't just a test of the Spectre fix. The most likely explanation here is upgrading to iOS 11.2 caused their iPhone 6 to start throttling due to battery wear (11.2 added throttling to iPhone 7, and it's plausible that it changed the conditions for throttling on iPhone 6). It's also possible that this is instead caused by the Meltdown patch, but these numbers are still way out of line with what was expected for Meltdown on iOS, whereas they're very much in line with what we've been seeing with battery throttling.