Intel Statement on 13th and 14th Gen Core Instability: Faulty Microcode Causes Excessive Voltages, Fix Out Soon

[efikkan]

Even if it is just a re-calibrated microcode that's missing the processors have been running at an over-voltage for quite a while now, I wonder if material is still the same/good (semiconductor and connections).

Assuming Intel's assessment is correct, all affected models will have (some) shortened lifespan due to higher than normal wear, but by how much will depend not only on the CPU's load but also variations in silicon quality. This is probably why it's been taking so long to pinpoint the underlying "problem". But I assume the vast majority will not notice anything, but you wouldn't know until symptoms appear, whether your sample have 90% or 1% lifespan left (at least not without some diagnostics).

That's crazy cause I haven't had a single crash

You may have a CPU of higher silicon quality. Relax and enjoy it

-----

While there are many theories in circulation, and many have their suspicions about Intel's explanation, theirs is the most logical explanation to date. But time will tell. I've too have followed the coverage from news sites, Level1techs, GamersNexsus and others, and while I think thorough third-party investigation is more than welcome (perhaps even necessary), they should make sure to have the right (engineering) expertise to assess it properly (possibly even request hardware debugging tools and data about silicon quality?).
It has been suggested that about half of affected CPUs may be fixed with firmware, but let's dismiss that one outright; if there is (for any reason) abnormal wear, no firmware will reverse that. The fixed firmware will (presumably) avoid further abnormal wear.
Also various estimates of rates of completely defective CPUs of 20%, 50% or even ~99%, which I highly doubt.

 

[TomWeng]

They said update can prevent it, but cann't fix it....they don't even dare to admit the issues......

 

[aytokpatop]

The question is, will the CPU retain the performance characteristics once the voltage is dropped? or was it just enough to keep the CPUs in the top of the benchmarks for long enough to compete with current/next gen CPUs and the quietly drop the voltage and the performance when reviewers stop looking and retesting?

very unlikely

 

[dont whant to set it"']

Did anyone bring popcorn?
I've just completed a long drive and have some catching up to do. Back to page one.

 

[ToxicTaZ]

These articles are funny as their is nothing wrong with their chips as it's a bad May Bios upgrade from all manufacturers.

Then it says Intel update fix is August??? MSI gave that fix June 20th!

My 7 months old Motherboard.

https://ca.msi.com/Motherboard/MEG-Z790-ACE-MAX/support

I'm a proud owner of both 14900K & 14900KS systems with zero issues. The real problem is people have no clue what they're doing and just easyer to play the blame game.

13900K have been out for 18 months until the bad MAY Bios upgrade.

Very unfortunate what's happening in this falling apart would.

Cheers

 

[b1k3rdude]

"It has found that faulty processor microcode has been causing the processors to operate under excessive core voltages, leading to their structural degradation over time."
So if the excessive core voltage degraded the CPU, then HTF is a microcode update going to fix those degraded chips like..?

• Users should be able to get a damaged CPU exchanged but will still end with lower performing parts than what was reviewed.
• Bar any wierd OS or atypical setup, the microcode will be delivered and then pushed to the CPU at every boot. You don't need EUFI updates.

• Eh, then you didnt get what you paid for. At that point I would expect a voucher or cash in addition to the replacement CPU.
• The code has to be written to the CPU, or at least the bios. As having it in the OS is not a fix.

 

[Visible Noise]

Intel is also enforcing with MB vendors that they cannot overclock CPUs without the users permission. Reining in MB “enhancements” being on by default.

 

[azrael]

No company wants to recall if they can avoid it - no doubt there is a legal / numbers game that decides on certain things.
Last time Intel had to recall CPUs in public domain in same way was the Pentium FPU bug I mentioned earlier.... they handled that quite badly actually (initially they knew but didn't mention it until public knowledge forced them to acknowledge the errata, then you could only get a replacement if you could prove you were impacted by it* until eventually pressure forced them to offer replacement to all), although to be fair back in the day this was a rarer event and I don't think many companies were quite geared up for the fall out not providing worthwhile RMA warranty support would bring.

* Kinda ridiculous as you have no way of knowing if some soon to be released software might trigger the issue repeatedly after the warranty has lapsed...

Are you referring to the Pentium FDIV bug back in the 90s? I was actually affected by this. My first ever PC, sporting a blisteringly fast Pentium 90 MHz and the legendary Intel Plato motherboard.

 

[pressing on]

"It has found that faulty processor microcode has been causing the processors to operate under excessive core voltages, leading to their structural degradation over time."
So if the excessive core voltage degraded the CPU, then HTF is a microcode update going to fix those degraded chips like..?

Intel make no reference to structural degradation in the statement on their website. They only talk about instability, a fault in the algorithm and the microcode update fixing the algorithm. Intel microcode updates are delivered as BIOS updates by their motherboard partners. The 'leading to their structural degradation over time' is an editorial comment.

The instability referenced is described as "...reported system instability issues such as OS/Application errors, crashes, hangs, and BSOD on boards and systems with 13th and 14th generation Intel K SKU (unlocked) processors are the signs/symptoms of the affected units". The 'affected units' means "...all K SKUs of 13th and 14th Generation Core Processors (i5, i7 and i9)" source: Intel.

 

[Geofrancis]

Intel have been running very high voltages for boost clocks for a very long time, now they are up at 1.525v on some 14th gen chips, I think they just kept pushing and pushing until now something is broken, thats an insane number for 10nm. the last intel chips to use over 1.5v was a pentium 4.

I find it very suspicious that intel suddently had 4 different issues, I think they are trying to act like its 4 different issues with most of them fixable, while they release patches that limit the damage and hope users think its fixed until its past the warranty. I think they are going to limit boost clock times a lot, so its only there for a much lower time, that way they can say "peak performance" was not reduced.

 

[Dr. Dro]

Intel make no reference to structural degradation in the statement on their website. They only talk about instability, a fault in the algorithm and the microcode update fixing the algorithm. Intel microcode updates are delivered as BIOS updates by their motherboard partners. The 'leading to their structural degradation over time' is an editorial comment.

The instability referenced is described by Intel as "...reported system instability issues such as OS/Application errors, crashes, hangs, and BSOD on boards and systems with 13th and 14th generation Intel K SKU (unlocked) processors are the signs/symptoms of the affected units". The 'affected units' means "...all K SKUs of 13th and 14th Generation Core Processors (i5, i7 and i9)".

Well, the processors affected remain affected even if you install them on another motherboard, so that electrical/physical damage has occurred is pretty much a given. If your CPU has become crash happy due to this issue, an RMA seems to be the only solution. The chips will need to be discarded and replaced with new ones.

 

[agent_x007]

Sudden Northwood Death Syndrome (v2) : Now more global, because CPUs can overclock themselves to death by being on stock/recommended settings

Cause : Greed (as always), but how do we got here ?

Intel HQ some years ago :
Boss : How to increase profitability/margins ?
...
Director : Why bother limiting Vcore to "reasonable" levels ?
Push sucker to max and get as much dies validated as "OK" (less wasted dies = better profits), top die quality stuff may even do 6GHz !
Which means there is possibly for even higher prices, and very good marketing opportunity.
Engineer #1 : Sure, that's doable. There will be higher power usage (than necessary), which will mean more expensive cooling and VRMs will be required for end user... but what about longevity ?
Director : Any safeguards and mitigations we can do to mitigate those cons on our part ?
Engineer #2 : We could limit power consumption artificially in BIOS by adding long term and short term power limits (no more power hungry CPUs [s but not really /s]), and prevent very fast CPU degradation by limiting current going through it (decreasing chances of worst scenario occurring when very high voltage and max. temperature are being experienced by CPUs at the same time).
Director : Genius idea - I like it. Our technology process is best in industry, and we never failed to deliver new process technology on time.
So, with those safeguards in place we should be set. Let's do it !
Engineer #3 : But... is this... is this enough safeguards ?
Engineer #2 : Both power and temperatures are covered, so I guess... they all still need to be implemented by MB manufacturers ?
Director #2 : That's their job. If someone wants to run without them we can add those options (being OC friendly company we are and all).
Boss : Enough chit-chat. Do not forget to put everything important into datasheets, so that we are covered if any motherboard manufacturer does anything stupid.
All : Sure thing boss !

Disclaimer : This is fictional story, above does not represent how things work at Intel, and I'm not and never were Intel employee.

 

[Shrek]

• CrowdStrike
• Intel 13/14 Gen
• AMD Ryzen 7000

We are being overwhelmed by complexity; some say that AI is the solution.

I'm Only Human | Clone Wars (youtube.com)

 

[ToxicTaZ]

• CrowdStrike
• Intel 13/14 Gen
• AMD Ryzen 7000

We are being overwhelmed by complexity; some say that AI is the solution.

I'm Only Human | Clone Wars (youtube.com)

As both Zen5 and Arrow Lake both have Ai

I rather not pay for AI.... besides who is the basis for the programming for all the AI makes it kind of scary because there's no ground work in place. Everybody's doing their own things making AI nightmare.

NPU on all new chips by 2025

Cheers

 

[MikeSnow]

How does any go about the process of RMA-ing the CPU in this case? An invoice is needed? Just send in the CPU? Something else?

No invoice needed, you just need to give them the information about the CPU (serial number, etc.). Also, you need to experience the stability issues with the default Intel settings from the recent BIOSes. If you don't, they'll ask you to test that first. But things could get more complicated if it's not a boxed CPU, but a tray CPU, in which case they may want you to contact the system builder instead.

Considering it's still in warranty window idk why there'd be a lawsuit unless they deny a bunch of warranty claims.

Well, consider how much time and money some people and companies have lost dealing with this. And Intel has been aware for at least a year that the early 13th gen chips had oxidation issues, and has been hiding that until the information started leaking, and GN was about to confirm that through independent analysis. So, I don't know. If a CPU suddenly failed, and you had to replace it, maybe even a couple of times, I could agree that warranty replacement would be enough. But this slow degradation causing more and more issues is even worse.

This overvoltage issue is somewhat similar to Dieselgate. In that case, recalls have taken place, but also countless law suits, and VW was also fined by various governments. That would probably be too extreme in this case, but I still feel Intel needs to do a bit more than just warranty replacements. Maybe giving the option to get refunds instead of replacements. Maybe extending the warranty for all 13th and 14th gen CPUs to 5 or 6 years. Maybe a discount when buying another Intel CPU in the next 3 years.

 

[mkppo]

Yeah I mean... is anyone really shocked here. I feel like "Hey bro, if you run your CPU at 6.2Ghz and 1.53-1.56v you will degrade it quickly" has been common knowledge for a while.

Combine that with - "Hey let's build a server with these" and you get ... "WHY ARE ALL THESE SERVERS CRASHING AFTER A FEW MONTHS!?!?"

The servers farms were 13900K/14900K's sold directly by intel to them and installed on server motherboards from various vendors. Yes they were designed to run 24/7 for years, not degrade in months. Intel directly contributed to them losing tons of money and not all of them can afford that.

You make it sound like they built server farms with CPU's that weren't designed for the task. Well, Intel should've warned them earlier that these will only last for a few months, which is absolutely positively unheard of when it comes to CPU's.

The CPU's were requesting those voltages from the motherboard, and that's what Intel plans to solve in that microcode update with performance regressions a likely scenario. Let's see.

Yes it is shocking. CPU's are supposed to be designed to run at full load for a number of years minimum and not degrade, end of. Making a CPU that only lasts a few months at their rated speed is at best false advertising and has resulted in a boatload of money lost by all these game devs and server farms, most of which is unrecoverable. Couple that with reports of denied RMA requests due to these companies putting out statements mentioning that their Intel CPU's are faulty only makes it worse for them. Most of these servers are switching over to their competitor because they actually run at their advertised speeds for, well, a long time. To top that off, one server farm and one game dev blatantly said they're faster anyway so it's a win win for them.

Having seen the voltages a 14900KS requests from the motherboard, I really want to see one at full load for a few months. I can bet that without a microcode update, >90% of those will degrade over a couple of years. With the update, it can't really run at the speeds it was rated at with full stability. Let's see what the supposed August update brings, 8 months after the initial reports started to come and almost 2 years after the launch of the CPU. Lame.

All I know is, I ain't touching any of those 13 and 14th gen chips on the second hand market with a ten foot pole even if they depreciate heavily. Which sucks, because over the (many) years i've had pretty good luck buying used CPU's as they get cheap a generation prior.

As both Zen5 and Arrow Lake both have Ai

I rather not pay for AI.... besides who is the basis for the programming for all the AI makes it kind of scary because there's no ground work in place. Everybody's doing their own things making AI nightmare.

NPU on all new chips by 2025

Cheers

Zen 5 desktop chips don't have the NPU, it's only the laptop chips.

 

[Minus Infinity]

Pat: I need more voltage dammit!

Engineer: Capain' I can canna give you anymore voltage without causing a warp core breach

Pat: I don't care man, we've got the AMD fleet bearing down on us and they about to launch Zen 5 quantum torpedos, we need more voltage man.

 

[theouto]

And this doesn't even get into the oxidation issues.

Happy faces all around.

 

[Crackong]

Not a Defect: Intel Blames 13th, 14th Gen CPU Crashes on Software Bug (msn.com)

That software is called "Microcode"

Also by Intel

 

[mkppo]

GN just posted results from the analysis lab and denied RMA's. Edit: They're still awaiting results from the lab, but changes nothing.

So uh..sell dodgy chips to server farms, deny RMA even though they knew about the oxidization issue at the time then release a half assed statement two years after the chips launch that the chips have an issue, sorry wait multiple issues.

So if intel apparently 'fixed' this oxidization issue which apparently plagued early 13th gen batches, obviously they knew about it. And then they did....nothing? For years? They release a statement saying that right when third party analysts start mentioning it? Also, they conveniently fail to mention what batches were affected by the issue. Still trying to figure that out after years eh?

Something smells funny.

edit2: Just putting this out there as well, he rambles for a whole hour but the first two minutes are pretty informative lol

 

[64K]

GN just posted results from the analysis lab and denied RMA's. Edit: They're still awaiting results from the lab, but changes nothing.

So uh..sell dodgy chips to server farms, deny RMA even though they knew about the oxidization issue at the time then release a half assed statement two years after the chips launch that the chips have an issue, sorry wait multiple issues.

So if intel apparently 'fixed' this oxidization issue which apparently plagued early 13th gen batches, obviously they knew about it. And then they did....nothing? For years? They release a statement saying that right when third party analysts start mentioning it? Also, they conveniently fail to mention what batches were affected by the issue. Still trying to figure that out after years eh?

Something smells funny.

The thing that really stands out for me is Intel trying to pass the blame to mobo makers in the beginning. Contemptible.

 

[Hossein Almet]

I've been up against the so called performance benchmarks and OC CPU nonsense all these time. That why I purchased the 5600 for my gaming PC.

 

[iameatingjam]

Well geeze when heard

Based on extensive analysis of Intel Core 13th/14th Gen desktop processors returned to us due to instability issues, we have determined that elevated operating voltage is causing instability issues in some 13th/14th Gen desktop processors. Our analysis of returned processors confirms that the elevated operating voltage is stemming from a microcode algorithm resulting in incorrect voltage requests to the processor.

Intel is delivering a microcode patch which addresses the root cause of exposure to elevated voltages. We are continuing validation to ensure that scenarios of instability reported to Intel regarding its Core 13th/14th Gen desktop processors are addressed. Intel is currently targeting mid-August for patch release to partners following full validation.

Intel is committed to making this right with our customers, and we continue asking any customers currently experiencing instability issues on their Intel Core 13th/14th Gen desktop processors reach out to Intel Customer Support for further assistance.

I was pretty excited.

But now apparently when oxidation enters the discussion, its just a factor again. Seriously? Trust -1. And it was already getting pretty low.

 

[Caring1]

Before GN's video, there was this:

It seems even setting Intel's default settings in the Bios isn't good enough.

 

[phanbuey]

The servers farms were 13900K/14900K's sold directly by intel to them and installed on server motherboards from various vendors. Yes they were designed to run 24/7 for years, not degrade in months. Intel directly contributed to them losing tons of money and not all of them can afford that.

You make it sound like they built server farms with CPU's that weren't designed for the task. Well, Intel should've warned them earlier that these will only last for a few months, which is absolutely positively unheard of when it comes to CPU's.

The CPU's were requesting those voltages from the motherboard, and that's what Intel plans to solve in that microcode update with performance regressions a likely scenario. Let's see.

Yes it is shocking. CPU's are supposed to be designed to run at full load for a number of years minimum and not degrade, end of. Making a CPU that only lasts a few months at their rated speed is at best false advertising and has resulted in a boatload of money lost by all these game devs and server farms, most of which is unrecoverable. Couple that with reports of denied RMA requests due to these companies putting out statements mentioning that their Intel CPU's are faulty only makes it worse for them. Most of these servers are switching over to their competitor because they actually run at their advertised speeds for, well, a long time. To top that off, one server farm and one game dev blatantly said they're faster anyway so it's a win win for them.

Having seen the voltages a 14900KS requests from the motherboard, I really want to see one at full load for a few months. I can bet that without a microcode update, >90% of those will degrade over a couple of years. With the update, it can't really run at the speeds it was rated at with full stability. Let's see what the supposed August update brings, 8 months after the initial reports started to come and almost 2 years after the launch of the CPU. Lame.

All I know is, I ain't touching any of those 13 and 14th gen chips on the second hand market with a ten foot pole even if they depreciate heavily. Which sucks, because over the (many) years i've had pretty good luck buying used CPU's as they get cheap a generation prior.


Zen 5 desktop chips don't have the NPU, it's only the laptop chips.

They put desktop CPUs in older blades that weren't designed for those chips -- it was a custom hack, with intel's datacenter team but a custom hack nonetheless. You're making it sound like this is a common thing -- it's not. We build our racks and servers as well, and "Blades of 14900Ks with 128gbs of ECC ram" are not a normal setup. Server chips are usually xeons or pentiums and they run at 2.1-3.2 ghz max - and they sit there for 20 years doing it.

Of course these are not designed to burn out in a few months - but if you have blades burning out your 14900Ks ... why... put... more... 14900Ks... in those blades? Not saying that the chip is good, but when you're putting a yolked 14900K into a blade to save money this is kind of exactly the downside.