MATLAB MKL Codepath Tweak Boosts AMD Ryzen MKL Performance Significantly

[ShredBird]

As a regular user of MATLAB, I have to say this really falls on Mathworks from my perspective. MATLAB runs on the Java virtual machine so that it can be hardware agnostic and high level. As a company, you want MATLAB to perform its best on any hardware, so whether that's using a proprietary library or a free one, they should strive to optimize for many architectures. Intel is totally in the right to have MKL fallback to SSE if a compatible processor is not detected, it's their library for their processors. However, shame on MathWorks for not investigating the implications for AMD chipsets when utilizing this design choice.

To be fair, Intel has been dominating the processor market for nearly over a decade, so for the clients where performance really matters they were probably running Intel already. But as the tables are turning there is more scrutiny on their design choices (and much deserved).

 

[Assimilator]

As a regular user of MATLAB, I have to say this really falls on Mathworks from my perspective. MATLAB runs on the Java virtual machine so that it can be hardware agnostic and high level. As a company, you want MATLAB to perform its best on any hardware, so whether that's using a proprietary library or a free one, they should strive to optimize for many architectures. Intel is totally in the right to have MKL fallback to SSE if a compatible processor is not detected, it's their library for their processors. However, shame on MathWorks for not investigating the implications for AMD chipsets when utilizing this design choice.

To be fair, Intel has been dominating the processor market for nearly over a decade, so for the clients where performance really matters they were probably running Intel already. But as the tables are turning there is more scrutiny on their design choices (and much deserved).

MathWorks doesn't really have any incentive to do anything because they have a captive market. For all we know they've signed an agreement with Intel to use MKL.

 

[Vya Domus]

Intel is totally in the right to have MKL fallback to SSE if a compatible processor is not detected

I have trouble understanding how this works.

Why doesn't Intel make it so that it will always fail to compile or run when anything other than their ID is found if they are indeed not obligated to support anything other than their own CPUs ? Why the arbitrary fallback to SSE ?

It's obvious to anyone that these were conscious choices meant to disadvantage other vendors. It's one thing to make your software compatible just with your hardware and a totally different mater when your software discriminates other vendors without disclosing this to end users.

MathWorks for not investigating the implications for AMD chipsets when utilizing this design choice.

I know Intel and what sort of deals they make to partners all to well, I am convinced they knew exactly how their compiler and libraries work.

 

[ShredBird]

I have trouble understanding how this works.

Why doesn't Intel make it so that it will always fail to compile when anything other than their ID is found if they are indeed not obligated to support anything other than their own CPUs ? Why the arbitrary fallback to SSE ?

It's obvious to anyone that these were conscious choices meant to disadvantage other vendors. It's one thing to make your software compatible just with your hardware and a totally different mater when your software discriminates other vendors without disclosing this to end users.

I agree with you there. It's not necessarily an intelligent or accommodating decision, but simply one they had the right to make.

This all reminds me a lot of what Nvidia does with some of their tech, they're totally in the right to lock out certain tech or features. They do it all the time to either hinder performance on non-Nvidia GPUs or to force the customer to be locked into their ecosystem. I have no problem if a game developer chooses to use Nvidia tech to get things running the best they can be on Nvidia GPUs, it bothers me when they simply neglect alternative code paths for other GPUs.

In this context, I won't comment on whether this design choice was a conscious decision by both MathWorks and Intel to put certain users at a disadvantage, there is simply no way of knowing. I'm just glad that the media is putting light to this issue so that those of us that do pay a lot for a license can make a stink about it. A lot of researchers are excited about what Zen 2 brings to the table and Mathworks is going to want a piece of that.

 

[ratirt]

They didn't spend money in the first place, but they are going to do it now just to keep them from optimizing for AMD. Until someone publicly shows the problem, like in this case.

They didn't? Why would you say that? they were working on the "different path for other processors than Intel" and that requires programming and what comes after resources to do so. So why you say they did not spend money on that? Maybe they will have to spend more which in my eyes is not surprising

 

[quadibloc]

If Intel wants to provide free software to Intel chip users as a bonus, and AMD chip users have problems using that software, that, in itself wouldn't be unfair competition by Intel. The problem, of course, is that MKL is being used in third-party applications that people buy without expecting them to be tied to "Genuine Intel" processors.

AMD, being a smaller company, doesn't spend as much on software to accompany its products as other companies. Thus, Nvidia gives away a free professional-quality compiler for Windows to make effective use of its graphics cards; AMD's offering in this area is a modified version of an open-source compiler, and it only runs and produces code for Linux.

I think that while hacks, and legal remedies, are a short-term option, AMD will have to look into putting some effort into the software side of things to avoid constantly getting blindsided this way.

 

[notb]

Anticompetitive in terms of it deliberately generating worse code for non-Intel processors for no good reason.

I believe the reason was: no one uses AMD CPUs anyway - why bother?

And as I mentioned multiple times: this was MathWorks decision. If someone find this unfair, he should criticize/contact that company - not Intel.
Obviously, AMD should have done it. They don't, because that's how they're doing business. They make hardware and don't care about anything else - eventually blaming the World that it doesn't change everything possible to support them.

Intel and Nvidia (and every serious hardware maker) provide support / consulting service as well.

which can only mean it's intended to disadvantage non-Intel CPUs, and that is the very definition of anticompetitive behaviour.

But Intel never said MKL should work on AMD CPUs. I've already linked to the site. Supported CPUs: Intel's. End of story.
There's absolutely no reason why they would have to optimize MKL for other hardware.

Given the above, and the fact that MKL is essentially the only library available that does what it does, it could likely be argued that Intel's behaviour here violates antitrust laws.

What??? It's just an Intel-optimized replacement for open source libraries.
Show me one example of what you suggested. Of course other than stuff Intel-specific (like MKL-DNN).

Moreover, AMD makes a very similar product: AOCL (AMD Optimizing CPU Libraries)

AMD Optimizing CPU Libraries (AOCL) - AMD

AMD CPU Libraries is a set of numerical libraries tuned specifically for AMD EPYC™ processor family.

developer.amd.com

(it used to be called ACML: "AMD Core Math Library")

The problem, of course, is that MKL is being used in third-party applications that people buy without expecting them to be tied to "Genuine Intel" processors.

Correct. So AMD should go to MathWorks and say: "hey, your software doesn't utilize our hardware very well - but we'll help you with the coding".
And it seems they haven't.

In fact Intel, Nvidia and any normal company does something like this already before launch of new products.
AMD famously didn't even give mobo makers full chipset specification before Zen launch. It's how they like doing their business. It's their choice.

AMD, being a smaller company, doesn't spend as much on software to accompany its products as other companies.

So they can raise prices and have more cash to spend on software (and documentation, and pre-launch tests, and website).
They don't, consciously. They want to be a cheap alternative. That's it.
AMD is certainly large enough to provide proper support. Many much smaller companies do.

Some milk catrons come with a comfortable opener or a screw cap, while others force you to cut a hole with scissors, but cost $0.1 less. It's a business strategy.

Thus, Nvidia gives away a free professional-quality compiler for Windows to make effective use of its graphics cards; AMD's offering in this area is a modified version of an open-source compiler, and it only runs and produces code for Linux.

Bad argument. You could have kept defending the Intel-AMD differences (which are enormous).

Nvidia employs just 30% more people than AMD (13k vs 10k) and is less than twice as big on revenue ($11.7B vs $6.5B in 2018).
But Nvidia is basically a gold standard when it comes to libraries, software and support.

 

[hurakura]

We all know Intel's way of doing business
no need for you to justify their wrongful actions

 

[DeathtoGnomes]

not sure if this was posted but here this explains how Intel can get away with this compiler cheat. Notice how its a jpeg and not plain text ( it is rumored to prevent it from being websearched).

Development Tools

Search for development software and tools from Intel the way you want.

software.intel.com

Apparantly the FTC is involved here too.

FTC Settles Charges of Anticompetitive Conduct Against Intel

The Federal Trade Commission approved a settlement with Intel Corp. that resolves charges the company illegally stifled competition in the market for computer chips.

www.ftc.gov

https://www.ftc.gov/sites/default/files/documents/cases/101102inteldo.pdf

[for my reference]

How to Bypass Matlab's 'Cripple AMD CPU' Function

If you run Matlab on an AMD processor, you aren't getting all the performance you're entitled to. Matlab refuses to execute certain code on AMD CPUs, even when those CPUs support the code in question. Here's how to fix that.

www.extremetech.com

 

[notb]

not sure if this was posted but here this explains how Intel can get away with this compiler cheat. Notice how its a jpeg and not plain text ( it is rumored to prevent it from being websearched).

Development Tools

Search for development software and tools from Intel the way you want.

software.intel.com

This is a standard and well-known note that Intel published few years ago.
On this particular site they provide a formatted graphics (in multiple languages) to be used by developers in presentations, on websites etc.

You can easily find the exact same notice in Intel's documentation (www, pdfs) - as text, obviously.
As for "being websearched": I wonder if anyone spreading this conspiracy rumor actually tried:
https://www.google.com/search?&q="compilers+may+or+may+not+optimize+to+the+same+degree"

Some results from the query above:
https://software.intel.com/sites/pr...ssue11/7080_2_IN_ParallelMag_Issue11_CBWR.PDF
https://software.intel.com/en-us/onemkl-linux-developer-guide-notices-and-disclaimers
https://software.intel.com/en-us/system-studio/features/build
https://software.intel.com/en-us/parallel-studio-xe/features/analyze

 

[R-T-B]

Of course we could argue if this is OK or not. Intel can't guarantee that AMD CPUs support AVX2 or not. What if they suddenly stopped? Would we them blame Intel for making a library that crashes on competition's CPUs?
But let's not do that.

You could check the cpu supported feature flags at runtime, like a normal program...

Still, I somehow got matlab mixed up with the Intel Compiler Suite. My apologies for blaming the wrong party in this instance.

 

[notb]

You could check the cpu supported feature flags at runtime, like a normal program...

Sure, but why would they?

It strikes me that people who criticize Intel for this may think that the CPU is their product, while the surrounding software is just an addition. A bonus for the community.
That is grossly incorrect.
It's not a fun side project. They've written optimized libraries that are faster than popular open-source variants (even without AVX-512). They've spent money doing it.
Intel MKL is a product as well. They don't make you pay for it, but it's there to make the paid hardware more attractive.
Why would they give it to AMD?

And moving to the inevitable car analogy: should we also expect Mercedes to provide their multimedia system installers compatible with other brands? So that you could run a feature-rich, polished multimedia software on a Dacia Sandero?

Seriously, Intel could have just limited MKL to their CPUs - just like Nvidia did with CUDA.
That would be totally fine and wouldn't provoke the shitstorm on gaming forums that we have today.

 

[DeathtoGnomes]

Sure, but why would they?

It strikes me that people who criticize Intel for this may think that the CPU is their product, while the surrounding software is just an addition. A bonus for the community.
That is grossly incorrect.
It's not a fun side project. They've written optimized libraries that are faster than popular open-source variants (even without AVX-512). They've spent money doing it.
Intel MKL is a product as well. They don't make you pay for it, but it's there to make the paid hardware more attractive.
Why would they give it to AMD?

And moving to the inevitable car analogy: should we also expect Mercedes to provide their multimedia system installers compatible with other brands? So that you could run a feature-rich, polished multimedia software on a Dacia Sandero?

Seriously, Intel could have just limited MKL to their CPUs - just like Nvidia did with CUDA.
That would be totally fine and wouldn't provoke the shitstorm on gaming forums that we have today.

when the FTC criticizes this and you defend INTEL....just makes my day.
EDIT: mix up there.

 

[notb]

when the FCC criticizes this and you defend INTEL....just makes my day.

Have you actually read the FTC article from your earlier post? Do you understand it?
And do you understand the idea of a calendar?

This FTC case is from 2009-2010.
The notice (which you've linked as well) is from 2011 and it's a response to that FTC ruling.

In short:
Intel's software is not offering full support of competing hardware products (which is shocking - I know!), but this wasn't "written on the box".
So now it is.

Seriously, no one expects you to be a C++ programmer or Intel compiler/MKL expert. But you should understand your mistake by now, so stop writing these pointless posts.
And, please, stop tagging me in other threads with that "Intel expert" irony. Get a life.

 

[DeathtoGnomes]

Intel's software is not offering full support of competing hardware products (which is shocking - I know!)

I'm confident that to say that not everyone expected them too, but thats not really the point now is it? it strikes me to call anyone a fanboi since fanboi's cherry pick things to "Discuss" and argue, but the fact that Intel carried on despite the ruling working thru loopholes. its a shady practice in this day and age, its not misunderstood and its not unexpected either. The scandal here, like any other tech, is in getting called out on it.

The only thing pointless here is arguing with me, I'm no expert in anything, but thats the problem with a jack of all trades.

 

[Ned Flanders]

@notb

I understand your frustration. The MKL is an Intel marketing instrument and it is their choice to allow customers to enable or disabled it's functionality to non Intel CPUs. They decided to exclude AMD based systems from using an efficient codepath. Not my style but fair enough.

However, what should not happen is that Intel creates this lib and advertises it to (and I am quoting from the frontpage of Intel's official MKL developers guide) "also perform well on non-Intel processors" while they implement a performance kill switch to prohibit exactly that.

That's just bizarre!

Mathworks did well implementing this lib. Of course it should be implemented in any comparable software because it is arguably the best numeric lib out there. However, where Mathworks failed is to not implement an alternative like BLIS or OpenBLAS and, at the same time state in the system requirements that an "Intel or AMD CPU with AVX2 support is recommended". AVX2 on AMD was never used because of the discriminative nature of the MKL anyway, yet this sentence in the Intel dev guide is there to provoke exactly that. Fool devs to use it without implementing alternatives.

 

[notb]

I understand your frustration.

Frustration?

Also, I'm flattered that someone felt the need to create an account just to share his feedback about my post.
And if you're a regular here, but felt to ashamed to write this under your normal nick - that still counts as an extra labour. So thanks!

However, what should not happen is that Intel creates this lib and advertises it to (and I am quoting from the frontpage of Intel's official MKL developers guide) "also perform well on non-Intel processors" while they implement a performance kill switch to prohibit exactly that.

Funny you've posted a screenshot of small part of the text, but not the link. And you say I'm frustrated.

Developer Guide for Intel® oneAPI Math Kernel Library for Linux*

Intel® oneAPI Math Kernel Library Developer Guide for Linux

software.intel.com

I'm not going to discuss the semantics of "performs well". Maybe you're right. They could have rephrased it.
The "optimization notice" we're discussing here is included on that website. And it's not like it's thousand lines long, so you'll never get there.

And BTW: as text again! @DeathtoGnomes

Mathworks did well implementing this lib. Of course it should be implemented in any comparable software because it is arguably the best numeric lib out there. However, where Mathworks failed is to not implement an alternative like BLIS or OpenBLAS and, at the same time state in the system requirements that an "Intel or AMD CPU with AVX2 support is recommended". AVX2 on AMD was never used because of the discriminative nature of the MKL anyway, yet this sentence in the Intel dev guide is there to provoke exactly that. Fool devs to use it without implementing alternatives.

I've said multiple times that - if anyone - Mathworks is the company that should be discussed here. I'm not sure if "blamed" is a good word as well. They've optimized the software for architecture their clients use. Most of current Matlab code was written in a period AMD had no presence in business/engineering computers.
AMD is getting popular now - I'm sure Mathworks will adjust to provide efficient software to those customers (because why lose them to competitors?).
But people here used this as an occasion to attack Intel, so I reacted. Yes, I know Matlab. Yes, I use MKL. So this was something I could share with people who may not be into this kind of topics.

PC forums used to be about sharing knowledge, while attacking companies and mocking fellow members always had a lower priority (but existed, it's the Internet after all).
I don't understand what happened to TPU.
Sure, this was always a very specific place (very focused on gaming and on desktops, not interested in engineering or pro use, very anti Apple - compared to other tech forums).
But recently it's all about attacking Intel, with some of the staff taking part or at least turning a blind eye. And the dodgy software keys issue on top...
I'm not sure this strategy will work in long term, but owners are entitled to do whatever they want with their site (just like Intel and Mathworks do with their software).

 

[Ned Flanders]

I'm flattered that someone felt the need to create an account just to share his feedback about my post.
And if you're a regular here, but felt to ashamed to write this under your normal nick - that still counts as an extra labour. So thanks!

You are most welcome! I created several accounts in the last weeks to participate in this discussion. I don't think its particularly shameful what I posted here. I did not expect my post to go far out of the matlab community but I have to say I am actually quite happy, that the many reports in most tech mags raised a great level of awareness to the problem. And awareness is the first step to avoidance. So that must be a good thing.

I'm not going to discuss the semantics of "performs well". Maybe you're right. They could have rephrased it.

We can discuss the meanings of "good performance" if you wish. However, I think its clear that this is very misleading and clearly should not be stated there. Instead they should replace it with "MKL does NOT perform well on non-Intel processors" But I feel you agree with that.

And I agree with you regarding Mathworks, it's their turn to implement MKL alternatives now, as already stated. I am quite sure that Intel will not remove their vendor string kill switch.