Monday, October 28th 2024
Interview with RISC-V International: High-Performance Chips, AI, Ecosystem Fragmentation, and The Future
RISC-V is an industry standard instruction set architecture (ISA) born in UC Berkeley. RISC-V is the fifth iteration in the lineage of historic RISC processors. The core value of the RISC-V ISA is the freedom of usage it offers. Any organization can leverage the ISA to design the best possible core for their specific needs, with no regional restrictions or licensing costs. It attracts a massive ecosystem of developers and companies building systems using the RISC-V ISA. To support these efforts and grow the ecosystem, the brains behind RISC decided to form RISC-V International—a non-profit foundation that governs the ISA and guides the ecosystem.
We had the privilege of talking with Andrea Gallo, Vice President of Technology at RISC-V International. Andrea oversees the technological advancement of RISC-V, collaborating with vendors and institutions to overcome challenges and expand its global presence. Andrea's career in technology spans several influential roles at major companies. Before joining RISC-V International, he worked at Linaro, where he pioneered Arm data center engineering initiatives, later overseeing diverse technological sectors as Vice President of Segment Groups, and ultimately managing crucial business development activities as executive Vice President. During his earlier tenure as a Fellow at ST-Ericsson, he focused on smartphone and application processor technology, and at STMicroelectronics he optimized hardware-software architectures and established international development teams.
Unlike Arm and x86, RISC-V allows anyone to implement the ISA in their processor cores without licensing fees, and encourages community contributions to the standard. It offers the flexibility to add custom extensions to the base ISA, enabling companies to develop specialized accelerators for specific applications. However, it's important to note that the standard maintains rigorous criteria for accepting contributions, involving an extensive review process. This approach helps mitigate the risk of ecosystem fragmentation that can occur when numerous companies create their own extensions, potentially leading to compatibility issues. We asked the hard questions about ecosystem fragmentation, the HPC sector, the mobile industry, AI, and the future of RISC-V. Below is our in-depth interview with Andrea Gallo.
TechPowerUp: RISC-V in the data center: How is the RISC-V foundation supporting companies in the high-performance computing sector?
Andrea: There are two strings of activities to grow presence in HPC: performance and security. We have ratified the vector extension and are working on defining matrix extension, all aimed at improving performance.
On the security side, we recently ratified important extensions related to control flow integrity, such as "Shadow Stack and Landing Pads." These ensure that when you have function calls, the return address remains intact and uncompromised. We have also ratified pointer masking, a crucial first step towards memory tagging, where masked address bits within a process address space can later support memory tagging. Additionally, we are working on supervisor domain access protection (SMMTT).
Altogether, these efforts strengthen both performance and security for high-performance computing and data centers.
TechPowerUp: What about mobile devices? We are seeing in a giant uptick in mobile computing power. How would the RISC-V foundation like to fit into that? Would that be pushing more high performance designs, more efficient designs, or anything else?
Andrea: There is an Android special interest group (SIG) and an Android RISC-V 64 project on GitHub where all the communication and documentation are stored. There's a lot of ongoing activity around Android on RISC-V. New chips on the market now support the RVV 1.0 vector extension. We are also starting to see development boards that use these vector extensions, such as the Banana Pi and the Deep Computing DC-Roma II laptop. This is very important from the developers' perspective since native development on the target platform is essential.
There are also performance initiatives similar to those in the HPC space, focusing on vector extensions and providing commercial development platforms. Additionally, we have a dev board program to review new development boards with the latest chips and extensions, ensuring they have optimal performance and security extensions. We stock these boards and provide them to key maintainers and developers in the ecosystem, making sure that operating system distributions are ported and tested.
Just this year, we have shipped more than 200 boards to individuals. If any key maintainer needs a board that they have not been able to get, they can contact us at help@riscv.org for evaluation and support.
TechPowerUp: So RISC-V International is actually helping out developers with development boards?
Andrea: Yeah.
TechPowerUp: So our next question is that, the current boom in technology is actually AI, and there are some accelerators being developed specifically to accelerate AI. That includes matrix multiplication, accumulation, and all those specific things. And there are companies like Esperanto AI and Tenstorrent, doing accelerators based on RISC-V. Is there any possibility that we would see AI specific instruction extensions in the future?
Andrea:It's not just Esperanto and Tenstorrent—Axelera, NVIDIA and Meta have all publicly shared that they're using RISC-V. NVIDIA integrates RISC-V into their GPUs and Meta uses it in their AI accelerators. So, yeah, RISC-V is everywhere AI is.
When it comes to custom instructions, we have an AI/ML SIG. The role of a SIG is to analyze a specific area, identify gaps, highlight product opportunities, and justify new development efforts.
When we ratify a new extension, we know that there's a need for real-world products. For example, think of the open source development in Linux. A subsystem maintainer or one of the higher architecture maintainers will not accept new code, subsystem, or contribution unless there's a demonstrated need. Every addition increases your cost of ownership and the baggage that you carry from one release to the next. The same principle applies to the RISC-V ISA. SIGs analyze gaps, identify solutions. In this case, for AI/ML, all the companies that we have listed are all in a position to propose specific new instructions.
The specification process that leads to the ratification of extensions is a rigorous process. The recent ratification of the BFloat16 reflects the needs of floating-point formats for AI/ML. And the ongoing work around matrix extensions is really driven by the machine learning algorithms.
TechPowerUp: Our follow-up on that would be, how fast are those special interest groups? How fast are they ratifying this specification for the ISA extension?
Andrea: The speed depends on the complexity of the proposal. If something is very minor, we can go for a fast track, and it can take a few months. If it is a major specification, then it shall go through all the process, with specific review windows, that can take six months or more. This really depends on the complexity because it's really important that we have a rigorous review.
There's a misconception with RISC-V is that everyone adds new custom instructions, and there's huge fragmentation. As I said, I joined just end of June. My first day was the summit in Munich, the European Summit. I've been impressed by the rigor and the thoroughness of the process of the review process. The specifications are reviewed by the task group that prepares them. There's an architecture review committee, then there's one month of public review. There's the review by the technical steering committee. There's a review by the all the committee chairs, by the board of directors. So there's an attention, towards a rigorous process avoiding unnecessary fragmentation.
TechPowerUp: We briefly touched upon when everyone is doing their own, custom instructions. So for example, if we wanted to build a RISC-V accelerator, we use the base ISA and add our application specific instruction sets that accelerate the AI program. We know that it's a feature to allow for these custom extensions, but it is creating giant fragmentation in the ecosystem. What is RISC-V International doing to solve that issue?
Andrea: I mentioned the rigor of the process to write, rectify and extend a new specification. If you want to claim that you are RISC-V compatible, then there's an architecture compatibility test suite that verifies that you are complying with the ISA. We run the same tests on a golden reference model and compare the signatures of the tests to ensure alignment with the specification.
The next step in preventing fragmentation is at the software porting level. In embedded, you may have a vertically integrated software approach with vendors or device makers who control the entire vertical software stack with the famous "spaghetti code" way of working. However, modern application processors need to run a binary OS distribution without changes. So here if an OS vendor targets just the minimal compatibility across products, then it would be the very basic RV64I or maybe RV64G, which is a very small subset. To address this, we are working on the profiles.
We have a significant number of extensions grouped by profiles. So, specifically, we have an application processor profile, and over time, we upgrade these profile specifications. And this is a set of mandatory extensions and some optional extensions.
We just ratified the RVA23 Profile. The newly ratified RVA23 Profile is a major release for the RISC-V software ecosystem and will help accelerate widespread implementation among toolchains and operating systems. You can learn more about it in our latest announcement.
The next step is platforms. To further improve and accelerate the reuse of software across verticals or within the same vertical across products, we are, as an ecosystem, agreeing on a set of hardware and software interfaces that will be the same and part of a platform specification. There's a team that is working on a server SOC and a server platform. This includes things like having the same interfaces for timers, clock, IOMMU, RAS and the related error reporting mechanisms. We all agree that we should use the same interfaces for specific peripherals, that are part, for example, of a server platform.
TechPowerUp: So, what is the need for yet another commercial instruction set? What is the RISC-V International Foundation doing better than competition like Arm and now, joint forces of x86?
Andrea:I would like to answer this question from two different perspectives: innovation and freedom from lock-in.
The rate, energy, and pace of innovation in the RISC-V ecosystem is unbelievable. The fact that anyone can start from a training course from the RISC-V website and learn how to develop a RISC-V core and add custom extensions is unleashing imagination. From a developer's perspective, being able to develop a RISC-V core from day zero is a huge value. I took one of these courses as part of my ramp up, and it blew my mind out. That was unbelievable. And at the same time, as custodians of the RISC-V ISA, we're able to funnel this energy towards new standards and compliance. All this is something that the other architectures that you mentioned cannot achieve. Companies that are market competitors collaborate within RISC-V International meetings towards common goals. We have more than 4,500 members. You cannot see this anywhere else.
Another very important aspect is freedom from lock-in. It's not just about licensing model or royalties, but it's about the ability to control your destiny without depending on another entity that may suddenly stop supporting you. Nowadays, this can be a national security issue. There are many countries and governments today investing in RISC-V from a digital sovereignty perspective. You were correctly pointing to AI. Today, AI is becoming critical in our lives, and countries are investing towards digital sovereignty to make sure that they are building competence to develop their own AI solutions in house in terms of competence, expertise, but also, IP.
We see this momentum globally. The EU is funding collaborative projects to develop software defined vehicles based on RISC-V. China has the famous "One Student One Chip" program, led by the Beijing Open Source Chip Research Institute and the University of the Chinese Academy of Sciences. They have thousands of students who propose and design chips based on RISC-V, and more than ten are taped out and working. A few months ago, Brazil joined RISC-V International as a member because they want to grow and accelerate programs based on RISC-V in Brazil. And of course, UC Berkeley continues to play a role in academic research. Universities, governments, and multinational companies around the world are taking control of their own destiny, investing in RISC-V to solve local problems, while engaging globally with the RISC-V ecosystem.
TechPowerUp: You're actually saying that the two paths to RISC-V success are: First, provide hardware to developers to train them on RISC-V, which will help them become skilled engineers who may eventually work in companies that create RISC-V software and hardware. And the second is taking matters into your own hands, basically.
Andrea: Yeah. It's students, academia, startups, multinational companies, and countries.
TechPowerUp: So we have one final question that would be pretty interesting to our readers. Where do you see RISC-V in about 10 years or so?
Andrea: Looking back, the growth from an academic project at UC Berkeley to where we are today is an unbelievable journey. Just in 2023, there was a 2.5x growth in terms of the overall business in the ecosystem over 2022. The SHD Group predicts that RISC-V will capture around 30% market share of industry market verticals from consumer, computer, and automotive, to datacenter and industrial by 2030, with over 20 billion RISC-V based SoCs shipping annually. We're no longer counting cores anymore—we're counting chips, which include many, many cores each. 10 years from now, I want to see RISC-V as the de facto ISA of choice for every new product design.
We had the privilege of talking with Andrea Gallo, Vice President of Technology at RISC-V International. Andrea oversees the technological advancement of RISC-V, collaborating with vendors and institutions to overcome challenges and expand its global presence. Andrea's career in technology spans several influential roles at major companies. Before joining RISC-V International, he worked at Linaro, where he pioneered Arm data center engineering initiatives, later overseeing diverse technological sectors as Vice President of Segment Groups, and ultimately managing crucial business development activities as executive Vice President. During his earlier tenure as a Fellow at ST-Ericsson, he focused on smartphone and application processor technology, and at STMicroelectronics he optimized hardware-software architectures and established international development teams.
TechPowerUp: RISC-V in the data center: How is the RISC-V foundation supporting companies in the high-performance computing sector?
Andrea: There are two strings of activities to grow presence in HPC: performance and security. We have ratified the vector extension and are working on defining matrix extension, all aimed at improving performance.
On the security side, we recently ratified important extensions related to control flow integrity, such as "Shadow Stack and Landing Pads." These ensure that when you have function calls, the return address remains intact and uncompromised. We have also ratified pointer masking, a crucial first step towards memory tagging, where masked address bits within a process address space can later support memory tagging. Additionally, we are working on supervisor domain access protection (SMMTT).
Altogether, these efforts strengthen both performance and security for high-performance computing and data centers.
TechPowerUp: What about mobile devices? We are seeing in a giant uptick in mobile computing power. How would the RISC-V foundation like to fit into that? Would that be pushing more high performance designs, more efficient designs, or anything else?
Andrea: There is an Android special interest group (SIG) and an Android RISC-V 64 project on GitHub where all the communication and documentation are stored. There's a lot of ongoing activity around Android on RISC-V. New chips on the market now support the RVV 1.0 vector extension. We are also starting to see development boards that use these vector extensions, such as the Banana Pi and the Deep Computing DC-Roma II laptop. This is very important from the developers' perspective since native development on the target platform is essential.
There are also performance initiatives similar to those in the HPC space, focusing on vector extensions and providing commercial development platforms. Additionally, we have a dev board program to review new development boards with the latest chips and extensions, ensuring they have optimal performance and security extensions. We stock these boards and provide them to key maintainers and developers in the ecosystem, making sure that operating system distributions are ported and tested.
Just this year, we have shipped more than 200 boards to individuals. If any key maintainer needs a board that they have not been able to get, they can contact us at help@riscv.org for evaluation and support.
TechPowerUp: So RISC-V International is actually helping out developers with development boards?
Andrea: Yeah.
TechPowerUp: So our next question is that, the current boom in technology is actually AI, and there are some accelerators being developed specifically to accelerate AI. That includes matrix multiplication, accumulation, and all those specific things. And there are companies like Esperanto AI and Tenstorrent, doing accelerators based on RISC-V. Is there any possibility that we would see AI specific instruction extensions in the future?
Andrea:It's not just Esperanto and Tenstorrent—Axelera, NVIDIA and Meta have all publicly shared that they're using RISC-V. NVIDIA integrates RISC-V into their GPUs and Meta uses it in their AI accelerators. So, yeah, RISC-V is everywhere AI is.
When it comes to custom instructions, we have an AI/ML SIG. The role of a SIG is to analyze a specific area, identify gaps, highlight product opportunities, and justify new development efforts.
When we ratify a new extension, we know that there's a need for real-world products. For example, think of the open source development in Linux. A subsystem maintainer or one of the higher architecture maintainers will not accept new code, subsystem, or contribution unless there's a demonstrated need. Every addition increases your cost of ownership and the baggage that you carry from one release to the next. The same principle applies to the RISC-V ISA. SIGs analyze gaps, identify solutions. In this case, for AI/ML, all the companies that we have listed are all in a position to propose specific new instructions.
The specification process that leads to the ratification of extensions is a rigorous process. The recent ratification of the BFloat16 reflects the needs of floating-point formats for AI/ML. And the ongoing work around matrix extensions is really driven by the machine learning algorithms.
TechPowerUp: Our follow-up on that would be, how fast are those special interest groups? How fast are they ratifying this specification for the ISA extension?
Andrea: The speed depends on the complexity of the proposal. If something is very minor, we can go for a fast track, and it can take a few months. If it is a major specification, then it shall go through all the process, with specific review windows, that can take six months or more. This really depends on the complexity because it's really important that we have a rigorous review.
There's a misconception with RISC-V is that everyone adds new custom instructions, and there's huge fragmentation. As I said, I joined just end of June. My first day was the summit in Munich, the European Summit. I've been impressed by the rigor and the thoroughness of the process of the review process. The specifications are reviewed by the task group that prepares them. There's an architecture review committee, then there's one month of public review. There's the review by the technical steering committee. There's a review by the all the committee chairs, by the board of directors. So there's an attention, towards a rigorous process avoiding unnecessary fragmentation.
TechPowerUp: We briefly touched upon when everyone is doing their own, custom instructions. So for example, if we wanted to build a RISC-V accelerator, we use the base ISA and add our application specific instruction sets that accelerate the AI program. We know that it's a feature to allow for these custom extensions, but it is creating giant fragmentation in the ecosystem. What is RISC-V International doing to solve that issue?
Andrea: I mentioned the rigor of the process to write, rectify and extend a new specification. If you want to claim that you are RISC-V compatible, then there's an architecture compatibility test suite that verifies that you are complying with the ISA. We run the same tests on a golden reference model and compare the signatures of the tests to ensure alignment with the specification.
The next step in preventing fragmentation is at the software porting level. In embedded, you may have a vertically integrated software approach with vendors or device makers who control the entire vertical software stack with the famous "spaghetti code" way of working. However, modern application processors need to run a binary OS distribution without changes. So here if an OS vendor targets just the minimal compatibility across products, then it would be the very basic RV64I or maybe RV64G, which is a very small subset. To address this, we are working on the profiles.
We have a significant number of extensions grouped by profiles. So, specifically, we have an application processor profile, and over time, we upgrade these profile specifications. And this is a set of mandatory extensions and some optional extensions.
We just ratified the RVA23 Profile. The newly ratified RVA23 Profile is a major release for the RISC-V software ecosystem and will help accelerate widespread implementation among toolchains and operating systems. You can learn more about it in our latest announcement.
The next step is platforms. To further improve and accelerate the reuse of software across verticals or within the same vertical across products, we are, as an ecosystem, agreeing on a set of hardware and software interfaces that will be the same and part of a platform specification. There's a team that is working on a server SOC and a server platform. This includes things like having the same interfaces for timers, clock, IOMMU, RAS and the related error reporting mechanisms. We all agree that we should use the same interfaces for specific peripherals, that are part, for example, of a server platform.
TechPowerUp: So, what is the need for yet another commercial instruction set? What is the RISC-V International Foundation doing better than competition like Arm and now, joint forces of x86?
Andrea:I would like to answer this question from two different perspectives: innovation and freedom from lock-in.
The rate, energy, and pace of innovation in the RISC-V ecosystem is unbelievable. The fact that anyone can start from a training course from the RISC-V website and learn how to develop a RISC-V core and add custom extensions is unleashing imagination. From a developer's perspective, being able to develop a RISC-V core from day zero is a huge value. I took one of these courses as part of my ramp up, and it blew my mind out. That was unbelievable. And at the same time, as custodians of the RISC-V ISA, we're able to funnel this energy towards new standards and compliance. All this is something that the other architectures that you mentioned cannot achieve. Companies that are market competitors collaborate within RISC-V International meetings towards common goals. We have more than 4,500 members. You cannot see this anywhere else.
Another very important aspect is freedom from lock-in. It's not just about licensing model or royalties, but it's about the ability to control your destiny without depending on another entity that may suddenly stop supporting you. Nowadays, this can be a national security issue. There are many countries and governments today investing in RISC-V from a digital sovereignty perspective. You were correctly pointing to AI. Today, AI is becoming critical in our lives, and countries are investing towards digital sovereignty to make sure that they are building competence to develop their own AI solutions in house in terms of competence, expertise, but also, IP.
We see this momentum globally. The EU is funding collaborative projects to develop software defined vehicles based on RISC-V. China has the famous "One Student One Chip" program, led by the Beijing Open Source Chip Research Institute and the University of the Chinese Academy of Sciences. They have thousands of students who propose and design chips based on RISC-V, and more than ten are taped out and working. A few months ago, Brazil joined RISC-V International as a member because they want to grow and accelerate programs based on RISC-V in Brazil. And of course, UC Berkeley continues to play a role in academic research. Universities, governments, and multinational companies around the world are taking control of their own destiny, investing in RISC-V to solve local problems, while engaging globally with the RISC-V ecosystem.
TechPowerUp: You're actually saying that the two paths to RISC-V success are: First, provide hardware to developers to train them on RISC-V, which will help them become skilled engineers who may eventually work in companies that create RISC-V software and hardware. And the second is taking matters into your own hands, basically.
Andrea: Yeah. It's students, academia, startups, multinational companies, and countries.
TechPowerUp: So we have one final question that would be pretty interesting to our readers. Where do you see RISC-V in about 10 years or so?
Andrea: Looking back, the growth from an academic project at UC Berkeley to where we are today is an unbelievable journey. Just in 2023, there was a 2.5x growth in terms of the overall business in the ecosystem over 2022. The SHD Group predicts that RISC-V will capture around 30% market share of industry market verticals from consumer, computer, and automotive, to datacenter and industrial by 2030, with over 20 billion RISC-V based SoCs shipping annually. We're no longer counting cores anymore—we're counting chips, which include many, many cores each. 10 years from now, I want to see RISC-V as the de facto ISA of choice for every new product design.
7 Comments on Interview with RISC-V International: High-Performance Chips, AI, Ecosystem Fragmentation, and The Future
This being said I’m happy to see TPU covering this emerging tech and busting out of the old Windows-x86 mold!
Very nice interview!
I dont see anyone else having any issues with Arm.
Some time ago Google declared a Tier-1 support for Android on RISC-V project ( for smartphones ). This is Not a good sign that Google is Not mentioned in the interview at all!
There is No a significant progress in consumer PCs ( notebooks / desktops / day-to-day usage and gaming ) and HPC ( scientific applications and data centers ).
Take into account that even if the market is flooded with RISC-V SBCs most of them are Development Boards ( ! ) and Not end-to-user consumer products.