Esperanto said that it plans to provide access to researchers in the RISC-V community as part of the company’s mission to help “democratise AI” for the broader industry and help accelerate development of Generative AI technology on RISC-V.
This development is part of Esperanto’s strategy to extend the benefits of RISC-V technology in AI and general-purpose applications from cloud to edge. Areas of focus for the company’s Generative AI efforts include use cases for large language models (LLMs) where using low power hardware with improved total cost of ownership (TCO) compared to existing offerings are key.
In particular, several versions of Meta’s Open Pre-Trained Transformer (OPT) model are now running on Esperanto’s hardware at multiple precision levels and context sizes with power levels as low as 25W per chip for inferencing. The rapid porting and bring-up of the OPT models onto ET-SoC-1 silicon were enabled by Esperanto’s machine learning software development kit, which is currently in use by the company’s commercial customers.
Researchers who are granted access to Esperanto’s solutions for R&D purposes will be required to comply with Meta’s open-source license as well as other program terms.
“Generative AI is one of the latest advancements in machine learning, and we are pleased to contribute elements of our efforts in the area of large language models to the RISC-V research community,” said Art Swift, president and CEO at Esperanto Technologies.
“The growing interest in RISC-V signals an important inflection point for the semiconductor industry. This development in the area of Generative AI will further accelerate the evaluation of RISC-V by researchers looking for breakthroughs in this exciting field, and by developers seeking to increase their competitive differentiation beyond what existing platforms can provide,” said Ben Bajarin, CEO and principal analyst at Creative Strategies.
“RISC-V offers unparalleled opportunities for collaboration and customisation, making it ideally suited for this next wave of AI innovation,” said Calista Redmond, CEO of RISC-V International. “Esperanto is one of the companies leading the charge in this space, pushing the limits of performance and power-efficiency to make Generative AI development more accessible.”
For commercial customers, Esperanto is currently shipping AI evaluation servers which deliver high performance combined with high energy efficiency and low TCO. Available in a standard 2U-high form factor, each Esperanto evaluation server includes dual Xeon host processors and either 8 or 16 ET-SoC-1 PCIe cards.
Each Esperanto PCIe card has over 1,000 64-bit RISC-V CPUs with attached vector/tensor units, delivering up to 16,000 RISC-V CPUs per server. Esperanto’s evaluation servers enable customers to obtain performance and power data from running a variety of industry standard AI models, as well as the ability to bring their own models and data.
