Application Infrastructure

CEVA Redefines High Performance AI/ML Processing for Edge AI and Edge Compute Devices

CEVA | January 06, 2022

Consumer Electronics Show  – CEVA, Inc.the leading licensor of wireless connectivity and smart sensing technologies and integrated IP solutions, today announced NeuPro-M, its latest generation processor architecture for artificial intelligence and machine learning (AI/ML) inference workloads. Targeting the broad markets of Edge AI and Edge Compute, NeuPro-M is a self-contained heterogeneous architecture that is composed of multiple specialized co-processors and configurable hardware accelerators that seamlessly and simultaneously process diverse workloads of Deep Neural Networks, boosting performance by 5-15X compared to its predecessor. An industry first, NeuPro-M supports both system-on-chip (SoC) as well as Heterogeneous SoC (HSoC) scalability to achieve up to 1,200 TOPS and offers optional robust secure boot and end-to-end data privacy.

NeuPro-M is the latest generation processor architecture from CEVA for artificial intelligence and machine learning (AI/ML) inference workloads. Targeting the broad markets of Edge AI and Edge Compute, NeuPro-M is a self-contained heterogeneous architecture that is composed of multiple specialized co-processors and configurable hardware accelerators that seamlessly and simultaneously process diverse workloads of Deep Neural Networks, boosting performance by 5-15X compared to its predecessor.
NeuPro-M is the latest generation processor architecture from CEVA for artificial intelligence and machine learning (AI/ML) inference workloads. Targeting the broad markets of Edge AI and Edge Compute, NeuPro-M is a self-contained heterogeneous architecture that is composed of multiple specialized co-processors and configurable hardware accelerators that seamlessly and simultaneously process diverse workloads of Deep Neural Networks, boosting performance by 5-15X compared to its predecessor.
NeuPro–M compliant processors initially include the following pre-configured cores:

  • NPM11 – single NeuPro-M engine, up to 20 TOPS at 1.25GHz
  • NPM18 – eight NeuPro-M engines, up to 160 TOPS at 1.25GHz

Illustrating its leading-edge performance, a single NPM11 core, when processing a ResNet50 convolutional neural network, achieves a 5X performance increase and 6X memory bandwidth reduction versus its predecessor, which results in exceptional power efficiency of up to 24 TOPS per watt.

Built on the success of its' predecessors, NeuPro-M is capable of processing all known neural network architectures, as well as integrated native support for next-generation networks like transformers, 3D convolution, self-attention and all types of recurrent neural networks. NeuPro-M has been optimized to process more than 250 neural networks, more than 450 AI kernels and more than 50 algorithms. The embedded vector processing unit (VPU) ensures future proof software-based support of new neural network topologies and new advances in AI workloads. Furthermore, the CDNN offline compression tool can increase the FPS/Watt of the NeuPro-M by a factor of 5-10X for common benchmarks, with very minimal impact on accuracy.

"The artificial intelligence and machine learning processing requirements of edge AI and edge compute are growing at an incredible rate, as more and more data is generated and sensor-related software workloads continue to migrate to neural networks for better performance and efficiencies. With the power budget remaining the same for these devices, we need to find new and innovative methods of utilizing AI at the edge in these increasingly sophisticated systems. NeuPro-M is designed on the back of our extensive experience deploying AI processors and accelerators in millions of devices, from drones to security cameras, smartphones and automotive systems. Its innovative, distributed architecture and shared memory system controllers reduces bandwidth and latency to an absolute minimum and provides superb overall utilization and power efficiency. With the ability to connect multiple NeuPro-M compliant cores in a SoC or Chiplet to address the most demanding AI workloads, our customers can take their smart edge processor designs to the next level."

Ran Snir, Vice President and General Manager of the Vision Business Unit at CEVA

The NeuPro-M heterogenic architecture is composed of function-specific co-processors and load balancing mechanisms that are the main contributors to the huge leap in performance and efficiency compared to its predecessor. By distributing control functions to local controllers and implementing local memory resources in a hierarchical manner, the NeuPro-M achieves data flow flexibility that result in more than 90% utilization and protects against data starvation of the different co-processors and accelerators at any given time. The optimal load balancing is obtained by practicing various data flow schemes that are adopted to the specific network, the desired bandwidth, the available memory and the target performance, by the CDNN framework.

NeuPro-M architecture highlights include:

  • Main grid array consisting of 4K MACs (Multiply And Accumulates), with mixed precision of 2-16 bits
  • Winograd transform engine for weights and activations, reducing convolution time by 2X and allowing 8-bit convolution processing with <0.5% precision degradation
  • Sparsity engine to avoid operations with zero-value weights or activations per layer, for up to 4X performance gain, while reducing memory bandwidth and power consumption
  • Fully programmable Vector Processing Unit, for handling new unsupported neural network architectures with all data types, from 32-bit Floating Point down to 2-bit Binary Neural Networks (BNN)
  • Configurable Weight and Data compression down to 2-bits while storing to memory, and real-time decompression upon reading, for reduced memory bandwidth
  • Dynamically configured two level memory architecture to minimize power consumption attributed to data transfers to and from an external SDRAM

To illustrate the benefit of these innovative features in the NeuPro-M architecture, concurrent use of the orthogonal mechanisms of Winograd transform, Sparsity engine, and low-resolution 4x4-bit activations, delivers more than a 3X reduction in cycle count of networks such as Resnet50 and Yolo V3.

As neural network Weights and Biases and the data set and network topology become key Intellectual Property of the owner, there is a strong need to protect these from unauthorized use. The NeuPro-M architecture supports secure access in the form of optional root of trust, authentication, and cryptographic accelerators.

For the automotive market, NeuPro-M cores and its CEVA Deep Neural Network (CDNN) deep learning compiler and software toolkit comply with Automotive ISO26262 ASIL-B functional safety standard and meets the stringent quality assurance standards IATF16949 and A-Spice.

Together with CEVA's multi award-winning neural network compiler – CDNN – and its robust software development environment, NeuPro-M provides a fully programmable hardware/software AI development environment for customers to maximize their AI performance. CDNN includes innovative software that can fully utilize the customers' NeuPro-M customized hardware to optimize power, performance & bandwidth. The CDNN software also includes a memory manager for memory reduction and optimal load balancing algorithms, and wide support of various network formats including ONNX, Caffe, TensorFlow, TensorFlow Lite, Pytorch and more. CDNN is compatible with common open-source frameworks, including Glow, tvm, Halide and TensorFlow and includes model optimization features like 'layer fusion' and 'post training quantization' all while using precision conservation methods.

NeuPro-M is available for licensing to lead customers today and for general licensing in Q2 this year. NeuPro-M customers can also benefit from Heterogenous SoC design services from CEVA to help integrate and support system design and chiplet development. 


About CEVA, Inc.
CEVA is the leading licensor of wireless connectivity and smart sensing technologies and integrated IP solutions for a smarter, safer, connected world. We provide Digital Signal Processors, AI engines, wireless platforms, cryptography cores and complementary software for sensor fusion, image enhancement, computer vision, voice input and artificial intelligence. These technologies are offered in combination with our Intrinsix IP integration services, helping our customers address their most complex and time-critical integrated circuit design projects. Leveraging our technologies and chip design skills, many of the world's leading semiconductors, system companies and OEMs create power-efficient, intelligent, secure and connected devices for a range of end markets, including mobile, consumer, automotive, robotics, industrial, aerospace & defense and IoT.

Our DSP-based solutions include platforms for 5G baseband processing in mobile, IoT and infrastructure, advanced imaging and computer vision for any camera-enabled device, audio/voice/speech and ultra-low-power always-on/sensing applications for multiple IoT markets. For sensor fusion, our Hillcrest Labs sensor processing technologies provide a broad range of sensor fusion software and inertial measurement unit ("IMU") solutions for markets including hearables, wearables, AR/VR, PC, robotics, remote controls and IoT. For wireless IoT, our platforms for Bluetooth (low energy and dual mode), Wi-Fi 4/5/6/6e (802.11n/ac/ax), Ultra-wideband (UWB), NB-IoT and GNSS are the most broadly licensed connectivity platforms in the industry.

Spotlight

Making the decision to use an electronic signature tool is a big step on the way to building an efficient agreement system. But not all e-signature providers are the same. Before you make that step, you want to make sure that you’re lined up in the right direction. Agreements don’t start or end when someone signs on the dotted d

Spotlight

Making the decision to use an electronic signature tool is a big step on the way to building an efficient agreement system. But not all e-signature providers are the same. Before you make that step, you want to make sure that you’re lined up in the right direction. Agreements don’t start or end when someone signs on the dotted d

Related News

Hyper-Converged Infrastructure, Windows Systems and Network, IT Systems Management

NetActuate Releases the 8th Generation of its Platform, Offering Streamlined, Intuitive Management of Complex Global Deployments

PRWeb | August 14, 2023

NetActuate, a leading provider of global infrastructure and network services, has announced today the release of the eighth generation of its global platform. Existing customers can now experience powerful new features for streamlined self-service management of their global deployments. The new release builds on NetActuate's years of experience operating self-service cloud and networking platforms. The eighth version incorporates an intuitive, robust UI that enables greater insight and visibility across a range of infrastructure and network services. From virtual servers to bare metal and colocation, the new platform allows for easier monitoring and optimization, as well as greater self-service options than ever before. "We couldn't be prouder of the work done by our development and engineering teams to deliver the eighth generation of our platform," said Mark Mahle, CEO of NetActuate. "From the data center up, we have always had full control over our entire stack. This allows us to innovate at all levels to deliver numerous improvements for our customers." Inside the new platform, users can intuitively and easily manage their entire global deployment. From spinning up new virtual servers, to monitoring bandwidth in the data center, NetActuate customers now have more control than ever before, right at their fingertips. "Unlike other companies in this space, NetActuate is truly engineering-led," said Mark Price, Vice President of Infrastructure. "Our development and engineering teams worked hand-in-hand to rework the entire platform experience for end users, and add in powerful new capabilities wherever we could." Anycast customers now have powerful new tools for node management. From adding and removing locations instantly, to enabling them to see their entire anycast network at-a-glance, network optimization is now easier than ever. About NetActuate NetActuate is a leading provider of highly available, low latency custom network and infrastructure services that reach every major global market. From the datacenter to the last mile, we help providers take their products and services to the global edge faster. Our customers can rapidly scale without fear of high costs or devastating performance issues. We built one of the world's largest global networks by number of peers, and it serves as the foundation for our performance BGP anycast platform that powers over 25 billion transactions a day.

Read More

Hyper-Converged Infrastructure, Windows Systems and Network

EdgeConneX Expands Cloud Connectivity Capabilities in Phoenix with AWS Direct Connect

prnewswire | July 12, 2023

EdgeConneX®, the pioneer in global Hyperlocal to Hyperscale Data Center Solutions, announces the deployment of AWS Direct Connect with both 10Gbps and 100Gbps capability at the Phoenix Edge Data Center® (PHX01). EdgeConneX is a member of the Amazon Web Services (AWS) Partner Network (APN), and AWS Direct Connect is already available in the Portland Edge Data Center® Campus (POR01 and POR02). AWS Direct Connect allows customers to establish direct edge cloud on-ramps, which can reduce costs, improve operations, and deliver a superior and consistent network experience for their data-intensive workloads. The greater Phoenix area has developed into a burgeoning tech and business ecosystem in the Southwest region, in part due to local business incentives, tax exemptions, and low latency routes to larger metros. By layering on the added flexibility of AWS Direct Connect in the EdgeConneX Phoenix data center, companies can advance their digital transformation strategies and application evolution by linking their internal network via direct point-to-point connections. This creates a secure, private cloud connection with access to both 10Gbps and 100Gbps AWS Direct Connect ports. Phillip Marangella, Chief Marketing and Product Officer at EdgeConneX: "EdgeConneX customers are increasingly leveraging hybrid architecture solutions to address high bandwidth workloads, such as artificial intelligence (AI), machine learning (ML), and augmented and virtual reality (AR/VR). EdgeConneX is working with AWS to offer customers in Phoenix expanded workload transport options and improved scalability and reliability." Emad Benjamin, General Manager of AWS Direct Connect at AWS: "Emerging tech applications and cloud-based IT architectures require high availability and lower latency connectivity. With AWS Direct Connect in the EdgeConneX Phoenix data center, customers can create virtual interfaces directly to AWS and access services such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3), allowing for increased security and consistent network experiences." Michael Reid, CEO for Megaport "Our customers with mission-critical applications require secure, on-premises infrastructure to meet optimal application performance. By leveraging AWS Direct Connect at the EdgeConneX Phoenix data center, our customers can benefit from low-latency connections to the cloud, essential to supporting the modern applications that power their businesses." The EdgeConneX Phoenix Data Center is engineered for customers requiring the lowest latency and is marked by Tier III design, and just 10 miles from downtown Phoenix. In addition, EdgeConneX is building a 100MW data center campus in nearby Mesa, Arizona. About EdgeConneX Backed by EQT Infrastructure, part of the global investment organization EQT, EdgeConneX provides a full range of sustainable data center solutions worldwide. We work closely with our customers to offer choices in location, scale, and type of facility, from Hyperlocal to Hyperscale. EdgeConneX is a global leader in anytime, anywhere, and any scale data center services for a diverse portfolio of industries, including Content, Cloud, Networks, Gaming, Automotive, SaaS, IoT, HPC, Security, and more. With a mission predicated on taking care of our customers, our people, and our planet, EdgeConneX strives to Empower Your Edge.

Read More

Hyper-Converged Infrastructure, Storage Management

Digital Realty Announces Joint Venture of Stabilized Hyperscale Data Centers in Chicago

Digital Realty | July 19, 2023

Digital Realty (NYSE: DLR), the largest global provider of cloud- and carrier-neutral data center, colocation and interconnection solutions, announced today it has partnered with GI Partners to establish a joint venture for the sale of a 65% interest in two stabilized hyperscale data center buildings and their associated equipment in the Chicago metro area. Digital Realty will receive approximately $743 million of gross proceeds related to the joint venture and the associated financing, and will maintain a 35% interest in the joint venture while continuing to manage the day-to-day operations of the assets, providing a seamless customer experience. Digital Realty has also granted GI Partners an option to purchase an interest in the third facility on the same hyperscale data center campus. "With Digital Realty's unmatched global footprint and the attractive fundamental outlook for the data center sector, we are pleased with the strong institutional demand for our high-quality facilities," said Digital Realty Chief Investment Officer Greg Wright. "This transaction further diversifies Digital Realty's sources of capital and enhances our capital efficiency, in support of our strategic priorities. We are pleased to partner with a data center investor of GI Partners' caliber on this initial joint venture and we look forward to the continued execution on our capital plan for 2023." Digital Realty originally acquired the facilities in 2017 through its merger with DuPont Fabros. The two data centers contributed to the joint venture contain approximately 67 megawatts of IT capacity and are 90% occupied in aggregate, primarily by investment grade customers. Based on annualized in-place cash NOI at June 30, 2023 and the benefit of leases signed but not yet commenced, the transaction values the two facilities at approximately a 6.5% cap rate. Net proceeds will be used to pay down debt and for general corporate purposes. About Digital Realty Digital Realty brings companies and data together by delivering the full spectrum of data center, colocation and interconnection solutions. PlatformDIGITAL®, the company's global data center platform, provides customers with a secure data "meeting place" and a proven Pervasive Datacenter Architecture (PDx®) solution methodology for powering innovation and efficiently managing Data Gravity challenges. Digital Realty gives its customers access to the connected communities that matter to them with a global data center footprint of 300+ facilities in 50+ metros across 27 countries on six continents.

Read More