APPLICATION INFRASTRUCTURE

CEVA Redefines High Performance AI/ML Processing for Edge AI and Edge Compute Devices

CEVA | January 06, 2022

CEVA Redefines High Performance AI/ML Processing for Edge AI and Edge Compute Devices
Consumer Electronics Show  – CEVA, Inc.the leading licensor of wireless connectivity and smart sensing technologies and integrated IP solutions, today announced NeuPro-M, its latest generation processor architecture for artificial intelligence and machine learning (AI/ML) inference workloads. Targeting the broad markets of Edge AI and Edge Compute, NeuPro-M is a self-contained heterogeneous architecture that is composed of multiple specialized co-processors and configurable hardware accelerators that seamlessly and simultaneously process diverse workloads of Deep Neural Networks, boosting performance by 5-15X compared to its predecessor. An industry first, NeuPro-M supports both system-on-chip (SoC) as well as Heterogeneous SoC (HSoC) scalability to achieve up to 1,200 TOPS and offers optional robust secure boot and end-to-end data privacy.

NeuPro-M is the latest generation processor architecture from CEVA for artificial intelligence and machine learning (AI/ML) inference workloads. Targeting the broad markets of Edge AI and Edge Compute, NeuPro-M is a self-contained heterogeneous architecture that is composed of multiple specialized co-processors and configurable hardware accelerators that seamlessly and simultaneously process diverse workloads of Deep Neural Networks, boosting performance by 5-15X compared to its predecessor.
NeuPro-M is the latest generation processor architecture from CEVA for artificial intelligence and machine learning (AI/ML) inference workloads. Targeting the broad markets of Edge AI and Edge Compute, NeuPro-M is a self-contained heterogeneous architecture that is composed of multiple specialized co-processors and configurable hardware accelerators that seamlessly and simultaneously process diverse workloads of Deep Neural Networks, boosting performance by 5-15X compared to its predecessor.
NeuPro–M compliant processors initially include the following pre-configured cores:

  • NPM11 – single NeuPro-M engine, up to 20 TOPS at 1.25GHz
  • NPM18 – eight NeuPro-M engines, up to 160 TOPS at 1.25GHz

Illustrating its leading-edge performance, a single NPM11 core, when processing a ResNet50 convolutional neural network, achieves a 5X performance increase and 6X memory bandwidth reduction versus its predecessor, which results in exceptional power efficiency of up to 24 TOPS per watt.

Built on the success of its' predecessors, NeuPro-M is capable of processing all known neural network architectures, as well as integrated native support for next-generation networks like transformers, 3D convolution, self-attention and all types of recurrent neural networks. NeuPro-M has been optimized to process more than 250 neural networks, more than 450 AI kernels and more than 50 algorithms. The embedded vector processing unit (VPU) ensures future proof software-based support of new neural network topologies and new advances in AI workloads. Furthermore, the CDNN offline compression tool can increase the FPS/Watt of the NeuPro-M by a factor of 5-10X for common benchmarks, with very minimal impact on accuracy.

"The artificial intelligence and machine learning processing requirements of edge AI and edge compute are growing at an incredible rate, as more and more data is generated and sensor-related software workloads continue to migrate to neural networks for better performance and efficiencies. With the power budget remaining the same for these devices, we need to find new and innovative methods of utilizing AI at the edge in these increasingly sophisticated systems. NeuPro-M is designed on the back of our extensive experience deploying AI processors and accelerators in millions of devices, from drones to security cameras, smartphones and automotive systems. Its innovative, distributed architecture and shared memory system controllers reduces bandwidth and latency to an absolute minimum and provides superb overall utilization and power efficiency. With the ability to connect multiple NeuPro-M compliant cores in a SoC or Chiplet to address the most demanding AI workloads, our customers can take their smart edge processor designs to the next level."

Ran Snir, Vice President and General Manager of the Vision Business Unit at CEVA

The NeuPro-M heterogenic architecture is composed of function-specific co-processors and load balancing mechanisms that are the main contributors to the huge leap in performance and efficiency compared to its predecessor. By distributing control functions to local controllers and implementing local memory resources in a hierarchical manner, the NeuPro-M achieves data flow flexibility that result in more than 90% utilization and protects against data starvation of the different co-processors and accelerators at any given time. The optimal load balancing is obtained by practicing various data flow schemes that are adopted to the specific network, the desired bandwidth, the available memory and the target performance, by the CDNN framework.

NeuPro-M architecture highlights include:

  • Main grid array consisting of 4K MACs (Multiply And Accumulates), with mixed precision of 2-16 bits
  • Winograd transform engine for weights and activations, reducing convolution time by 2X and allowing 8-bit convolution processing with <0.5% precision degradation
  • Sparsity engine to avoid operations with zero-value weights or activations per layer, for up to 4X performance gain, while reducing memory bandwidth and power consumption
  • Fully programmable Vector Processing Unit, for handling new unsupported neural network architectures with all data types, from 32-bit Floating Point down to 2-bit Binary Neural Networks (BNN)
  • Configurable Weight and Data compression down to 2-bits while storing to memory, and real-time decompression upon reading, for reduced memory bandwidth
  • Dynamically configured two level memory architecture to minimize power consumption attributed to data transfers to and from an external SDRAM

To illustrate the benefit of these innovative features in the NeuPro-M architecture, concurrent use of the orthogonal mechanisms of Winograd transform, Sparsity engine, and low-resolution 4x4-bit activations, delivers more than a 3X reduction in cycle count of networks such as Resnet50 and Yolo V3.

As neural network Weights and Biases and the data set and network topology become key Intellectual Property of the owner, there is a strong need to protect these from unauthorized use. The NeuPro-M architecture supports secure access in the form of optional root of trust, authentication, and cryptographic accelerators.

For the automotive market, NeuPro-M cores and its CEVA Deep Neural Network (CDNN) deep learning compiler and software toolkit comply with Automotive ISO26262 ASIL-B functional safety standard and meets the stringent quality assurance standards IATF16949 and A-Spice.

Together with CEVA's multi award-winning neural network compiler – CDNN – and its robust software development environment, NeuPro-M provides a fully programmable hardware/software AI development environment for customers to maximize their AI performance. CDNN includes innovative software that can fully utilize the customers' NeuPro-M customized hardware to optimize power, performance & bandwidth. The CDNN software also includes a memory manager for memory reduction and optimal load balancing algorithms, and wide support of various network formats including ONNX, Caffe, TensorFlow, TensorFlow Lite, Pytorch and more. CDNN is compatible with common open-source frameworks, including Glow, tvm, Halide and TensorFlow and includes model optimization features like 'layer fusion' and 'post training quantization' all while using precision conservation methods.

NeuPro-M is available for licensing to lead customers today and for general licensing in Q2 this year. NeuPro-M customers can also benefit from Heterogenous SoC design services from CEVA to help integrate and support system design and chiplet development. 


About CEVA, Inc.
CEVA is the leading licensor of wireless connectivity and smart sensing technologies and integrated IP solutions for a smarter, safer, connected world. We provide Digital Signal Processors, AI engines, wireless platforms, cryptography cores and complementary software for sensor fusion, image enhancement, computer vision, voice input and artificial intelligence. These technologies are offered in combination with our Intrinsix IP integration services, helping our customers address their most complex and time-critical integrated circuit design projects. Leveraging our technologies and chip design skills, many of the world's leading semiconductors, system companies and OEMs create power-efficient, intelligent, secure and connected devices for a range of end markets, including mobile, consumer, automotive, robotics, industrial, aerospace & defense and IoT.

Our DSP-based solutions include platforms for 5G baseband processing in mobile, IoT and infrastructure, advanced imaging and computer vision for any camera-enabled device, audio/voice/speech and ultra-low-power always-on/sensing applications for multiple IoT markets. For sensor fusion, our Hillcrest Labs sensor processing technologies provide a broad range of sensor fusion software and inertial measurement unit ("IMU") solutions for markets including hearables, wearables, AR/VR, PC, robotics, remote controls and IoT. For wireless IoT, our platforms for Bluetooth (low energy and dual mode), Wi-Fi 4/5/6/6e (802.11n/ac/ax), Ultra-wideband (UWB), NB-IoT and GNSS are the most broadly licensed connectivity platforms in the industry.

Spotlight

Oracle provides enterprise customers with a one-stop shop for IaaS and PaaS by adding Oraclecloud.net’s best-in-class DNS solution to the Oracle cloud computing platform. This video details the features and functions of Oracle Cloud Infrastructure DNS. You will learn about the supported record types and the complete set of functions for zone management that the service offers right from the UI.

Related News

APPLICATION INFRASTRUCTURE

'AWS for Web3' Pocket Network Shakes Up Infrastructure of the Internet

Pocket Network | January 17, 2022

Pocket Network an infrastructure middleware protocol which facilitates decentralized cloud computing and abundant bandwidth on full nodes interoperable with DApps across all 21+ blockchains, doubles down on its market-based approach to infrastructure by incentivizing a global community of independent node operators and service providers running 21,000+ nodes adding to the network's resilience. Pocket Network is secured by over $320M worth of network infrastructure distributed globally across 23 countries and reduces the risk of service downtime to near zero for any layer 1 or industry DApps as work is distributed evenly across thousands of full nodes, which also protects end users' privacy. Pocket Network, who last week closed a strategic round led by Republic Capital, RockTree Capital, Arrington Capital, is undergoing exponential growth as demand for its network measured in API calls or relays jumped to 5.78 Billion relays in December, up 44% From November 2021. "Pocket is the First Mover, changing the game as core infrastructure for the Web3 revolution" said Omer Ozden, CEO of RockTree Capital "For those that are seeking the next high growth first mover like AWS or Ethereum, you can see transparently on-chain the parabolic growth of actual usage and of actual revenues of the Pocket Network. The metrics have been super impressive and we look forward to the scaling 2022 will bring." Pocket Network has generated $56m revenue in December up from $29m in November and currently supports over 2000+ DApps run across its network which is integrated with Solana along with Ethereum, Polygon, Avalanche, Binance, Harmony, Gnosis (formerly xDai), Fuse and many more. "There is increased investment in public, open-source infrastructure through crypto networks that continues to chip away at the dominance of today's providers, reducing them to commodities,Our attractive economic flywheel model for our network is simple and naturally drives growth - attractive node running incentives adds more nodes which in turn creates more redundancy and better DApp service experience, more DApp usage then generates more node revenue." Michael O'Rourke, CEO at Pocket Network Pocket meaningfully encourages and incentivizes niche cloud providers which commercially have been less viable over the last decade with massively well known incumbents dominating the traditional market, and aims to provide Web3 with a much more compatible infrastructure network. About Pocket Network Pocket Network, a blockchain data ecosystem for Web3 applications, is a platform built for applications that uses cost-efficient economics to coordinate and distribute data at scale. It enables seamless and secure interactions between blockchains and across applications. With Pocket, the use of blockchains can be simply integrated into web..sites, mobile apps, IoT and more, giving developers the freedom to put blockchain enabled applications into the "pocket" of every mainstream consumer.

Read More

Multi-Vendor Infrastructures Are Easier Than Ever to Manage

Informationweek | April 24, 2020

Limiting the number of vendors used in an IT infrastructure has been common tactic for decades. This strategy has proven time and again to save time and money when it comes to end-to-end implementation, troubleshooting, monitoring and interoperation. Yet, the single vendor approach may no longer be the right method to take these days. In fact, many of the previously overwhelming factors against multi-vendor infrastructures can now be addressed.

Read More

APPLICATION INFRASTRUCTURE

For its Groundbreaking 5 G Buildout, DISH chooses Intel as a Technology Partner

prnewswire | November 02, 2020

Today, DISH and Intel reported an organization to incorporate Intel 5G framework innovation into DISH's buildout of the country's first virtualized, open Radio Access Network (O-RAN) 5G network arrangement. the Intel vRAN Dedicated Accelerator ACC100 and Intel's FlexRAN programming reference engineering for its arrangements. With a greenfield 5G network that is being architected starting from the earliest stage, DISH is working with a few U.S.- based sellers to consolidate cloud and availability innovation to control cutting edge administrations and endeavor use cases. As the world's driving organization silicon supplier, Intel brings a full set-up of silicon, programming and apparatuses to change the organization with a product characterized, deft and versatile framework. Through close coordinated effort with Intel, DISH can solidify and upgrade numerous remaining tasks at hand at areas over its organization for expanded productivity. "Intel has been a trusted advisor throughout the design of our O-RAN network, working in concert with our software vendors Mavenir, Altiostar, and many OEM hardware providers. We have tested several commercial off-the-shelf (COTS) designs from a large number of server vendors using Intel's O-RAN compliant FlexRAN architecture and are pleased by the maturity and power of the solutions, together with the cost benefits of COTS solutions," said Marc Rouanne, executive vice president and chief network officer, DISH. "We are using the power of the VMware abstraction solution and the ubiquity of Intel-based servers to load and mix different types of cloud-native workloads like distributed unit (DU), centralized unit (CU), virtual routers, mobile edge computing applications, and 5G Core containerized network functions." "Fully-virtualized, cloud-native networks like the one DISH is building bring the same server economics that transformed the data center," said Dan Rodriguez, Intel corporate vice president and general manager of the Network Platforms Group. "We are excited to partner with DISH to lay the foundation for a truly agile network and have already begun working with our OEM partners who have designed FlexRAN-based servers to enable a variety of new innovative use cases and services." DISH and Intel are cooperating on the completely virtualized RAN, including radio reference plans, fronthaul improvement, equipment based security, and outlines for workers. The two organizations are likewise working together to empower edge applications for ventures and driving O-RAN principles, and are participating in the territories of information streamlining and Machine Learning for future periods of the organization buildout. About DISH DISH Network Corporation is a connectivity company. Since 1980, it has served as a disruptive force, driving innovation and value on behalf of consumers. Through its subsidiaries, the company provides television entertainment and award-winning technology to millions of customers with its satellite DISH TV and streaming SLING TV services. In 2020, the company became a nationwide U.S. wireless carrier through the acquisition of Boost Mobile. DISH continues to innovate in wireless, building the nation's first cloud-native, Open RAN-based 5G broadband network. About Intel Intel is an industry leader, creating world-changing technology that enables global progress and enriches lives. Inspired by Moore's Law, we continuously work to advance the design and manufacturing of semiconductors to help address our customers' greatest challenges. By embedding intelligence in the cloud, network, edge and every kind of computing device, we unleash the potential of data to transform business and society for the better. Intel Corporation, Intel, the Intel logo and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.

Read More

Spotlight

Oracle provides enterprise customers with a one-stop shop for IaaS and PaaS by adding Oraclecloud.net’s best-in-class DNS solution to the Oracle cloud computing platform. This video details the features and functions of Oracle Cloud Infrastructure DNS. You will learn about the supported record types and the complete set of functions for zone management that the service offers right from the UI.