GitHub

NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference

Online LLM inference powers many exciting applications such as intelligent chatbots and autonomous agents. Modern LLM inference engines widely rely on request batching to improve inference throughput, ...
IEEE

Enabling High-Throughput Inference of Transformers on Near-Data Processing Architectures

Abstract: The rapid adoption of Transformer models in AI has exposed critical inefficiencies in conventional computing architectures, particularly due to their large memory footprint and low data ...
GitHub

Scalable Neural Vocoder from Range-Null Space Decomposition

Although deep neural networks have facilitated significant progress of neural vocoders in recent years, they usually suffer from intrinsic challenges like opaque modeling, inflexible retraining under ...

Nvidia: OpenAI's AGI Admission Should Send Shivers

NVIDIA Corporation is a strong sell with a $27 price target by the end of 2027. Click here to read the latest analysis on ...
IEEE

Design and Implementation of Lightweight Neural Network Inference Accelerator Based on FPGA

Abstract: This study focuses on the design and optimization of lightweight neural network inference accelerator based on FPGA, and proposes an efficient accelerator architecture suitable for ...