Wuwei

Wuwei offers an automated compiler framework that optimizes quantized deep learning models for CUDA architectures. It generates high-performance kernels using TVM and hardware-specific instructions, reducing manual engineering effort for efficient GPU deployment.

Updated 2 months ago

Funding

Funding not disclosed

Funding rounds are not available yet.

Founders

Founder details are not available yet.

Product

Problem

Deploying deep learning models, particularly quantized versions, on diverse hardware requires specialized optimization for specific compute architectures. Manually optimizing these models for platforms like CUDA involves significant engineering effort and expertise in low-level kernel development, making efficient deployment challenging and time-consuming.

Solution

Wuwei provides an automated compiler framework that optimizes quantized deep learning models for CUDA architectures. The system leverages TVM's compiler stack and machine learning-based optimization techniques to generate high-performance kernels. It supports expressing quantized CUDA operations using tensorization, enabling the use of hardware-specific instructions like `dp4a` for efficient computation. The framework also handles data layout transformations, such as `NCHW4c` and `OIHW4o4i`, to maximize memory locality and compatibility with intrinsic operations. This automated approach significantly reduces the manual effort required for optimizing quantized models, allowing for faster inference and efficient deployment on NVIDIA GPUs.

Target Audience

The primary users are machine learning engineers and researchers focused on deploying deep learning models, particularly those requiring high-performance inference on NVIDIA GPUs, such as those working with autonomous vehicles or real-time AI applications.

Features

Automated optimization of quantized deep learning models for CUDA.
Utilization of TVM's compiler stack for end-to-end optimization.
Tensorization support for expressing and optimizing quantized CUDA kernels using hardware intrinsics (e.g., `dp4a`).
Custom data layout transformations (`NCHW4c`, `OIHW4o4i`) for optimized memory access patterns.
Machine learning-based search for optimal schedule configurations.
Support for importing models from frameworks like MXNet via Relay IR.
Generation of deployable libraries for inference on GPUs.
Performance benchmarking against state-of-the-art solutions like TensorRT and cuDNN.