Neural Magic is on a mission to bring the power of open-source LLMs and vLLM to every enterprise on the planet. The Future of AI is Open.

vLLM Office Hours - Speculative Decoding in vLLM - October 3, 2024
1:04:28
vLLM Office Hours - Speculative Decoding in vLLM - October 3, 2024
vLLM Office Hours - Advanced Techniques for Maximizing vLLM Performance - September 19, 2024
52:35
vLLM Office Hours - Advanced Techniques for Maximizing vLLM Performance - September 19, 2024
vLLM Office Hours - Using NVIDIA CUTLASS for High-Performance Inference - September 05, 2024
1:13:14
vLLM Office Hours - Using NVIDIA CUTLASS for High-Performance Inference - September 05, 2024
vLLM Office Hours - vLLM on AMD GPUs and Google TPUs - August 21, 2024
48:13
vLLM Office Hours - vLLM on AMD GPUs and Google TPUs - August 21, 2024
vLLM Office Hours - Multimodal Models in vLLM with Roblox - August 8, 2024
50:03
vLLM Office Hours - Multimodal Models in vLLM with Roblox - August 8, 2024
vLLM Office Hours - Model Quantization for Efficient vLLM Inference - July 25, 2024
50:37
vLLM Office Hours - Model Quantization for Efficient vLLM Inference - July 25, 2024
Deploy LLMs More Efficiently with vLLM and Neural Magic
33:21
Deploy LLMs More Efficiently with vLLM and Neural Magic
vLLM Office Hours - FP8 Quantization Deep Dive - July 9, 2024
56:09
vLLM Office Hours - FP8 Quantization Deep Dive - July 9, 2024
vLLM Office Hours - June 20, 2024
53:19
vLLM Office Hours - June 20, 2024
vLLM and Neural Magic Office Hours - June 5, 2024
44:47
vLLM and Neural Magic Office Hours - June 5, 2024
Are MLOps disappearing?
6:31
Are MLOps disappearing?
5x Faster YOLOv8 on CPUs
1:06
5x Faster YOLOv8 on CPUs
Deploy Fast and Accurate YOLOv8 Object Detection Models on CPUs You Already Have
47:52
Deploy Fast and Accurate YOLOv8 Object Detection Models on CPUs You Already Have
Unlock Faster and More Efficient LLMs with SparseGPT
42:27
Unlock Faster and More Efficient LLMs with SparseGPT
Pruning and Quantizing ML Models With One Shot Without Retraining
52:31
Pruning and Quantizing ML Models With One Shot Without Retraining
Sparse Transferring Hugging Face Models With SparseML
8:15
Sparse Transferring Hugging Face Models With SparseML
Apply Second-Order Pruning Algorithms for SOTA Model Compression
41:42
Apply Second-Order Pruning Algorithms for SOTA Model Compression
Use Sparse Transfer Learning to Create Sparse Models Fine-Tuned to Your Datasets
6:53
Use Sparse Transfer Learning to Create Sparse Models Fine-Tuned to Your Datasets
Intro to SparseML
5:02
Intro to SparseML
Accelerate Image Segmentation Tasks With Sparsity and the DeepSparse Runtime
4:23
Accelerate Image Segmentation Tasks With Sparsity and the DeepSparse Runtime
Accelerate Image Classification Tasks With Sparsity and the DeepSparse Runtime
4:20
Accelerate Image Classification Tasks With Sparsity and the DeepSparse Runtime
Accelerate Object Detection Tasks With Sparsity and the DeepSparse Runtime
4:50
Accelerate Object Detection Tasks With Sparsity and the DeepSparse Runtime
Intro to DeepSparse Runtime
7:38
Intro to DeepSparse Runtime
Intro to Deep Learning Model Sparsification
7:08
Intro to Deep Learning Model Sparsification
Intro to SparseZoo
6:16
Intro to SparseZoo
Intro to Neural Magic & Software-Delivered AI
9:56
Intro to Neural Magic & Software-Delivered AI
Accelerate NLP Tasks With Sparsity and the DeepSparse Runtime
4:51
Accelerate NLP Tasks With Sparsity and the DeepSparse Runtime
Sparse Training of Neural Networks Using AC/DC
41:38
Sparse Training of Neural Networks Using AC/DC
How Well Do Sparse Models Transfer?
50:27
How Well Do Sparse Models Transfer?
How to Achieve the Fastest CPU Inference Performance for Object Detection YOLO Models
59:10
How to Achieve the Fastest CPU Inference Performance for Object Detection YOLO Models