Edge Machine Learning and Inference Optimization

Students will learn to design, optimize, and deploy machine learning models for low-power microcontrollers and embedded systems. The curriculum encompasses model quantization, pruning, efficient model conversion, and leveraging hardware accelerators, with a focus on real-world constraints like memory and latency budgets. Practical skills in TFLite Micro and on-device inference pipelines will be developed.

Course outline

Lectures, virtual labs, and graded assignments — completed in your browser.

01Introduction to Edge ML and the TinyML Landscapelecture

02Embedded Systems Foundations and Resource Limitationslecture

03Deep Learning for Resource-Constrained Environmentslecture

04Post-Training Quantization for Model Compressionlecture

05Quantization-Aware Training and Advanced Quantizationlecture

06Neural Network Pruning and Sparsity Methodslecture

07Model Compression via Pruning and Knowledge Distillationlecture

08TensorFlow Lite: Model Conversion and Optimizationlecture

09TFLite Micro for Microcontrollers: Deep Divelecture

10Designing Efficient On-Device Inference Pipelineslecture

11Hardware Accelerators: NPUs and Custom ASICs for Edge AIlecture

12Performance Engineering: Latency and Memory Budgetinglecture

13Power Optimization and Energy-Efficient Edge AIlecture

14Edge ML Project Showcase and Future Directionsassignment

Syllabus

Week 1: Introduction to Edge Machine Learning and TinyML Ecosystem
Week 2: Fundamentals of Embedded Systems and Resource Constraints
Week 3: Deep Learning Basics for Edge Devices and Model Architectures
Week 4: Model Quantization Theory and Techniques (Post-Training)
Week 5: Quantization-Aware Training and Mixed-Precision Quantization
Week 6: Model Pruning Techniques: Sparsity, Structured Pruning, and Compression
Week 7: Neural Network Compression and Knowledge Distillation
Week 8: Introduction to TensorFlow Lite and Model Conversion for Edge
Week 9: TFLite Micro: Architectures, Deployment, and Optimizations
Week 10: On-Device Inference Pipeline Design and Optimization
Week 11: NPU and Hardware Accelerator Architectures for Edge ML
Week 12: Latency and Memory Budgeting for Real-time Edge Applications
Week 13: Power Efficiency and Energy Optimization in Edge AI Deployments
Week 14: Project Presentations and Advanced Topics in Edge ML Research