The System for AI Lab (SAIL) at Georgia Tech, led by Prof. Alexey Tumanov, specializes in AI Systems: systems, resource management, and scheduling for LLMs, reasoning models, and physical AI, both on the edge and in the cloud. We co-design systems and AI algorithms to maximize intelligence per unit cost under both static and dynamic deployment conditions. We make intelligence real-time adaptive, leading to safer, more resilient, and more intelligent AI-powered applications (e.g., safer embodied intelligence). The lab has consistently been innovating on soft real-time ML (SRTML) inference efficiency since 2019. Our research encompasses the entire AI infrastructure stack, from foundational system design to the development of efficient training and inference algorithms.
Recent News
- 🎉 RocketKV (ICML’25) Sparse Attention is now integrated in TensorRT-LLM
- Our paper on LLM training performance modeling via GPU emulation, Maya has been accepted at EuroSys’26.
- 🎉 Congratulations to Payman Behnam, Amey Agrawal, Alind Khare, and Dhruv Garg! Three papers accepted at ACM SIGOPS Operating Systems Review, July 2025.
- Our papers on common anti-patterns in LLm Inference systems evaluations is now on Arxiv.
- Prof. Tumanov recognized with the Spring 2025 CIOS Honor Roll - third semester in a row.
- Congratulations to Prof. Alexey Tumanov for being recognized with the College of Computing Outstanding Junior Faculty Research Award in the Spring of 2025!
- SuperServe accepted at NSDI’25! Paper and Video
- Client Availability in Federated Learning presented at EuroMLSys’25, co-located with EuroSys ‘25. Paper
- Our papers on Medha for efficient multi-million context LLM inference and Maya for optimizing deep learning training workloads are now public.
- Congratulations to Prof. Tumanov on being awarded tenure and promotion to the position of Associate Professor 🎉.
- RocketKV 🚀, fast long-context inference with is now KV-cache compression is now on Arxiv.
- Dhruv Garg secures CRNCH PhD Fellowship 2025 for his research in Federated Fine-tuning of LLMs.
- Mnemosyne, our paper on efficient inference up to 10M token context lengths is now public.
- Our paper on DL training checkpoint compression system, DynaQuant has been accepted at SoCC’24.
- Metron 📐 – our LLM inference system benchmark is now public.
- DεpS and SuperFedNAS have been accepted at ECCV’24.
- Congratulations to Prof. Alexey Tumanov for being awarded college of computing outstanding junior faculty teaching award.
- Sarathi-Serve ☸️, our paper on efficient LLM inference has been accepted at OSDI’24.
- Vidur 👳🏽, our paper on large scale LLM inference cluster simulation has been accepted at MLSys’24.
- Payman Behnam awarded NVIDIA Graduate Fellowship 2024 for advancing machine learning and systems with high-performance, low-latency, and energy-efficient hardware designs.
- Payman Behnam receives Qualcomm Innovation Fellowship 2023 for his work on Hardware-Software Co-Design for DNN inference systems.
- Amey Agrawal secures CRNCH PhD Fellowship 2023 for his research in LLM inference.