Nava

Role &amp; Responsibilities
<ul>
<li>Optimize LLM inference pipelines for latency, throughput, and memory efficiency across GPU/TPU hardware—using quantization, pruning, kernel fusion, and runtime scheduling.</li>
<li>Implement and benchmark model compression techniques (INT4/INT8/FP8, LoRA, Sparse attention) to reduce model size while preserving performance.</li>
<li>Integrate and tune LLM serving frameworks (vLLM, TensorRT-LLM, HuggingFace TGI, ONNX Runtime) for high-throughput, low-latency inference in cloud and on-premise environments.</li>
<li>Collaborate with ML researchers and infrastructure engineers to profile bottlenecks and implement hardware-aware optimizations using CUDA, Triton, or custom kernels.</li>
<li>Design and automate model benchmarking suites to track performance regression, memory footprint, and cost-per-token across model versions and hardware targets.</li>
<li>Document optimization playbooks and contribute to internal tooling for repeatable, scalable model deployment workflows.</li>
</ul>
Skills &amp; Qualifications
Must-Have
<ul>
<li>PyTorch</li>
<li>TensorRT</li>
<li>vLLM</li>
<li>Quantization (INT4/INT8/FP8)</li>
<li>CUDA</li>
<li>ONNX Runtime</li>
<li>Triton Inference Server</li>
<li>LLM Inference Optimization</li>
</ul>
Preferred
<ul>
<li>Experience with Mixture-of-Experts (MoE) models</li>
<li>Familiarity with HuggingFace Transformers and TGI</li>
<li>Knowledge of NPU/ASIC inference backends (e.g., Qualcomm, Groq, Cerebras)</li>
</ul>