Nava

Role &amp; Responsibilities
<ul>
<li>Design and deploy low-latency inference pipelines for LLMs, diffusion models, and vision transformers across CPU, GPU, and NPU architectures.</li>
<li>Optimize model serving stacks using Triton Inference Server, vLLM, or TGI—tuning batch sizes, quantization, and memory layout for peak performance.</li>
<li>Containerize and orchestrate inference services via Docker and Kubernetes, ensuring high availability and auto-scaling under fluctuating workloads.</li>
<li>Implement model monitoring, health checks, and A/B testing frameworks to validate performance and drift in production.</li>
<li>Collaborate with ML Engineers to convert trained models into production-ready formats (ONNX, TensorRT, GGUF) with minimal accuracy loss.</li>
<li>Build observability dashboards (Prometheus/Grafana) and alerting logic to detect and mitigate inference bottlenecks in real time.</li>
</ul>
Skills &amp; Qualifications Must-Have
<ul>
<li>Python</li>
<li>Docker</li>
<li>Kubernetes</li>
<li>Triton Inference Server</li>
<li>ONNX Runtime</li>
<li>PyTorch</li>
<li>TensorRT</li>
<li>Prometheus</li>
<li>Grafana</li>
<li>CI/CD (GitHub Actions, GitLab CI)</li>
</ul>
Preferred
<ul>
<li>Experience with vLLM or TGI</li>
<li>Knowledge of Model Quantization (AWQ, GPTQ, GGUF)</li>
<li>Familiarity with NVIDIA Triton model ensemble pipelines</li>
</ul>