Experience optimizing inference performance for both latency and throughput on such large models across the stack from system level optimizations through to Pytorch or JAX is a must have.
Bachelor's degree in computer science or equivalent
3+ years of non-internship professional software development experience
3+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
Fundamentals of Machine learning and LLMs, their architecture, training and inference lifecycles along with work experience on some optimizations for improving the model execution.
Software development experience in C++, Python (experience in at least one language is required).
Strong understanding of system performance, memory management, and parallel computing principles.
Proficiency in debugging, profiling, and implementing best software engineering practices in large-scale systems.