Senior/Staff Software Engineer - LLM Inference Systems & Open-Source
Technology, Data & Digital · Data, AI & Analytics · Machine Learning · Software Engineering · DevOps
In short
Qualcomm seeks experienced software engineers to build and advance LLM inference on their platforms, focusing on open-source LLM serving systems, distributed systems, and AI accelerator software. You will contribute to open-source projects and develop efficient, reliable inference infrastructure for Qualcomm AI accelerators.
Responsibilities
- Contribute features, improvements, bug fixes, and architectural enhancements to open-source LLM inference projects, including llm-d and other LLM inference and serving projects.
- Develop and improve support for Qualcomm AI accelerators in open-source LLM serving frameworks.
- Work with upstream maintainers and open-source communities to design, review, and upstream Qualcomm-related functionality.
- Develop and enhance inference engine integrations for Qualcomm platforms, including device abstraction, accelerator runtime integration, memory management, distributed execution, compilation, and performance optimization.
- Develop and improve llm-d integration and deployment capabilities for Qualcomm platforms, including reference architectures, deployment blueprints, configuration, and operational tooling.
- Design and implement distributed LLM inference capabilities such as prefill/decode disaggregation, KV-cache management and transfer, distributed communication, request routing, model parallelism, multi-node inference, autoscaling, and failure recovery.
- Integrate LLM inference software with Kubernetes and cloud-native infrastructure.
- Profile, benchmark, and optimize end-to-end inference performance, including time to first token (TTFT), inter-token latency (ITL), throughput, memory utilization, and scalability.
- Diagnose complex issues spanning model-serving frameworks, distributed runtimes, operating systems, networking, accelerator runtimes, and hardware.
- Collaborate with hardware, firmware, compiler, runtime, Kubernetes, and product teams.
- Develop technical documentation, examples, reference deployments, and best practices.
- Participate in architecture discussions and influence Qualcomm's open-source LLM inference software strategy.
Requirements
- Bachelor's degree in Computer Science, Engineering, Information Systems, or related field and 4+ years of Hardware Engineering, Software Engineering, Systems Engineering, or related work experience.
- OR Master's degree in Computer Science, Engineering, Information Systems, or related field and 3+ years of Hardware Engineering, Software Engineering, Systems Engineering, or related work experience.
- OR PhD in Computer Science, Engineering, Information Systems, or related field and 2+ years of Hardware Engineering, Software Engineering, Systems Engineering, or related work experience.
- Strong software engineering and debugging skills.
- Experience developing production-quality software in one or more of Python, C++, Go, or Rust.
- Strong understanding of at least one of the following areas: Machine learning or LLM inference systems, Distributed systems, AI accelerator or GPU software, Cloud-native/Kubernetes infrastructure, ML compiler or runtime systems.
- Experience working with complex software systems and debugging problems across multiple layers of the stack.
- Ability to collaborate effectively across teams and communicate technical concepts clearly.
Desired Qualifications
- Experience in LLM inference engines, model-serving frameworks, or distributed inference systems.
- Architecture and implementation of high-performance LLM inference runtimes, including model execution, scheduling, memory management, attention, and KV-cache management.
- Distributed LLM inference architectures, including prefill/decode disaggregation and KV-cache transfer.
- llm-d, KServe, Kubernetes Gateway API, Kubernetes operators, or cloud-native AI infrastructure.
- PyTorch internals, torch.compile, Dynamo, Inductor, Triton, MLIR, or other compiler/runtime technologies.
- Hardware accelerator software stacks, GPU programming, or custom AI accelerator integration.
- RDMA, RoCE, GPU Direct-style communication, PCIe, high-speed networking, or distributed communication libraries.
- Model parallelism, including tensor parallelism, pipeline parallelism, expert parallelism, and data parallelism.
- Performance profiling and optimization of distributed AI workloads.
- Kubernetes scheduling, autoscaling, networking, observability, and production operations.
- Open-source development, including contributing patches, participating in design discussions, reviewing code, and collaborating with upstream maintainers.
- Experience working across hardware/software boundaries with hardware, firmware, compiler, runtime, and systems teams.
Benefits
- Competitive annual discretionary bonus program
- Opportunity for annual RSU grants
- Highly competitive benefits package designed to support your success at work, at home, and at play.
- Access to details about US benefits via a provided link.
Skills
PythonC++GoRustKubernetesLLMMachine LearningOpen-source
#LLM#Inference#Open-source#Software Engineering#Machine Learning#Distributed Systems#Cloud-native#AI Accelerators
About Qualcomm
Enabling a world where everyone and everything can be intelligently connected.
Enabling a world where everyone and everything can be intelligently connected.\n\n
Company
QualcommJob Posted
1 week ago
Employment Type
Full Time
Work mode
On Site
Salary
US$162,000–243,000 · Negotiable
Experience Level
Senior
Locations
Santa Clara, United States of America
Qualification
Bachelor, Master, Doctoral
Applicants
Be an early applicant
Similar Jobs
Qualcomm
Singapore, SingaporeManufacturing Specialist (Operations)
Full Time Be an early applicant Posted 2 days ago
Microsoft
Singapore, SingaporePartner Development Manager
Full Time Be an early applicant Posted 14 hours ago
Microsoft
Asia +1 moreCritical Infrastructure Network Engineer
Full Time Be an early applicant Posted 1 week ago
Qualcomm
Singapore, SingaporeSenior Wet Process Engineer
Full Time Be an early applicant Posted 1 week ago
ABB
Singapore, SingaporeSales Manager - Low Voltage Systems
Full Time Be an early applicant Posted 1 week ago
Emerson
Singapore, SingaporeSenior / Manufacturing Engineer
Full Time Be an early applicant Posted 1 week ago