Senior/Staff Software Engineer - LLM Inference Systems & Open-Source

Technology, Data & Digital · Data, AI & Analytics · Machine Learning · Software Engineering · DevOps

In short

Qualcomm seeks experienced software engineers to build and advance LLM inference on their platforms, focusing on open-source LLM serving systems, distributed systems, and AI accelerator software. You will contribute to open-source projects and develop efficient, reliable inference infrastructure for Qualcomm AI accelerators.

Responsibilities

Contribute features, improvements, bug fixes, and architectural enhancements to open-source LLM inference projects, including llm-d and other LLM inference and serving projects.
Develop and improve support for Qualcomm AI accelerators in open-source LLM serving frameworks.
Work with upstream maintainers and open-source communities to design, review, and upstream Qualcomm-related functionality.
Develop and enhance inference engine integrations for Qualcomm platforms, including device abstraction, accelerator runtime integration, memory management, distributed execution, compilation, and performance optimization.
Develop and improve llm-d integration and deployment capabilities for Qualcomm platforms, including reference architectures, deployment blueprints, configuration, and operational tooling.
Design and implement distributed LLM inference capabilities such as prefill/decode disaggregation, KV-cache management and transfer, distributed communication, request routing, model parallelism, multi-node inference, autoscaling, and failure recovery.
Integrate LLM inference software with Kubernetes and cloud-native infrastructure.
Profile, benchmark, and optimize end-to-end inference performance, including time to first token (TTFT), inter-token latency (ITL), throughput, memory utilization, and scalability.
Diagnose complex issues spanning model-serving frameworks, distributed runtimes, operating systems, networking, accelerator runtimes, and hardware.
Collaborate with hardware, firmware, compiler, runtime, Kubernetes, and product teams.
Develop technical documentation, examples, reference deployments, and best practices.
Participate in architecture discussions and influence Qualcomm's open-source LLM inference software strategy.

Requirements

Bachelor's degree in Computer Science, Engineering, Information Systems, or related field and 4+ years of Hardware Engineering, Software Engineering, Systems Engineering, or related work experience.
OR Master's degree in Computer Science, Engineering, Information Systems, or related field and 3+ years of Hardware Engineering, Software Engineering, Systems Engineering, or related work experience.
OR PhD in Computer Science, Engineering, Information Systems, or related field and 2+ years of Hardware Engineering, Software Engineering, Systems Engineering, or related work experience.
Strong software engineering and debugging skills.
Experience developing production-quality software in one or more of Python, C++, Go, or Rust.
Strong understanding of at least one of the following areas: Machine learning or LLM inference systems, Distributed systems, AI accelerator or GPU software, Cloud-native/Kubernetes infrastructure, ML compiler or runtime systems.
Experience working with complex software systems and debugging problems across multiple layers of the stack.
Ability to collaborate effectively across teams and communicate technical concepts clearly.

Desired Qualifications

Experience in LLM inference engines, model-serving frameworks, or distributed inference systems.
Architecture and implementation of high-performance LLM inference runtimes, including model execution, scheduling, memory management, attention, and KV-cache management.
Distributed LLM inference architectures, including prefill/decode disaggregation and KV-cache transfer.
llm-d, KServe, Kubernetes Gateway API, Kubernetes operators, or cloud-native AI infrastructure.
PyTorch internals, torch.compile, Dynamo, Inductor, Triton, MLIR, or other compiler/runtime technologies.
Hardware accelerator software stacks, GPU programming, or custom AI accelerator integration.
RDMA, RoCE, GPU Direct-style communication, PCIe, high-speed networking, or distributed communication libraries.
Model parallelism, including tensor parallelism, pipeline parallelism, expert parallelism, and data parallelism.
Performance profiling and optimization of distributed AI workloads.
Kubernetes scheduling, autoscaling, networking, observability, and production operations.
Open-source development, including contributing patches, participating in design discussions, reviewing code, and collaborating with upstream maintainers.
Experience working across hardware/software boundaries with hardware, firmware, compiler, runtime, and systems teams.

Benefits

Competitive annual discretionary bonus program
Opportunity for annual RSU grants
Highly competitive benefits package designed to support your success at work, at home, and at play.
Access to details about US benefits via a provided link.

Skills

PythonC++GoRustKubernetesLLMMachine LearningOpen-source

#LLM#Inference#Open-source#Software Engineering#Machine Learning#Distributed Systems#Cloud-native#AI Accelerators

About Qualcomm

Enabling a world where everyone and everything can be intelligently connected.

Enabling a world where everyone and everything can be intelligently connected.\n\n

See all open jobs at Qualcomm Read more about Qualcomm Website

Company

Qualcomm

Job Posted

1 week ago

Employment Type

Full Time

Work mode

On Site

Salary

US$162,000–243,000 · Negotiable

Experience Level

Senior

Locations

Santa Clara, United States of America

Qualification

Bachelor, Master, Doctoral

Applicants

Be an early applicant

Similar Jobs

Ericsson

Budapest, Hungary

Senior/Staff Software Engineer - LLM Inference Systems & Open-Source

Responsibilities

Requirements

Desired Qualifications

Benefits

Skills

About Qualcomm

Similar Jobs

Student Data Engineer

Software Designer

Senior Electrical Engineer

Principal Corporate Counsel

Utilities Strategy Manager

Junior Software Developer