Software Engineer

Technology, Data & Digital · Software & Web Development · Software Engineering · Machine Learning

In short

As a Senior or Principal Software Engineer on the Microsoft AI Frameworks team, you will optimize the software and systems that enable state-of-the-art AI models to run efficiently at cloud scale. This role focuses on improving the performance of Large Language Model (LLM) serving stacks through GPU kernel and distributed inference system optimizations.

Responsibilities

Develop and optimize GPU kernels and runtime components for state-of-the-art LLM serving performance.
Profile workloads and improve latency, throughput, and hardware efficiency.
Integrate robust, maintainable optimizations into production serving systems.
Collaborate across model, compiler, infrastructure, and hardware teams.
Use AI-assisted development tools effectively to accelerate the development process.
Contribute to technical reviews, engineering standards, and mentoring.
Drive cross-stack optimization across kernels, compilers, runtimes, and distributed serving.
Perform difficult, low-level optimization and resolve critical performance bottlenecks.
Turn performance innovations into reusable production capabilities.
Collaborate across research, infrastructure, and hardware teams to deliver measurable improvements.
Mentor senior engineers and raise engineering standards.

Requirements

Bachelor’s Degree in Computer Science or a related technical field and 4+ years of technical engineering experience coding in languages such as C++, or Python, or equivalent experience.
Ability to meet Microsoft, customer and/or government security screening requirements.
Expertise in GPU or equivalent accelerator programming and kernel optimization.
Familiarity with vLLM, SGLang, or equivalent LLM inference libraries.
Demonstrated cross-team collaboration and technical ownership.
Ability to use AI-assisted development tools efficiently and validate their output.
Deep expertise in GPU or equivalent accelerator compilation pipelines and low-level architecture.
Expertise in LLM training and inference parallelism and acceleration.
Demonstrated ownership of cross-team technical initiatives from strategy through production.
Track record of creating reusable platforms, influencing senior stakeholders, and mentoring technical leaders.

Desired Qualifications

Expertise in GPU or equivalent accelerator programming and kernel optimization.
Familiarity with vLLM, SGLang, or equivalent LLM inference libraries.
Demonstrated cross-team collaboration and technical ownership.
Ability to use AI-assisted development tools efficiently and validate their output.
Deep expertise in GPU or equivalent accelerator compilation pipelines and low-level architecture.
Expertise in LLM training and inference parallelism and acceleration.
Demonstrated ownership of cross-team technical initiatives from strategy through production.
Track record of creating reusable platforms, influencing senior stakeholders, and mentoring technical leaders.

Benefits

The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year (IC5) or $119,800 - $234,700 per year (IC4).
Specific ranges apply to San Francisco Bay area and New York City metropolitan area.
Certain roles may be eligible for benefits and other compensation.
Applications accepted on an ongoing basis until the position is filled.

Skills

C++PythonGPULLMvLLMSGLang

#AI#LLM#GPU#optimization#inference

About Microsoft

Empowering every person and organization on the planet to achieve more

Every company has a mission. What's ours? To empower every person and every organization to achieve more. We believe technology can and should be a force for good and that meaningful innovation contributes to a brighter world in the future and today. Our culture doesn’t just encourage curiosity; it embraces it. Each day we make progress together by showing up as our authentic selves. We show up with a learn-it-all mentality. We show up cheering on others, knowing their success doesn't diminish our own. We show up every day open to learning our own biases, changing our behavior, and inviting…

See all open jobs at Microsoft Read more about Microsoft Website