Principal Software Engineering, AI Infrastructure

Microsoft’s FIT Infrastructure team builds accelerated compute platforms for large-scale AI training and inference. The Principal Software Engineering, AI Infrastructure role develops and operates secure, reliable, high-performance GPU infrastructure and AI workload management and serving systems, while influencing platform architecture and technical direction.

Responsibilities

• Develop systems for AI Workload management, scheduling, preemption, and failover
• Develop systems for AI Workload serving
• Design and build GPU accelerated infrastructure for training and inference workloads, spanning bare metal, virtual machines, and containerized environments
• Build and operate advanced orchestration and resource governance scenarios using platforms such as AKS and related Kubernetes ecosystem capabilities to enable fair sharing, isolation, and efficient utilization of accelerated resources
• Design, Develop, test, and maintain services written in C#, Golang hosted on Kubernetes clusters
• Optimize performance, reliability, and utilization across large GPU fleets, including scaleup and scale out configurations
• Drive end-to-end platform features from design through production, including observability, diagnostics, and operational excellence
• Influence platform architecture and technical direction across teams through design reviews and technical leadership
• Embody our [Culture](https://../sites/Culture/) and [Values](https://../sites/Values)

Preferred

• Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Linux, Go or Python + OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, linux, Go or Python + OR equivalent experience
• Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Linux, Go or Python + OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, linux, Go or Python + OR equivalent experience.
• Computer Science
• Technical Engineering
• Coding

Company

• Software, cloud computing, consumer electronics, and enterprise systems.
• Redmond, Washington, USA
• https://www.microsoft.com/
• Founded in 1975
• 10001+ employees

Qualifications

• Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Linux, Go or Python + OR equivalent experience
• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter
• Proven ability to design and operate large scale, production infrastructure with high reliability and performance requirements
• Demonstrated problem-solving skills and the ability to debug complex, cross layer systems issues
• Demonstrated technical leadership, including mentoring engineers and driving cross team architectural alignment
• Hands-on experience with resource scheduling, preemption, priority handling on large clusters
• Advanced collaboration and communication skills, with the ability to work across organizational boundaries
• Experience with large-scale online Inferencing services
• Experience in building or operating multitenant AI platforms in cloud environments
• Familiarity with inference and/or distributed training stacks (e.g., NCCL style collectives, model/data parallelism, Large-Language Models)
• Familiarity with GPU accelerated computing (e.g., CUDA, GPU drivers, device plugins, or runtime integration)
• Dev-ops experience with microservices architecture in a complex infrastructure and operational environment
• Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Linux, Go or Python + OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, linux, Go or Python + OR equivalent experience
• Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Linux, Go or Python + OR equivalent experience.
• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.

Required skills

• C#, Distributed Training, GPU-Accelerated Computing, Go, Kubernetes, Large-Scale Production Infrastructure, Linux, Microservices Architecture, Multitenant AI Platforms, Online Inference Services, Python, Resource Scheduling

Preferred skills

• (5083), (7677), (9192), (9343), *2026, 2022, 2023, 2024, 2025, 3.9, 80, Accounting and Finance, Creatives and Design, Customer Service and Support, Data Powered by, Distribution of Different Job Fields Receiving Sponsorship, Education and Training, Engineering and Development, Glassdoor, H1B Sponsorship, Human Resources, Legal and Compliance, Management and Executive, Marketing, Product Management, Redmond, Washington, USA, Represents job field similar to this job, Sales, Trends of Total Sponsorships, https://www.microsoft.com

Experience

• 6+ years