Overview
The Principal Product Manager for Fine-Tuning and Distillation in Microsoft’s CoreAI organization is a senior product leader responsible for shaping the strategy, execution, and impact of advanced model customization capabilities within Microsoft Foundry. This role sits at the center of Microsoft’s AI platform strategy, partnering closely with flagship internal customers including M365, GitHub, Microsoft AI to turn internal innovation and proof points into scalable, secure, and world-class platform capabilities for external developers and enterprises. By driving the end-to-end product vision for fine-tuning and distillation, this role is critical to enabling differentiated model performance, cost efficiency, and customization at scale, while helping establish Microsoft Foundry as the industry’s leading AI platform.
Responsibilities
At Microsoft, our mission is to empower every developer and organization to achieve more with AI. Within the CoreAI organization, Microsoft Foundry is the platform where cutting-edge AI research, infrastructure, and product innovation converge to power Microsoft’s most important AI experiences—and to bring those same capabilities to customers worldwide.
As the Principal Product Manager for Fine-Tuning and Distillation, you will define and drive the vision for model customization capabilities across Microsoft Foundry, enabling teams to adapt frontier and domain-specific models for performance, cost, latency, and safety. This role sits at the intersection of platform, research, and applied AI, working closely with Microsoft’s largest internal customers such as M365, GitHub, Microsoft AI, and other flagship AI teams to turn internal innovation and proof points into scalable, secure, and developer-friendly platform offerings for external customers.
You will partner deeply with engineering, research, applied science, and partner teams to design end-to-end fine-tuning and distillation workflows that support a range of scenarios, from enterprise customization to high-volume consumer workloads.
This role is critical to establishing Microsoft Foundry as a world-class AI platform by making advanced model optimization techniques accessible, reliable, and cost-effective at scale.
We are looking for an AI/ML product leader who combines technical intuition with platform thinking - someone who thrives in ambiguity, can translate frontier research into practical systems, and has a track record of shipping foundational capabilities used by multiple teams at scale.
Success in Role
In your first 3-6 months, success looks like:
- Delivering production-ready fine-tuning and distillation capabilities that are adopted by multiple Tier-1 internal customers.
- Translating internal proof points (e.g., M365, GitHub, Microsoft AI) into repeatable, platformized offerings within Microsoft Foundry.
- Demonstrating measurable improvements in model quality, inference cost, latency, and operational efficiency through fine-tuning and distillation.
- Establishing clear abstractions, APIs, and workflows that simplify advanced model customization for both internal and external developers.
- Building strong alignment across research, engineering, and product teams around a cohesive roadmap for model optimization.
- Becoming a trusted product leader for one of Foundry’s most strategic and technically complex platform pillars.
Responsibilities
- Platform Vision & Strategy: Define and drive the long-term vision and roadmap for fine-tuning and distillation capabilities within Microsoft Foundry.
- Customer-Informed Design: Partner closely with flagship internal teams (M365, GitHub, Microsoft AI, and others) to identify needs, validate solutions, and convert internal innovation into scalable platform features.
- End-to-End Workflows: Own the product experience for fine-tuning and distillation, spanning data preparation, training, evaluation, deployment, and lifecycle management.
- Model Optimization: Enable techniques that improve model performance, cost efficiency, latency, and safety across diverse workloads and model families.
- Research-to-Product Translation: Collaborate with research and applied science teams to operationalize cutting-edge techniques into reliable, supportable platform capabilities.
- Developer Experience: Design intuitive APIs, tooling, and abstractions that make advanced model customization accessible to a broad developer audience.
- Cross-Functional Leadership: Drive alignment across engineering, infrastructure, security, compliance, and business stakeholders to deliver high-impact outcomes.
- Roadmap & Prioritization: Establish a customer-driven prioritization framework balancing near-term impact with long-term platform scalability.
- Operational Excellence: Define success metrics, feedback loops, and continuous improvement processes to ensure reliability and adoption at scale.
Qualifications
Required Qualifications
- Bachelor's Degree AND 8+ years experience in product/service/program management or software development OR equivalent experience.
Other Requirements
- Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:
- Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Preferred Qualifications
- 3+ years of hands-on experience with Azure (or similar cloud) services, including resource provisioning, IAM, networking, and scripting (PowerShell or Azure CLI).
- 6+ years experience managing cross-functional and/or cross-team projects.
- Deep expertise in AI/ML data pipelines—pre-training, fine-tuning, evaluation, and deployment.
- Track record of leading cross-functional efforts at scale in high-stakes, fast-paced environments.
- Startup founder/early-stage experience or leadership roles at Meta, Google, Amazon, or similar AI-first companies
- Proven experience in product management for large-scale data or AI platforms.
- Technical background in data lifecycle management, security, and compliance.
- Demonstrated ability to lead cross-functional teams and drive strategic initiatives.
- Entrepreneurial mindset; thrives in fast-paced, ambiguous environments.
Product Management IC5 - The typical base pay range for this role across the U.S. is USD $139,900 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
