← Back to jobsJob ID col-cmwpe8ptv0blsd8hahi9465fm

NVIDIAPosted 1 day ago

Senior Deep Learning Engineer, Cosmo 3D SpatialNew

Full-time
On-site
12+ years
$224K – $356K/yr
Posted 1 day ago

NVIDIA is hiring a Senior Deep Learning Engineer for Cosmos spatial reasoning and perception. The role owns the 3D data engine, end-to-end training verification, and spatial evaluation suite, while partnering with research scientists to translate 3D research into measurable capabilities in shipped models.

Posted
1 day ago
Source
nvidia.wd5.myworkdayjobs.com
Sector
Information Technology & AI

Opens the employer's application page in a new tab

Responsibilities

  • Own the 3D data engine for Cosmos spatial reasoning: source, curate, filter, and balance large-scale real-world image and video corpora into vision-language training data with the coverage and diversity that spatial understanding demands
  • Build the annotation and auto-labeling pipelines that produce 3D-grounded supervision at scale, camera-relative 3D boxes, referring and spatial question answering, free space and reachability, ego-, world-, and object-centric reference frames, cross-view correspondence, camera motion, distance and size, and chain-of-thought traces, validated by programmatic and model-based critics
  • Own data quality end to end: semantic deduplication, automated quality scoring for faithfulness, completeness, and correctness, coverage analysis across scene types and reference frames, and the sampling strategies that keep pre-training and supervised fine-tuning mixtures balanced
  • Verify end-to-end model training: run and validate full pre-training and supervised fine-tuning pipelines, guard reproducibility, catch data and checkpoint regressions, diagnose throughput and loss anomalies, and attribute capability changes back to the specific data and recipe decisions that caused them
  • Build and operate the 3D and spatial evaluation suite, public benchmarks such as CV-Bench, BLINK, RefSpatial, VSI-Bench, SPAR-Bench, and RoboSpatial, NVIDIA's VANTAGE-Bench for real-world fixed-camera video understanding, and in-house benchmarks you design with continuous evaluation and full traceability from every reported score back to the exact weights, inputs, configuration, and evaluation code
  • Partner closely with Cosmos Lab research scientists: translate 3D research hypotheses into dataset and ablation experiments, run them at scale, and feed honest results back into recipe and architecture decisions
  • Operate on large multi-node GPU clusters, tuning data throughput, sharding, and dataloader performance so that data is never the bottleneck on a long training run
  • Ship the results into Cosmos releases, open-source datasets and benchmarks where appropriate, and raise the bar for data and evaluation rigor across the team

Qualifications

  • MS or PhD in Computer Science, Electrical/Computer Engineering, Robotics, or a related field, or equivalent experience
  • 12+ years of proven experience building deep learning systems in Python with PyTorch or JAX on Linux
  • Deep expertise in 3D computer vision, multi-view geometry, structure-from-motion or SLAM, depth and camera pose estimation, point cloud processing, or 3D reconstruction with the practical ability to produce and validate 3D ground truth at scale, not just consume it
  • Hands-on experience with vision-language models, including building the training data and evaluations that measurably improve visual grounding and reasoning quality
  • Demonstrated experience building large-scale multimodal data pipelines: distributed video and image processing, deduplication, captioning and annotation, automated quality metrics, and dataset versioning
  • Experience running and validating large model training on multi-GPU, multi-node clusters, with working knowledge of distributed training and sharding strategies such as data, tensor, and pipeline parallelism or FSDP
  • Rigorous evaluation methodology: designing benchmarks that resist gaming, building clean ablations, and reading results honestly enough to kill your own ideas
  • Excellent written and verbal communication, with a track record of partnering effectively with research scientists and translating research direction into engineering execution

Preferred

  • PhD and/or publications at CVPR, ICCV, ECCV, NeurIPS, ICLR, or CoRL in 3D vision, multimodal learning, or embodied AI
  • Experience curating web-scale or petabyte-scale video corpora, and building the distributed processing infrastructure behind it using Ray, Spark, Slurm, or similar
  • Familiarity with 3D foundation models and modern auto-labeling techniques for geometry, camera estimation, and correspondence
  • Experience with vision-language model post-training, supervised fine-tuning, chain-of-thought data design, reward modeling, or reinforcement learning with verifiable rewards applied to reasoning quality
  • Experience building evaluation infrastructure and harnesses such as VLMEvalKit, including leaderboards, dashboards, and example-level failure inspection
  • Chain-of-thought data design

Required skills

3D Computer Vision3D ReconstructionDistributed TrainingEvaluation DesignJAXMulti-view GeometryMultimodal Data PipelinesPoint Cloud ProcessingPyTorchPythonStructure-from-Motion and SLAMVision-Language Models

Full job description

NVIDIA is hiring a Senior Deep Learning Engineer for Cosmos spatial reasoning and perception. The role owns the 3D data engine, end-to-end training verification, and spatial evaluation suite, while partnering with research scientists to translate 3D research into measurable capabilities in shipped models.

Responsibilities

  • Own the 3D data engine for Cosmos spatial reasoning: source, curate, filter, and balance large-scale real-world image and video corpora into vision-language training data with the coverage and diversity that spatial understanding demands
  • Build the annotation and auto-labeling pipelines that produce 3D-grounded supervision at scale, camera-relative 3D boxes, referring and spatial question answering, free space and reachability, ego-, world-, and object-centric reference frames, cross-view correspondence, camera motion, distance and size, and chain-of-thought traces, validated by programmatic and model-based critics
  • Own data quality end to end: semantic deduplication, automated quality scoring for faithfulness, completeness, and correctness, coverage analysis across scene types and reference frames, and the sampling strategies that keep pre-training and supervised fine-tuning mixtures balanced
  • Verify end-to-end model training: run and validate full pre-training and supervised fine-tuning pipelines, guard reproducibility, catch data and checkpoint regressions, diagnose throughput and loss anomalies, and attribute capability changes back to the specific data and recipe decisions that caused them
  • Build and operate the 3D and spatial evaluation suite, public benchmarks such as CV-Bench, BLINK, RefSpatial, VSI-Bench, SPAR-Bench, and RoboSpatial, NVIDIA's VANTAGE-Bench for real-world fixed-camera video understanding, and in-house benchmarks you design with continuous evaluation and full traceability from every reported score back to the exact weights, inputs, configuration, and evaluation code
  • Partner closely with Cosmos Lab research scientists: translate 3D research hypotheses into dataset and ablation experiments, run them at scale, and feed honest results back into recipe and architecture decisions
  • Operate on large multi-node GPU clusters, tuning data throughput, sharding, and dataloader performance so that data is never the bottleneck on a long training run
  • Ship the results into Cosmos releases, open-source datasets and benchmarks where appropriate, and raise the bar for data and evaluation rigor across the team

Preferred

  • PhD and/or publications at CVPR, ICCV, ECCV, NeurIPS, ICLR, or CoRL in 3D vision, multimodal learning, or embodied AI
  • Experience curating web-scale or petabyte-scale video corpora, and building the distributed processing infrastructure behind it using Ray, Spark, Slurm, or similar
  • Familiarity with 3D foundation models and modern auto-labeling techniques for geometry, camera estimation, and correspondence
  • Experience with vision-language model post-training, supervised fine-tuning, chain-of-thought data design, reward modeling, or reinforcement learning with verifiable rewards applied to reasoning quality
  • Experience building evaluation infrastructure and harnesses such as VLMEvalKit, including leaderboards, dashboards, and example-level failure inspection
  • PhD
  • Conference publications
  • 3D vision
  • Multimodal learning
  • Embodied AI
  • Video corpus curation
  • Web-scale data processing
  • Petabyte-scale data processing
  • Distributed processing infrastructure
  • Ray
  • Spark
  • Slurm
  • 3D foundation models
  • Auto-labeling techniques
  • Geometry labeling
  • Camera estimation
  • Correspondence estimation
  • Vision-language model post-training
  • Supervised fine-tuning
  • Chain-of-thought data design
  • Reward modeling
  • Reinforcement learning
  • Verifiable rewards
  • Reasoning quality evaluation
  • Evaluation infrastructure
  • Evaluation harnesses
  • VLMEvalKit
  • Leaderboard development
  • Dashboard development
  • Example-level failure inspection

Benefits

  • Eligible for equity

Company

  • Accelerated computing platforms for AI and compute-intensive workloads
  • Santa Clara, California, USA
  • https://www.nvidia.com/
  • Founded in 1993
  • 10001+ employees

Qualifications

  • MS or PhD in Computer Science, Electrical/Computer Engineering, Robotics, or a related field, or equivalent experience
  • 12+ years of proven experience building deep learning systems in Python with PyTorch or JAX on Linux
  • Deep expertise in 3D computer vision, multi-view geometry, structure-from-motion or SLAM, depth and camera pose estimation, point cloud processing, or 3D reconstruction with the practical ability to produce and validate 3D ground truth at scale, not just consume it
  • Hands-on experience with vision-language models, including building the training data and evaluations that measurably improve visual grounding and reasoning quality
  • Demonstrated experience building large-scale multimodal data pipelines: distributed video and image processing, deduplication, captioning and annotation, automated quality metrics, and dataset versioning
  • Experience running and validating large model training on multi-GPU, multi-node clusters, with working knowledge of distributed training and sharding strategies such as data, tensor, and pipeline parallelism or FSDP
  • Rigorous evaluation methodology: designing benchmarks that resist gaming, building clean ablations, and reading results honestly enough to kill your own ideas
  • Excellent written and verbal communication, with a track record of partnering effectively with research scientists and translating research direction into engineering execution
  • PhD and/or publications at CVPR, ICCV, ECCV, NeurIPS, ICLR, or CoRL in 3D vision, multimodal learning, or embodied AI
  • Experience curating web-scale or petabyte-scale video corpora, and building the distributed processing infrastructure behind it using Ray, Spark, Slurm, or similar
  • Familiarity with 3D foundation models and modern auto-labeling techniques for geometry, camera estimation, and correspondence
  • Experience with vision-language model post-training, supervised fine-tuning, chain-of-thought data design, reward modeling, or reinforcement learning with verifiable rewards applied to reasoning quality
  • Experience building evaluation infrastructure and harnesses such as VLMEvalKit, including leaderboards, dashboards, and example-level failure inspection
  • 12+ years of proven experience building deep learning systems in Python with PyTorch or JAX on Linux.

Required skills

  • 3D Computer Vision, 3D Reconstruction, Depth and Camera Pose Estimation, Distributed Training, Evaluation Design, JAX, Multi-view Geometry, Multimodal Data Pipelines, Point Cloud Processing, PyTorch, Python, Structure-from-Motion and SLAM, Vision-Language Models

Experience

  • 12+ years

Job details

Employment type

Full-time

Seniority level

12+ years

Date posted

Nov 20, 2026

Work mode

On-site

Job source

nvidia.wd5.myworkdayjobs.com

Company

NVIDIA

Information Technology & AI

Accelerated computing platforms for AI and compute-intensive workloads

See all 24 jobs from NVIDIA →
  • Santa Clara, California, USA
  • Information Technology & AI
  • 10001+ employees
  • Founded 1993
  • nvidia.com

Similar jobs

Matched on role, skills and location · live jobs only

See more Backend Engineer jobs in Santa Clara

Scribe

1d ago

New

Senior Backend Engineer

Full-timeRemoteUnited States$175K – $240K/yr

Scribe is hiring a Senior Backend Engineer to own a product area end to end, from design through post-launch results. The role builds and improves backend features, performance, and reliability while collaborating across product and engineering teams.

Why similarSame rolePythonRemote

#Backend Engineering Opportunities

1d ago

New

#Backend Engineering Opportunities | Backend Engineer & Lead Backend Engineer, AI Platform

Full-timeRemoteUnited States$140K – $240K/yr

Lakeview Loan Servicing is hiring Backend Engineers and Lead Backend Engineers for its AI Platform Backend Engineering team.

Why similarSame rolePythonRemote

SentiLink

1d ago

New

Senior Software Engineer, Platform/Backend

Full-timeRemoteUnited States$170K – $230K/yr

SentiLink builds identity verification and fraud-risk products for financial institutions. The Senior Software Engineer will build and enhance the API platform powering these products, collaborating across product, engineering, and security to deliver…

Why similarSame rolePythonRemote

Get new Backend Engineer jobs in Santa Clara by email

Create job alert

Senior Deep Learning Engineer, Cosmo 3D Spatial

NVIDIA · $224K – $356K/yr

Apply on company site