← Back to jobsJob ID col-cmvv4uiva04ia42b9ygu993jy

RespanPosted 30+ days ago

Software Engineer, AI

Full-time
Remote
3+ years
Competitive
Posted 30+ days ago

Respan is building an observability and evaluation platform for AI teams. The Software Engineer, AI will build production-grade AI systems, including automated evaluations, prompt optimization, model routing, and AI-powered analysis of traces and logs, while collaborating with customers to address AI workflow challenges.

Posted
30+ Days Ago
Source
ycombinator.com
Sector
Information Technology & AI

Opens the employer's application page in a new tab

Responsibilities

  • Design and build AI-powered features including eval frameworks and prompt optimization
  • Develop LLM-as-judge pipelines for automated quality assessment
  • Build and maintain model routing, caching, and fallback systems
  • Experiment with new model capabilities and translate them into product features
  • Collaborate with customers to understand AI workflow pain points and build solutions

Qualifications

  • Experience: 3+ years
  • Visa: US citizen/visa only
  • Strong software engineering skills with Python and TypeScript
  • Hands-on experience building with LLMs in production (not just prototypes)
  • Deep understanding of prompt engineering, eval methodologies, and model behavior

Preferred

  • Experience with fine-tuning, RAG, or agent frameworks
  • Background in ML infrastructure or MLOps
  • Familiarity with observability systems or developer tools
  • Contributions to open-source AI/ML projects

Required skills

LLM Evaluation MethodologiesLarge Language Models (LLMs)Model Behavior AnalysisPrompt EngineeringPythonTypeScript

Full job description

Respan is building an observability and evaluation platform for AI teams. The Software Engineer, AI will build production-grade AI systems, including automated evaluations, prompt optimization, model routing, and AI-powered analysis of traces and logs, while collaborating with customers to address AI workflow challenges.

Responsibilities

  • Design and build AI-powered features including eval frameworks and prompt optimization
  • Develop LLM-as-judge pipelines for automated quality assessment
  • Build and maintain model routing, caching, and fallback systems
  • Experiment with new model capabilities and translate them into product features
  • Collaborate with customers to understand AI workflow pain points and build solutions

Preferred

  • Experience with fine-tuning, RAG, or agent frameworks
  • Background in ML infrastructure or MLOps
  • Familiarity with observability systems or developer tools
  • Contributions to open-source AI/ML projects
  • Fine-Tuning
  • RAG
  • Agent Frameworks
  • ML Infrastructure
  • MLOps
  • Observability Systems
  • Developer Tools
  • Open-Source AI/ML Contributions

Company

  • LLM observability, evals, prompt optimization, and gateway platform
  • San Francisco, California, USA
  • https://respan.ai/
  • Founded in 2023
  • 11-50 employees

Qualifications

  • Experience: 3+ years
  • Visa: US citizen/visa only
  • Strong software engineering skills with Python and TypeScript
  • Hands-on experience building with LLMs in production (not just prototypes)
  • Deep understanding of prompt engineering, eval methodologies, and model behavior
  • Experience with fine-tuning, RAG, or agent frameworks
  • Background in ML infrastructure or MLOps
  • Familiarity with observability systems or developer tools
  • Contributions to open-source AI/ML projects

Required skills

  • LLM Evaluation Methodologies, Large Language Models (LLMs), Model Behavior Analysis, Prompt Engineering, Python, TypeScript

Preferred skills

  • Background in ML infrastructure or MLOps, Contributions to open-source AI/ML projects, Experience with fine-tuning, RAG, or agent frameworks, Familiarity with observability systems or developer tools

Experience

  • 3+ years

Job details

Employment type

Full-time

Seniority level

3+ years

Date posted

May 28, 2026

Work mode

Remote

Job source

ycombinator.com

Company

Respan

Information Technology & AI

LLM observability, evals, prompt optimization, and gateway platform

  • San Francisco, California, USA
  • Information Technology & AI
  • 11-50 employees
  • Founded 2023
  • respan.ai

Similar jobs

Matched on role, skills and location · live jobs only

See more AI Engineer jobs

Landscape Forms

1d ago

New

Senior Software Engineer – AI & Business Applications

Full-timeRemoteUnited StatesCompetitive

Landscape Forms is hiring a Senior Software Engineer to develop software and AI-powered solutions that improve how teams work across the company.

Why similarSame rolePythonTypeScript

Indigo Slate

1d ago

New

AI-Native Software Engineer

Full-timeRemoteUnited StatesCompetitive

Zensar Technologies is seeking a full-stack, AI-native software engineer to build capabilities that secure current products and AI-built, agent-operated systems.

Why similarSame rolePython

CCC Intelligent Solutions

1d ago

New

Senior AI Engineer

Full-timeRemoteUnited States$180K – $225K/yr

CCC Intelligent Solutions develops technology for the insurance, claims, and repair industries. The Senior AI Engineer owns LLM-powered systems and agentic workflows end to end, including delivery, evaluation, security, and production operation.

Why similarSame rolePythonTypeScript

Get new AI Engineer jobs by email

Create job alert

Software Engineer, AI

Respan · Competitive

Apply on company site