← Back to jobsJob ID col-cmv8b1vaj0115oz01ef4wthb1

H1Posted 30+ days ago

Data Engineer II- Life Sciences

Full-time
Hybrid
3+ years
$110K – $135K/yr
Posted 30+ days ago

H1 provides a healthcare information platform that uses data and AI technology to support better medical insights, patient outcomes, and drug development. The Data Engineer II will build and operate clinical trial data pipelines, transform diverse data sources into internal models, and ensure reliable, validated output. The role also partners with clinical subject matter experts and customer-facing teams to deliver production-ready changes under enterprise SLAs.

Posted
30+ Days Ago
Source
Ethic Leaf recruiter
Sector
Information Technology & AI

No employer link yet — an Ethic Leaf recruiter forwards your application

Responsibilities

  • Build and maintain the Python and PySpark pipelines behind the CTMS trial data pipeline intake workflows, including scoring and status logic
  • Develop the transformation logic that maps raw trial and customer data to H1's internal data models, handling diverse source formats including CSV, JSON, Parquet, and APIs
  • Write and tune SQL against large datasets to investigate data questions, validate pipeline output, and support analysis that clinical SMEs and customer-facing teams depend on
  • Turn around customer-driven changes quickly, scoping requests as they arrive, shipping changes that hold up under enterprise SLAs, and reworking logic as customer needs shift mid-flight
  • Partner with clinical SMEs to translate domain expertise into concrete data rules, then walk them through the results, explain what the pipeline did and why, and fold their feedback back into the logic
  • Build the data quality checks, validation logic, and reconciliation that let non-engineers trust pipeline output without reading the code
  • Participate in code reviews, maintaining a high bar for quality and adherence to engineering standards
  • Monitor and improve pipeline observability, contributing to alerting and dashboards that surface job health and data anomalies for both the team and internal users

Qualifications

  • Building and shipping production data pipelines in Python, with an understanding of what makes them reliable and maintainable under real load
  • Working with PySpark or a comparable distributed processing framework on datasets too large for a single machine
  • Writing SQL well enough to answer hard questions about data, not just retrieve it
  • Working in an operationally-driven environment where reliability and on-time delivery matter as much as new feature work
  • Working directly with non-engineering partners, subject matter experts, analysts, or customer-facing teams, and communicating clearly about data with people who don't read code
  • Holding a high bar in code review and expecting the same from those who review your work
  • Identifying data problems early and seeing work through to resolution rather than handing it off
  • 3+ years of experience in software or data engineering, with meaningful Python in your background
  • Demonstrated experience building and maintaining production-grade data pipelines in Python
  • Hands-on experience with PySpark or a similar distributed data processing framework
  • Strong SQL skills, including working with large, messy, multi-source datasets
  • Strong understanding of software quality practices: testing, code review, documentation, and CI/CD
  • Experience working with cross-functional and non-technical stakeholders

Preferred

  • Experience with pipeline orchestration tooling (Argo, Airflow, Databricks, dbt, or similar)
  • Familiarity with clinical trial data, healthcare data, or another regulated data domain
  • Familiarity with entity matching or data mastering
  • Familiarity with AWS services (S3, Lambda, ECS, or similar)

Required skills

PythonPySparkSQLProduction Data PipelinesData Quality ValidationSoftware TestingCode ReviewCI/CDPipeline OrchestrationClinical Trial DataEntity MatchingAWS

Full job description

H1 provides a healthcare information platform that uses data and AI technology to support better medical insights, patient outcomes, and drug development. The Data Engineer II will build and operate clinical trial data pipelines, transform diverse data sources into internal models, and ensure reliable, validated output. The role also partners with clinical subject matter experts and customer-facing teams to deliver production-ready changes under enterprise SLAs.

Responsibilities

  • Build and maintain the Python and PySpark pipelines behind the CTMS trial data pipeline intake workflows, including scoring and status logic
  • Develop the transformation logic that maps raw trial and customer data to H1's internal data models, handling diverse source formats including CSV, JSON, Parquet, and APIs
  • Write and tune SQL against large datasets to investigate data questions, validate pipeline output, and support analysis that clinical SMEs and customer-facing teams depend on
  • Turn around customer-driven changes quickly, scoping requests as they arrive, shipping changes that hold up under enterprise SLAs, and reworking logic as customer needs shift mid-flight
  • Partner with clinical SMEs to translate domain expertise into concrete data rules, then walk them through the results, explain what the pipeline did and why, and fold their feedback back into the logic
  • Build the data quality checks, validation logic, and reconciliation that let non-engineers trust pipeline output without reading the code
  • Participate in code reviews, maintaining a high bar for quality and adherence to engineering standards
  • Monitor and improve pipeline observability, contributing to alerting and dashboards that surface job health and data anomalies for both the team and internal users

Preferred

  • Experience with pipeline orchestration tooling (Argo, Airflow, Databricks, dbt, or similar)
  • Familiarity with clinical trial data, healthcare data, or another regulated data domain
  • Familiarity with entity matching or data mastering
  • Familiarity with AWS services (S3, Lambda, ECS, or similar)
  • Pipeline Orchestration
  • Argo
  • Airflow
  • Databricks
  • dbt
  • Clinical Trial Data
  • Healthcare Data
  • Regulated Data
  • Entity Matching
  • Data Mastering
  • AWS
  • Amazon S3
  • AWS Lambda
  • Amazon ECS

Benefits

  • Full suite of health insurance options, in addition to generous paid time off
  • Pre-planned company-wide wellness holidays
  • Retirement options
  • Health & charitable donation stipends
  • Impactful Business Resource Groups
  • The opportunity to work with leading biotech and life sciences companies in an innovative industry with a mission to improve healthcare around the globe
  • Stock options
  • Hybrid work arrangement

Company

  • AI platform for identifying and engaging doctors in healthcare
  • New York, New York, USA
  • https://h1.com/
  • Founded in 2017
  • 201-500 employees

Qualifications

  • Building and shipping production data pipelines in Python, with an understanding of what makes them reliable and maintainable under real load
  • Working with PySpark or a comparable distributed processing framework on datasets too large for a single machine
  • Writing SQL well enough to answer hard questions about data, not just retrieve it
  • Working in an operationally-driven environment where reliability and on-time delivery matter as much as new feature work
  • Working directly with non-engineering partners, subject matter experts, analysts, or customer-facing teams, and communicating clearly about data with people who don't read code
  • Holding a high bar in code review and expecting the same from those who review your work
  • Identifying data problems early and seeing work through to resolution rather than handing it off
  • 3+ years of experience in software or data engineering, with meaningful Python in your background
  • Demonstrated experience building and maintaining production-grade data pipelines in Python
  • Hands-on experience with PySpark or a similar distributed data processing framework
  • Strong SQL skills, including working with large, messy, multi-source datasets
  • Strong understanding of software quality practices: testing, code review, documentation, and CI/CD
  • Experience working with cross-functional and non-technical stakeholders
  • Experience with pipeline orchestration tooling (Argo, Airflow, Databricks, dbt, or similar)
  • Familiarity with clinical trial data, healthcare data, or another regulated data domain
  • Familiarity with entity matching or data mastering
  • Familiarity with AWS services (S3, Lambda, ECS, or similar)

Required skills

  • Python, PySpark, SQL, Production Data Pipelines, Data Quality Validation, Software Testing, Code Review, CI/CD, Pipeline Orchestration, Clinical Trial Data, Entity Matching, AWS

Preferred skills

  • Familiarity with entity matching or data mastering, Familiarity with AWS services (S3, Lambda, ECS, or similar)

Experience

  • 3+ years

Job details

Employment type

Full-time

Seniority level

3+ years

Date posted

Oct 6, 2026

Work mode

Hybrid

Job source

Ethic Leaf recruiter

Company

H1

Information Technology & AI

AI platform for identifying and engaging doctors in healthcare

See all 4 jobs from H1 →
  • New York, New York, USA
  • Information Technology & AI
  • 201-500 employees
  • Founded 2017
  • h1.com

Similar jobs

Matched on role, skills and location · live jobs only

See more Data Engineer jobs in New York

M Science

1d ago

New

Data Engineer

Full-timeOn-siteNew York, NY$110K – $150K/yr

M Science is hiring a Data Engineer to build and improve large-scale data pipelines and datasets for analytics, experimentation, and machine learning.

Why similarSame roleAWSNew York

Dice

1d ago

New

Senior Data Engineer - Databricks

Full-timeOn-siteNew York, NY$84K – $156K/yr

Capgemini is seeking a Senior Databricks Data Engineer to design, build, and optimize scalable cloud-based data platforms and pipelines.

Why similarSame roleAWSNew York

Capgemini

1d ago

New

Senior Data Engineer - Databricks

Full-timeOn-siteNew York, NY$84K – $156K/yr

Capgemini is seeking a Senior Databricks Data Engineer to design, build, and optimize scalable data platforms and pipelines in cloud-based environments.

Why similarSame roleAWSNew York

Get new Data Engineer jobs in New York by email

Create job alert

Data Engineer II- Life Sciences

H1 · $110K – $135K/yr

Apply via Ethic Leaf