← Back to jobsJob ID col-cmwp6o3oj06vtd8haixxkyjwi

Intuitive.aiPosted 1 day ago

Senior Data EngineerNew

Full-time
Remote
6+ years
Competitive
Posted 1 day ago

Intuitive.AI is hiring a Senior Data Engineer to build and operate a modern data platform that transforms operational data into trusted datasets for analytics, reporting, and AI applications. The role owns data pipelines and datasets end-to-end, from source analysis and modelling through production operations, with a focus on Python, SQL, and Snowflake.

Posted
1 day ago
Source
ca.linkedin.com
Sector
Information Technology & AI

Opens the employer's application page in a new tab

Responsibilities

  • Build and operate production data pipelines in Python and SQL that ingest data from databases, APIs, files, SaaS sources, and event streams, in both batch and incremental or streaming modes
  • Own datasets end-to-end — from source analysis and ingestion through modelling, transformation, testing, rollout, and production operation. You own the dataset from raw landing zone to consumer-facing table, and you answer the page when it breaks
  • Explore, profile, and analyse data in notebooks — use Snowflake Notebooks or Jupyter with Snowpark, pandas, or Polars to understand new sources, validate assumptions, prototype transformations, and investigate data issues before and after they reach production
  • Design and maintain data models — layered raw / staging / curated structures, dimensional and wide-table models, clear keys and grain, and schema evolution that doesn't break consumers
  • Develop transformations in SQL and dbt (or an equivalent framework) with tests, documentation, and lineage, so every curated table has a clear owner, contract, and definition
  • Orchestrate pipelines with Airflow or a similar scheduler — dependencies, retries, backfills, SLAs, and alerting — so runs are predictable and recoverable
  • Work deeply in Snowflake — warehouse schemas, roles and access, Snowpark Python, loading and incremental patterns, performance tuning, and cost awareness, with Spark for large-scale processing where Snowflake alone isn't the right tool
  • Implement data-quality controls — validation, freshness checks, reconciliation against sources, anomaly detection, and failure handling that alerts before consumers notice
  • Operate pipelines in production — logs, metrics, dashboards, actionable alerts, runbooks, and disciplined incident investigation and post-incident fixes
  • Define and maintain schemas and data contracts so analysts, dashboards, APIs, and AI applications can rely on stable, well-understood interfaces while sources evolve
  • Ship through CI/CD — automated tests for pipelines, transformations, and contracts; environment promotion; and release validation for data changes
  • Prepare analysis-ready and AI-ready datasets — curated tables, aggregates, and feature-style views that analysts, BI tools, and AI applications can consume safely
  • Raise the engineering bar — write clean, well-tested Python and SQL; review others' work; document data flows and decisions with concise diagrams and notes; and mentor engineers on modelling, testing, and operability
  • Use AI-assisted engineering tools effectively to accelerate development, testing, review, documentation, and debugging, while validating generated output
  • Apply shift-left security and governance — data classification, PII handling, and least-privilege access considered during implementation, not as a final release check

Qualifications

  • 6+ years of data engineering or software engineering experience, with 2+ years in senior roles building and operating production data pipelines
  • 2+ years running data pipelines in production — you've owned pipelines past the initial ship: on-call, incident response, backfills, schema migrations, and the operational grind of keeping datasets fresh and correct for real consumers
  • Strong Python — you write clear, idiomatic, well-tested Python for data pipelines, transformations, and tooling, with hands-on depth in pandas or Polars and the standard data stack (typing, packaging, pytest, virtual environments)
  • Strong SQL — complex analytical queries, window functions, CTEs, query optimisation, and the ability to read a query plan and fix a slow query
  • Notebook-driven data analysis — comfortable in Snowflake Notebooks or Jupyter for exploratory data analysis, profiling, data validation, and prototyping, and disciplined about turning notebook work into production code
  • Strong hands-on Snowflake experience — schemas, roles and access control, Snowpark Python, loading and incremental patterns, performance tuning, and cost awareness
  • Experience with dbt or an equivalent SQL transformation framework — models, tests, documentation, and environment promotion
  • Experience with a workflow orchestrator — Airflow, Dagster, Prefect, or similar — including scheduling, dependencies, retries, backfills, and alerting
  • Strong data modelling skills — dimensional and wide-table modelling, layered warehouse / medallion architectures, schema evolution, and data contracts
  • Production experience with both batch and incremental or streaming pipelines, with a working understanding of idempotency, deduplication, replay, late-arriving data, and recovery
  • Hands-on experience with at least one major cloud platform (AWS, GCP, or Azure) and its storage, identity, and data services
  • Experience building observable pipelines — logs, metrics, freshness checks, data-quality tests, dashboards, and actionable alerts
  • Experience delivering changes through CI/CD with automated tests for pipelines, transformations, and contracts
  • Experience using AI-assisted engineering tools as part of routine delivery, with the judgment to validate and improve generated output

Preferred

  • Apache Spark or PySpark experience for large-scale batch and streaming processing
  • Databricks exposure — Delta Lake and Unity Catalog
  • Apache Kafka, Amazon Kinesis, or equivalent event-streaming platforms in production
  • Open table formats — Apache Iceberg or Delta Lake
  • Statistical analysis or data-science fundamentals — distributions, outlier detection, basic hypothesis testing — applied to data validation and investigation
  • Experience preparing datasets for machine learning, AI agents, Retrieval-Augmented Generation (RAG), or analytical APIs
  • Experience with data-quality and observability tooling such as Great Expectations, Soda, or Monte Carlo
  • Infrastructure-as-code experience (Terraform, OpenTofu, or similar) for data-platform resources
  • Experience handling PII or sensitive data — masking, tokenisation, row-level and column-level controls
  • Experience optimising Snowflake, Spark, and object-storage workloads for performance and cost
  • Experience mentoring engineers or leading technical design discussions
  • Machine Learning Dataset Preparation
  • AI Agent Dataset Preparation
  • Analytical API Dataset Preparation

Required skills

AWS, GCP, or AzureAirflow, Dagster, or PrefectApache Spark or PySparkCI/CDData ModelingData Quality and ObservabilityPythonSQLSnowflakedbtpandas or Polars

Full job description

Intuitive.AI is hiring a Senior Data Engineer to build and operate a modern data platform that transforms operational data into trusted datasets for analytics, reporting, and AI applications. The role owns data pipelines and datasets end-to-end, from source analysis and modelling through production operations, with a focus on Python, SQL, and Snowflake.

Responsibilities

  • Build and operate production data pipelines in Python and SQL that ingest data from databases, APIs, files, SaaS sources, and event streams, in both batch and incremental or streaming modes
  • Own datasets end-to-end — from source analysis and ingestion through modelling, transformation, testing, rollout, and production operation. You own the dataset from raw landing zone to consumer-facing table, and you answer the page when it breaks
  • Explore, profile, and analyse data in notebooks — use Snowflake Notebooks or Jupyter with Snowpark, pandas, or Polars to understand new sources, validate assumptions, prototype transformations, and investigate data issues before and after they reach production
  • Design and maintain data models — layered raw / staging / curated structures, dimensional and wide-table models, clear keys and grain, and schema evolution that doesn't break consumers
  • Develop transformations in SQL and dbt (or an equivalent framework) with tests, documentation, and lineage, so every curated table has a clear owner, contract, and definition
  • Orchestrate pipelines with Airflow or a similar scheduler — dependencies, retries, backfills, SLAs, and alerting — so runs are predictable and recoverable
  • Work deeply in Snowflake — warehouse schemas, roles and access, Snowpark Python, loading and incremental patterns, performance tuning, and cost awareness, with Spark for large-scale processing where Snowflake alone isn't the right tool
  • Implement data-quality controls — validation, freshness checks, reconciliation against sources, anomaly detection, and failure handling that alerts before consumers notice
  • Operate pipelines in production — logs, metrics, dashboards, actionable alerts, runbooks, and disciplined incident investigation and post-incident fixes
  • Define and maintain schemas and data contracts so analysts, dashboards, APIs, and AI applications can rely on stable, well-understood interfaces while sources evolve
  • Ship through CI/CD — automated tests for pipelines, transformations, and contracts; environment promotion; and release validation for data changes
  • Prepare analysis-ready and AI-ready datasets — curated tables, aggregates, and feature-style views that analysts, BI tools, and AI applications can consume safely
  • Raise the engineering bar — write clean, well-tested Python and SQL; review others' work; document data flows and decisions with concise diagrams and notes; and mentor engineers on modelling, testing, and operability
  • Use AI-assisted engineering tools effectively to accelerate development, testing, review, documentation, and debugging, while validating generated output
  • Apply shift-left security and governance — data classification, PII handling, and least-privilege access considered during implementation, not as a final release check

Preferred

  • Apache Spark or PySpark experience for large-scale batch and streaming processing
  • Databricks exposure — Delta Lake and Unity Catalog
  • Apache Kafka, Amazon Kinesis, or equivalent event-streaming platforms in production
  • Open table formats — Apache Iceberg or Delta Lake
  • Statistical analysis or data-science fundamentals — distributions, outlier detection, basic hypothesis testing — applied to data validation and investigation
  • Experience preparing datasets for machine learning, AI agents, Retrieval-Augmented Generation (RAG), or analytical APIs
  • Experience with data-quality and observability tooling such as Great Expectations, Soda, or Monte Carlo
  • Infrastructure-as-code experience (Terraform, OpenTofu, or similar) for data-platform resources
  • Experience handling PII or sensitive data — masking, tokenisation, row-level and column-level controls
  • Experience optimising Snowflake, Spark, and object-storage workloads for performance and cost
  • Experience mentoring engineers or leading technical design discussions
  • Apache Spark
  • PySpark
  • Large-Scale Batch Processing
  • Streaming Data Processing
  • Databricks
  • Delta Lake
  • Unity Catalog
  • Apache Kafka
  • Amazon Kinesis
  • Event-Streaming Platforms
  • Apache Iceberg
  • Open Table Formats
  • Statistical Analysis
  • Data Science Fundamentals
  • Distribution Analysis
  • Outlier Detection
  • Hypothesis Testing
  • Data Investigation
  • Machine Learning Dataset Preparation
  • AI Agent Dataset Preparation
  • RAG Dataset Preparation
  • Analytical API Dataset Preparation
  • Data Quality Tooling
  • Data Observability
  • Great Expectations
  • Soda
  • Monte Carlo
  • Infrastructure as Code
  • Terraform
  • OpenTofu
  • PII Handling
  • Sensitive Data Handling
  • Data Masking
  • Data Tokenization
  • Row-Level Access Controls
  • Column-Level Access Controls
  • Snowflake Optimization
  • Spark Optimization
  • Object Storage Optimization
  • Performance Optimization
  • Cost Optimization
  • Mentoring Engineers
  • Technical Design Leadership

Benefits

  • Remote across USA/Canada

Company

  • Enterprise AI and platform engineering services
  • Iselin, New Jersey, USA
  • https://www.intuitive.ai/
  • Founded in 2012
  • 1001-5000 employees

Qualifications

  • 6+ years of data engineering or software engineering experience, with 2+ years in senior roles building and operating production data pipelines
  • 2+ years running data pipelines in production — you've owned pipelines past the initial ship: on-call, incident response, backfills, schema migrations, and the operational grind of keeping datasets fresh and correct for real consumers
  • Strong Python — you write clear, idiomatic, well-tested Python for data pipelines, transformations, and tooling, with hands-on depth in pandas or Polars and the standard data stack (typing, packaging, pytest, virtual environments)
  • Strong SQL — complex analytical queries, window functions, CTEs, query optimisation, and the ability to read a query plan and fix a slow query
  • Notebook-driven data analysis — comfortable in Snowflake Notebooks or Jupyter for exploratory data analysis, profiling, data validation, and prototyping, and disciplined about turning notebook work into production code
  • Strong hands-on Snowflake experience — schemas, roles and access control, Snowpark Python, loading and incremental patterns, performance tuning, and cost awareness
  • Experience with dbt or an equivalent SQL transformation framework — models, tests, documentation, and environment promotion
  • Experience with a workflow orchestrator — Airflow, Dagster, Prefect, or similar — including scheduling, dependencies, retries, backfills, and alerting
  • Strong data modelling skills — dimensional and wide-table modelling, layered warehouse / medallion architectures, schema evolution, and data contracts
  • Production experience with both batch and incremental or streaming pipelines, with a working understanding of idempotency, deduplication, replay, late-arriving data, and recovery
  • Hands-on experience with at least one major cloud platform (AWS, GCP, or Azure) and its storage, identity, and data services
  • Experience building observable pipelines — logs, metrics, freshness checks, data-quality tests, dashboards, and actionable alerts
  • Experience delivering changes through CI/CD with automated tests for pipelines, transformations, and contracts
  • Experience using AI-assisted engineering tools as part of routine delivery, with the judgment to validate and improve generated output
  • Apache Spark or PySpark experience for large-scale batch and streaming processing
  • Databricks exposure — Delta Lake and Unity Catalog
  • Apache Kafka, Amazon Kinesis, or equivalent event-streaming platforms in production
  • Open table formats — Apache Iceberg or Delta Lake
  • Statistical analysis or data-science fundamentals — distributions, outlier detection, basic hypothesis testing — applied to data validation and investigation
  • Experience preparing datasets for machine learning, AI agents, Retrieval-Augmented Generation (RAG), or analytical APIs
  • Experience with data-quality and observability tooling such as Great Expectations, Soda, or Monte Carlo
  • Infrastructure-as-code experience (Terraform, OpenTofu, or similar) for data-platform resources
  • Experience handling PII or sensitive data — masking, tokenisation, row-level and column-level controls
  • Experience optimising Snowflake, Spark, and object-storage workloads for performance and cost
  • Experience mentoring engineers or leading technical design discussions
  • 6+ years of data engineering or software engineering experience, with 2+ years in senior roles building and operating production data pipelines.
  • 2+ years running data pipelines in production — you've owned pipelines past the initial ship: on-call, incident response, backfills, schema migrations, and the operational grind of keeping datasets fresh and correct for real consumers.

Required skills

  • AWS, GCP, or Azure, Airflow, Dagster, or Prefect, Apache Spark or PySpark, Batch and Streaming Data Pipelines, CI/CD, Data Modeling, Data Quality and Observability, Python, SQL, Snowflake, dbt, pandas or Polars

Preferred skills

  • Databricks exposure — Delta Lake and Unity Catalog, Open table formats — Apache Iceberg or Delta Lake

Experience

  • 6+ years

Job details

Employment type

Full-time

Seniority level

6+ years

Date posted

Nov 19, 2026

Work mode

Remote

Job source

ca.linkedin.com

Company

Intuitive.ai

Information Technology & AI

Enterprise AI and platform engineering services

  • Iselin, New Jersey, USA
  • Information Technology & AI
  • 1001-5000 employees
  • Founded 2012
  • intuitive.ai

Similar jobs

Matched on role, skills and location · live jobs only

See more Data Engineer jobs

Finance

1d ago

New

Data Engineer

Full-timeRemoteUnited States$140K – $190K/yr

Lakeview Loan Servicing is hiring a Data Engineer for the Nebula team to build and evolve the data foundation powering analytics, reporting, AI development, and operational decision-making.

Why similarSame roleAWS, GCP, or AzureCI/CD

Data Engineer

1d ago

New

Data Engineer | 10057

Full-timeRemoteUnited StatesCompetitive

SiTime is hiring a Data Engineer to build and maintain the data pipelines, models, and platform components powering its enterprise analytics and reporting environment.

Why similarSame roleCI/CDPython

Carrot

1d ago

New

Sr. Data Engineer

Full-timeRemoteUnited States$145K – $165K/yr

Carrot is a global fertility and family care platform that provides digitally enabled clinical guidance and support. The Sr.

Why similarSame rolePythonSQL

Get new Data Engineer jobs by email

Create job alert

Senior Data Engineer

Intuitive.ai · Competitive

Apply on company site