← Back to jobsJob ID col-cmwp7ectq07a9d8haux310yfc

DicePosted 1 day ago

Senior Data Engineer with AINew

Contract
Hybrid
Competitive
Posted 1 day ago

The hiring employer is not identified in the posting. The role builds and maintains scalable data infrastructure across ingestion, transformation, orchestration, and delivery, while using AI-assisted development tools and collaborating with backend, BI, and ML teams.

Posted
1 day ago
Source
dice.com
Sector
Information Technology & AI

Opens the employer's application page in a new tab

Responsibilities

  • Design, build, and maintain scalable data ingestion and processing pipelines using SQL , Spark, Hadoop, Airflow, SnapLogic, or similar tools
  • Optimize existing ETL/ELT workflows and resolve bottlenecks in production jobs
  • Integrate multiple structured and unstructured data sources from internal and external systems
  • Design and implement event-driven data pipelines using event bus architectures (Kafka, AWS EventBridge, SNS/SQS, or similar)
  • Build and maintain event producers, consumers, and stream processors for real-time and near-real-time data flows
  • Manage event schema design, schema registry, and versioning to ensure backward compatibility across consumers
  • Monitor event bus health, lag, throughput, and reliability; respond to failures before SLA breach
  • Implement and manage CI/CD pipelines for data workflows using Git, Jenkins, or GitHub Actions
  • Ensure robust job scheduling, monitoring, logging, and end-to-end testing of data pipelines
  • Debug and resolve production issues before SLA breach; participate in on-call rotation
  • Collaborate with backend and BI teams for API and data integration
  • Design and develop schema definitions; support data warehouse/mart to integrate disparate data sources and make them available for analysis
  • Support large data volumes and flexible provisioning of new data sources
  • Contribute to data architecture design across the data landscape
  • Write clean, reusable, well-tested Python and Shell code across the full software development lifecycle
  • Gather functional requirements, develop technical specifications, and own project and test planning
  • Work in Agile/Scrum environments; help align team priorities across the organization
  • Leverage AI-assisted dev tools (Claude, Cursor .. etc) to accelerate delivery and code quality

Qualifications

  • Strong Python, Advanced SQL, PySpark or Scala, Unix scripting
  • Data orchestration: Airflow, SnapLogic, DBT, or equivalent
  • Event bus & streaming: Kafka, AWS EventBridge, SNS/SQS, Kinesis, or similar
  • Event-driven architecture: producers, consumers, schema registry, dead-letter queues
  • AWS cloud: EC2, EMR, S3, Athena, Glue; Databricks
  • Data lake & warehouse: Snowflake, Delta Lake, Redshift, BigQuery
  • CI/CD tooling: Git, Jenkins / GitHub Actions, Docker
  • Experience with APIs for data exchange and integration
  • Data quality mindset, ownership, and SLA awareness
  • Agile/Scrum development experience
  • SQL, AWS, PySpark, Data Engineering, Python PySpark, ETL & Data Analytics, Data Migration, Confluence, Jira

Preferred

  • NoSQL, Neo4j, GraphQL is plus
  • Experience with AI/ML models and feature pipelines
  • Exposure to BI platforms (Tableau, Power BI) or data serving for dashboards
  • Spark Streaming or Flink for stateful stream processing

Required skills

AI/MLAWSAirflowDatabricksDockerKafkaPySparkPythonSQLScalaSnowflakeUnix Scripting

Full job description

The hiring employer is not identified in the posting. The role builds and maintains scalable data infrastructure across ingestion, transformation, orchestration, and delivery, while using AI-assisted development tools and collaborating with backend, BI, and ML teams.

Responsibilities

  • Design, build, and maintain scalable data ingestion and processing pipelines using SQL , Spark, Hadoop, Airflow, SnapLogic, or similar tools
  • Optimize existing ETL/ELT workflows and resolve bottlenecks in production jobs
  • Integrate multiple structured and unstructured data sources from internal and external systems
  • Design and implement event-driven data pipelines using event bus architectures (Kafka, AWS EventBridge, SNS/SQS, or similar)
  • Build and maintain event producers, consumers, and stream processors for real-time and near-real-time data flows
  • Manage event schema design, schema registry, and versioning to ensure backward compatibility across consumers
  • Monitor event bus health, lag, throughput, and reliability; respond to failures before SLA breach
  • Implement and manage CI/CD pipelines for data workflows using Git, Jenkins, or GitHub Actions
  • Ensure robust job scheduling, monitoring, logging, and end-to-end testing of data pipelines
  • Debug and resolve production issues before SLA breach; participate in on-call rotation
  • Collaborate with backend and BI teams for API and data integration
  • Design and develop schema definitions; support data warehouse/mart to integrate disparate data sources and make them available for analysis
  • Support large data volumes and flexible provisioning of new data sources
  • Contribute to data architecture design across the data landscape
  • Write clean, reusable, well-tested Python and Shell code across the full software development lifecycle
  • Gather functional requirements, develop technical specifications, and own project and test planning
  • Work in Agile/Scrum environments; help align team priorities across the organization
  • Leverage AI-assisted dev tools (Claude, Cursor .. etc) to accelerate delivery and code quality

Preferred

  • NoSQL, Neo4j, GraphQL is plus
  • Experience with AI/ML models and feature pipelines
  • Exposure to BI platforms (Tableau, Power BI) or data serving for dashboards
  • Spark Streaming or Flink for stateful stream processing
  • NoSQL
  • Neo4j
  • GraphQL
  • AI/ML Models
  • Feature Pipelines
  • BI Platforms
  • Tableau
  • Power BI
  • Dashboard Data Serving
  • Spark Streaming
  • Flink
  • Stateful Stream Processing

Benefits

  • Hybrid work in Dallas, TX

Company

  • Tech career marketplace connecting professionals with recruiters
  • Centennial, Colorado, USA
  • https://www.or-quest.nl/
  • Founded in 1990
  • 201-500 employees

Qualifications

  • Strong Python, Advanced SQL, PySpark or Scala, Unix scripting
  • Data orchestration: Airflow, SnapLogic, DBT, or equivalent
  • Event bus & streaming: Kafka, AWS EventBridge, SNS/SQS, Kinesis, or similar
  • Event-driven architecture: producers, consumers, schema registry, dead-letter queues
  • AWS cloud: EC2, EMR, S3, Athena, Glue; Databricks
  • Data lake & warehouse: Snowflake, Delta Lake, Redshift, BigQuery
  • CI/CD tooling: Git, Jenkins / GitHub Actions, Docker
  • Experience with APIs for data exchange and integration
  • Data quality mindset, ownership, and SLA awareness
  • Agile/Scrum development experience
  • SQL, AWS, PySpark, Data Engineering, Python PySpark, ETL & Data Analytics, Data Migration, Confluence, Jira
  • NoSQL, Neo4j, GraphQL is plus
  • Experience with AI/ML models and feature pipelines
  • Exposure to BI platforms (Tableau, Power BI) or data serving for dashboards
  • Spark Streaming or Flink for stateful stream processing

Required skills

  • AI/ML, AWS, Airflow, Databricks, Docker, Kafka, PySpark, Python, SQL, Scala, Snowflake, Unix Scripting

Preferred skills

  • Experience with AI/ML models and feature pipelines, NoSQL, Neo4j, GraphQL is plus, Spark Streaming or Flink for stateful stream processing

Job details

Employment type

Contract

Seniority level

Not stated

Date posted

Nov 20, 2026

Work mode

Hybrid

Job source

dice.com

Company

Dice

Information Technology & AI

Tech career marketplace connecting professionals with recruiters

See all 301 jobs from Dice →
  • Centennial, Colorado, USA
  • Information Technology & AI
  • 201-500 employees
  • Founded 1990
  • or-quest.nl

Similar jobs

Matched on role, skills and location · live jobs only

See more Data Engineer jobs in Dallas

Senior Data Engineer (Fraud) Job Details

1d ago

New

Senior Data Engineer (Fraud)

Full-timeOn-siteRichardson, TXCompetitive

BOK Financial is hiring a Senior Data Engineer (Fraud) to design, develop, and deliver scalable data pipelines and solutions for enterprise analytics and data platform initiatives.

Why similarSame roleAWSRichardson

R Systems

1d ago

New

AI Data Engineer

Full-timeOn-siteUnited StatesCompetitive

R Systems is hiring an AI Data Engineer to design, build, and maintain data pipelines, lakehouses, and retrieval infrastructure for production AI, machine learning, and generative AI applications.

Why similarSame roleKafkaPython

EPITEC

1d ago

New

Data Engineer

TemporaryHybridRichmond, VACompetitive

EPITEC is seeking a Data Engineer to build and maintain large-scale ETL pipelines on AWS EMR, primarily using Scala and Spark. The role also supports Python and SQL workflows, Databricks platform evolution, production support, and engineering standards.

Why similarSame roleAirflowDatabricks

Get new Data Engineer jobs in Dallas by email

Create job alert

Senior Data Engineer with AI

Dice · Competitive

Apply on company site