← Back to jobsJob ID col-cmwpbruhi09p9d8hatl34j36s

DicePosted 1 day ago

Senior Data Engineer - DatabricksNew

Full-time
On-site
7+ years
$84K – $156K/yr
Posted 1 day ago

Capgemini is seeking a Senior Databricks Data Engineer to design, build, and optimize scalable cloud-based data platforms and pipelines. The role focuses on reliable data solutions that support analytics, AI, and business intelligence, working closely with architects, analysts, and business stakeholders.

Consulting

Posted
1 day ago
Source
dice.com
Sector
Information Technology & AI

Opens the employer's application page in a new tab

Responsibilities

  • Design, develop, and maintain data pipelines using Databricks, Apache Spark, and Delta Lake
  • Build and optimize ETL/ELT processes for ingesting structured and unstructured data from multiple sources
  • Develop and maintain notebooks using PySpark, Scala, or SQL for data transformation and analysis
  • Implement Delta Lake architectures (bronze, silver, gold layers) following medallion architecture best practices
  • Collaborate with data architects, analysts, and business stakeholders to translate requirements into technical solutions
  • Optimize Spark jobs for performance, including cluster configuration, partitioning, and caching strategies
  • Implement data quality checks, validation frameworks, and monitoring for pipeline reliability
  • Integrate Databricks with cloud platforms (AWS)
  • Work with CI/CD pipelines for automated deployment of notebooks and workflows
  • Implement version control and code review practices using Git-based workflows

Qualifications

  • Bachelor's degree in Computer Science, Information Systems, Engineering, or a related discipline
  • 7+ years of experience in Data Engineering, Data Warehousing, or Big Data technologies
  • 3+ years of hands-on experience with **Databricks** and **Apache Spark**
  • Strong expertise in **PySpark**, SQL, and data transformation frameworks
  • Experience implementing **Delta Lake** and Medallion Architecture patterns
  • Strong understanding of distributed computing concepts and Spark performance optimization
  • Experience developing and supporting data solutions in **AWS Cloud** environments
  • Hands-on experience with CI/CD tools and DevOps practices
  • Proficiency with Git-based version control systems
  • Knowledge of data modeling, ETL/ELT design patterns, and modern data lakehouse architectures
  • Strong communication and stakeholder management skills

Required skills

AWSApache SparkCI/CDData ModelingDatabricksDelta LakeDistributed ComputingETL/ELTGitPySparkSQLSpark Performance OptimizationStakeholder Management

Full job description

Capgemini is seeking a Senior Databricks Data Engineer to design, build, and optimize scalable cloud-based data platforms and pipelines. The role focuses on reliable data solutions that support analytics, AI, and business intelligence, working closely with architects, analysts, and business stakeholders.

Consulting

  • Human Resources
  • Recruiting
  • Staffing Agency

Responsibilities

  • Design, develop, and maintain data pipelines using Databricks, Apache Spark, and Delta Lake
  • Build and optimize ETL/ELT processes for ingesting structured and unstructured data from multiple sources
  • Develop and maintain notebooks using PySpark, Scala, or SQL for data transformation and analysis
  • Implement Delta Lake architectures (bronze, silver, gold layers) following medallion architecture best practices
  • Collaborate with data architects, analysts, and business stakeholders to translate requirements into technical solutions
  • Optimize Spark jobs for performance, including cluster configuration, partitioning, and caching strategies
  • Implement data quality checks, validation frameworks, and monitoring for pipeline reliability
  • Integrate Databricks with cloud platforms (AWS)
  • Work with CI/CD pipelines for automated deployment of notebooks and workflows
  • Implement version control and code review practices using Git-based workflows

Benefits

  • Paid time off based on employee grade (A-F), defined by policy: Vacation: 12-25 days, depending on grade
  • Company paid holidays
  • Personal Days
  • Sick Leave
  • Medical, dental, and vision coverage (or provincial healthcare coordination in Canada)
  • Retirement savings plans (e.g., 401(k) in the U.S., RRSP in Canada)
  • Life and disability insurance
  • Employee assistance programs
  • This role may be eligible for additional compensation such as variable incentives, bonuses, or commissions, depending on the position and applicable laws.

Company

  • Tech career marketplace connecting professionals with recruiters.
  • Centennial, Colorado, USA
  • https://www.or-quest.nl/

Qualifications

  • Bachelor's degree in Computer Science, Information Systems, Engineering, or a related discipline
  • 7+ years of experience in Data Engineering, Data Warehousing, or Big Data technologies
  • 3+ years of hands-on experience with **Databricks** and **Apache Spark**
  • Strong expertise in **PySpark**, SQL, and data transformation frameworks
  • Experience implementing **Delta Lake** and Medallion Architecture patterns
  • Strong understanding of distributed computing concepts and Spark performance optimization
  • Experience developing and supporting data solutions in **AWS Cloud** environments
  • Hands-on experience with CI/CD tools and DevOps practices
  • Proficiency with Git-based version control systems
  • Knowledge of data modeling, ETL/ELT design patterns, and modern data lakehouse architectures
  • Strong communication and stakeholder management skills
  • 7+ years of experience in Data Engineering, Data Warehousing, or Big Data technologies.
  • 3+ years of hands-on experience with **Databricks** and **Apache Spark**.

Required skills

  • AWS, Apache Spark, CI/CD, Data Modeling, Databricks, Delta Lake, Distributed Computing, ETL/ELT, Git, PySpark, SQL, Spark Performance Optimization, Stakeholder Management

Experience

  • 7+ years

Job details

Employment type

Full-time

Seniority level

7+ years

Date posted

Nov 20, 2026

Work mode

On-site

Job source

dice.com

Company

Dice

Information Technology & AI

Tech career marketplace connecting professionals with recruiters.

See all 301 jobs from Dice →
  • Centennial, Colorado, USA
  • Information Technology & AI
  • or-quest.nl

Similar jobs

Matched on role, skills and location · live jobs only

See more Data Engineer jobs in New York

Capgemini

1d ago

New

Senior Data Engineer - Databricks

Full-timeOn-siteNew York, NY$84K – $156K/yr

Capgemini is seeking a Senior Databricks Data Engineer to design, build, and optimize scalable data platforms and pipelines in cloud-based environments.

Why similarSame roleAWSNew York

M Science

1d ago

New

Data Engineer

Full-timeOn-siteNew York, NY$110K – $150K/yr

M Science is hiring a Data Engineer to build and improve large-scale data pipelines and datasets for analytics, experimentation, and machine learning.

Why similarSame roleAWSNew York

#Data Engineering Opportunities

1d ago

New

#Data Engineering Opportunities | Data Engineer, Senior & Lead

Full-timeOn-siteNew York, NY$140K – $190K/yr

Lakeview Loan Servicing, LLC is hiring Data Engineers at multiple levels to build and evolve the data foundation supporting analytics, reporting, AI, products, and operational decisions.

Why similarSame roleAWSNew York

Get new Data Engineer jobs in New York by email

Create job alert

Senior Data Engineer - Databricks

Dice · $84K – $156K/yr

Apply on company site