← Back to jobsJob ID col-cmwp69hjq06l9d8ha19n39nm8

eGrove SystemsPosted today

Data EngineerNew

Full-time
Hybrid
6+ years
Competitive
Posted today

eGrove Systems is seeking a Data Engineer to build, operate, and optimize scalable data pipelines in Databricks and AWS. The role focuses on data ingestion and quality, pipeline reliability and performance, and collaboration with data science, analytics, and business stakeholders.

Posted
Today
Source
egrove-systems-corporation.careerplug.com
Sector
Information Technology & AI

Opens the employer's application page in a new tab

Responsibilities

  • Design, build, and optimize robust, scalable pipelines in Databricks (PySpark, SQL, Delta Lake) for structured, semi-structured, and unstructured data
  • Ingest and process data from diverse sources: relational databases, APIs, PDFs, Excel, flat files, and web scraping
  • Implement data quality frameworks, validation checks, and automated tests to ensure reliability across the pipeline lifecycle
  • Conduct UAT/UAD processes to align outputs with business requirements and ensure trust in data
  • Own pipelines end-to-end—from development through deployment, monitoring, scaling, and continuous improvement
  • Apply best practices for dataset versioning, reproducibility, and lineage (Delta Lake, MLflow, or equivalent)
  • Optimize pipelines for large-scale performance and cost-efficiency in Databricks and AWS environments
  • Collaborate with data science and analytics teams to deliver model-ready datasets and ensure schema stability
  • Produce clear technical documentation (design docs, lineage diagrams, data dictionaries) and communicate effectively with technical and business stakeholders

Qualifications

  • 6+ years of professional experience in Data Engineering or related roles
  • Hands-on expertise in Databricks (PySpark, SQL, Delta Lake)
  • Proven track record of building and deploying end-to-end production pipelines at scale
  • Experience working with structured and unstructured data, including extracting information from PDFs, Excel, web scraping, and APIs
  • Strong foundation in data quality, testing, and observability (e.g., Great Expectations, Deequ, dbt tests, or similar)
  • Ability to troubleshoot, debug, and optimize Spark/Databricks workloads for scale and performance

Required skills

API Data ExtractionData Pipeline EngineeringData Quality TestingDatabricksDeequDelta LakeGreat ExpectationsPySparkSQLWeb Scrapingdbt Tests

Full job description

eGrove Systems is seeking a Data Engineer to build, operate, and optimize scalable data pipelines in Databricks and AWS. The role focuses on data ingestion and quality, pipeline reliability and performance, and collaboration with data science, analytics, and business stakeholders.

Responsibilities

  • Design, build, and optimize robust, scalable pipelines in Databricks (PySpark, SQL, Delta Lake) for structured, semi-structured, and unstructured data
  • Ingest and process data from diverse sources: relational databases, APIs, PDFs, Excel, flat files, and web scraping
  • Implement data quality frameworks, validation checks, and automated tests to ensure reliability across the pipeline lifecycle
  • Conduct UAT/UAD processes to align outputs with business requirements and ensure trust in data
  • Own pipelines end-to-end—from development through deployment, monitoring, scaling, and continuous improvement
  • Apply best practices for dataset versioning, reproducibility, and lineage (Delta Lake, MLflow, or equivalent)
  • Optimize pipelines for large-scale performance and cost-efficiency in Databricks and AWS environments
  • Collaborate with data science and analytics teams to deliver model-ready datasets and ensure schema stability
  • Produce clear technical documentation (design docs, lineage diagrams, data dictionaries) and communicate effectively with technical and business stakeholders

Benefits

  • Flexible work from home options available.

Company

  • IT solutions and services for e-commerce businesses
  • East Brunswick, New Jersey, USA
  • https://www.egrovesys.com/
  • Founded in 2007
  • 51-200 employees

Qualifications

  • 6+ years of professional experience in Data Engineering or related roles
  • Hands-on expertise in Databricks (PySpark, SQL, Delta Lake)
  • Proven track record of building and deploying end-to-end production pipelines at scale
  • Experience working with structured and unstructured data, including extracting information from PDFs, Excel, web scraping, and APIs
  • Strong foundation in data quality, testing, and observability (e.g., Great Expectations, Deequ, dbt tests, or similar)
  • Ability to troubleshoot, debug, and optimize Spark/Databricks workloads for scale and performance
  • 6+ years of professional experience in Data Engineering or related roles.

Required skills

  • API Data Extraction, Data Pipeline Engineering, Data Quality Testing, Databricks, Deequ, Delta Lake, Great Expectations, PDF and Excel Data Extraction, PySpark, SQL, Web Scraping, dbt Tests

Experience

  • 6+ years

Job details

Employment type

Full-time

Seniority level

6+ years

Date posted

Nov 20, 2026

Work mode

Hybrid

Job source

egrove-systems-corporation.careerplug.com

Company

eGrove Systems

Information Technology & AI

IT solutions and services for e-commerce businesses

  • East Brunswick, New Jersey, USA
  • Information Technology & AI
  • 51-200 employees
  • Founded 2007
  • egrovesys.com

Similar jobs

Matched on role, skills and location · live jobs only

See more Data Engineer jobs in New York

Gemini

Today

New

Senior Data Engineer

Full-timeHybridNew York, NY$126K – $180K/yr

Gemini is a global crypto and Web3 platform offering a range of crypto products and services. As a Senior Data Engineer, you will design and operate data infrastructure, mentor junior engineers, and build high-scale systems that impact various teams within…

Why similarSame roleDatabricksNew York

NTT DATA

Today

New

Senior Data Engineering - Support Engineer

Full-timeRemoteUnited States$92K – $138K/yr

NTT DATA is hiring a Senior Data Engineering Support Engineer to anchor enterprise big data operations and maintain the availability, performance, and cost efficiency of production data pipelines.

Why similarSame roleDatabricksRemote

Flow Traders

Today

New

Core Data Engineer

Full-timeOn-siteNew York, NY$165K – $210K/yr

Flow Traders is hiring a Core Data Engineer to own the lifecycle and quality of foundational data products, including market, trading, and reference data.

Why similarSame roleDatabricksNew York

Get new Data Engineer jobs in New York by email

Create job alert

Data Engineer

eGrove Systems · Competitive

Apply on company site