← Back to jobsJob ID col-cmv8d2fu703i3oz010hjeo2o1

Cyber SecurityPosted 30+ days ago

Data Engineer - AWS/Databricks

Full-time
On-site
2+ years
$90K – $100K/yr
Posted 30+ days ago

Responsibilities Build and maintain scalable PySpark-based data pipelines in Databricks notebooks to support ingestion, transformation, and enrichment of structured and semi-structured data Design and implement Delta Lake tables optimized for ACID compliance, partition pruning, schema enforcement, and query performance across large datasets Develop ETL and ELT workflows that integrate multiple source systems into a c

Posted
30+ Days Ago
Source
Ethic Leaf recruiter
Sector
Information Technology & AI

No employer link yet — an Ethic Leaf recruiter forwards your application

Responsibilities

  • Build and maintain scalable PySpark-based data pipelines in Databricks notebooks to support ingestion, transformation, and enrichment of structured and semi-structured data
  • Design and implement Delta Lake tables optimized for ACID compliance, partition pruning, schema enforcement, and query performance across large datasets
  • Develop ETL and ELT workflows that integrate multiple source systems into a centralized, query-optimized data warehouse architecture
  • Leverage Spark SQL and DataFrame APIs to implement business rules, dimensional joins, and aggregation logic aligned to warehouse modeling best practices
  • Collaborate with data architects and engineers to implement cloud-native data solutions on AWS using S3, Glue, RDS, and IAM for secure, scalable storage and access control
  • Optimize pipeline performance through intelligent partitioning, caching, broadcast joins, and adaptive query tuning
  • Deploy and version data engineering assets using Git-integrated development workflows and automate deployment with CI/CD tools such as GitLab or Jenkins
  • Monitor pipeline health, job execution, and cluster utilization using native Databricks tools and AWS CloudWatch, identifying bottlenecks and optimizing cost-performance tradeoffs
  • Conduct technical discovery and mapping of legacy source systems, identifying required transformations and designing end-to-end data flows
  • Implement governance practices including metadata tagging, data quality validation, audit logging, and lineage tracking using platform-native features and custom logic
  • Support ad hoc data access requests, develop reusable data assets, and maintain shared notebooks that meet operational reporting and analytics needs across teams

Qualifications

  • 2+ years of experience in data engineering and Agile analytics
  • 2+ years of experience creating software for retrieving, parsing and processing structured and unstructured data
  • 1 to 2 years of experience building scalable ETL and ELT workflows for reporting and analytics
  • Experience with data quality, validation frameworks, and storage optimization strategies
  • BA or BS degree
  • Must be US Citizen with an ability to obtain and maintain US Suitability

Preferred

  • 1 or more years experience building enterprise data engineering solutions in the cloud, with preferred experience with cloud native technologies from AWS and Databricks

Required skills

AWSDatabricksPySparkApache SparkDelta LakeETL/ELT WorkflowsData WarehousingSpark SQLAWS S3AWS GlueAWS RDSAWS IAMGitCI/CD

Full job description

Responsibilities

  • Build and maintain scalable PySpark-based data pipelines in Databricks notebooks to support ingestion, transformation, and enrichment of structured and semi-structured data
  • Design and implement Delta Lake tables optimized for ACID compliance, partition pruning, schema enforcement, and query performance across large datasets
  • Develop ETL and ELT workflows that integrate multiple source systems into a centralized, query-optimized data warehouse architecture
  • Leverage Spark SQL and DataFrame APIs to implement business rules, dimensional joins, and aggregation logic aligned to warehouse modeling best practices
  • Collaborate with data architects and engineers to implement cloud-native data solutions on AWS using S3, Glue, RDS, and IAM for secure, scalable storage and access control
  • Optimize pipeline performance through intelligent partitioning, caching, broadcast joins, and adaptive query tuning
  • Deploy and version data engineering assets using Git-integrated development workflows and automate deployment with CI/CD tools such as GitLab or Jenkins
  • Monitor pipeline health, job execution, and cluster utilization using native Databricks tools and AWS CloudWatch, identifying bottlenecks and optimizing cost-performance tradeoffs
  • Conduct technical discovery and mapping of legacy source systems, identifying required transformations and designing end-to-end data flows
  • Implement governance practices including metadata tagging, data quality validation, audit logging, and lineage tracking using platform-native features and custom logic
  • Support ad hoc data access requests, develop reusable data assets, and maintain shared notebooks that meet operational reporting and analytics needs across teams

Preferred

  • 1 or more years experience building enterprise data engineering solutions in the cloud, with preferred experience with cloud native technologies from AWS and Databricks

Qualifications

  • 2+ years of experience in data engineering and Agile analytics
  • 2+ years of experience creating software for retrieving, parsing and processing structured and unstructured data
  • 1 to 2 years of experience building scalable ETL and ELT workflows for reporting and analytics
  • Experience with data quality, validation frameworks, and storage optimization strategies
  • BA or BS degree
  • Must be US Citizen with an ability to obtain and maintain US Suitability
  • 1 or more years experience building enterprise data engineering solutions in the cloud, with preferred experience with cloud native technologies from AWS and Databricks

Required skills

  • AWS, Databricks, PySpark, Apache Spark, Delta Lake, ETL/ELT Workflows, Data Warehousing, Spark SQL, AWS S3, AWS Glue, AWS RDS, AWS IAM, Git, CI/CD, Data Quality Validation

Experience

  • 2+ years

Company

  • Acuity is an information technology company that offers cybersecurity and project management services.
  • Reston, Virginia, USA
  • http://myacuity.com

Job details

Employment type

Full-time

Seniority level

2+ years

Date posted

Oct 9, 2026

Work mode

On-site

Job source

Ethic Leaf recruiter

Company

Cyber Security

Information Technology & AI

Acuity is an information technology company that offers cybersecurity and project management services.

See all 5 jobs from Cyber Security →
  • Reston, Virginia, USA
  • Information Technology & AI
  • myacuity.com

Similar jobs

Matched on role, skills and location · live jobs only

See more Data Engineer jobs in United States

SoftStandard Solutions

1d ago

New

Data Engineer

TemporaryOn-siteUnited StatesCompetitive

SoftStandard Solutions is seeking a Data Engineer to build and maintain scalable data pipelines and ETL/ELT processes.

Why similarSame roleAWSApache Spark

Meritore Technologies

1d ago

New

AWS Data Engineer

Full-timeOn-siteTorrance, CACompetitive

Meritore Technologies is hiring an AWS Data Engineer to build and operate scalable AWS data pipelines and cloud-based data lake and warehouse solutions.

Why similarSame roleAWS GlueAWS IAM

Capgemini

1d ago

New

Senior Data Engineer - Databricks

Full-timeOn-siteNew York, NY$84K – $156K/yr

Capgemini is seeking a Senior Databricks Data Engineer to design, build, and optimize scalable data platforms and pipelines in cloud-based environments.

Why similarSame roleAWSApache Spark

Get new Data Engineer jobs in United States by email

Create job alert

Data Engineer - AWS/Databricks

Cyber Security · $90K – $100K/yr

Apply via Ethic Leaf