Data Engineer – Cloud Data Pipelines, ETL/ELT, SQL & Python

Onsite IT Advisors is seeking a Data Engineer to design, develop, optimize, and maintain cloud-based data pipelines and storage solutions. The role transforms diverse datasets into analysis-ready formats, validates data quality, and supports reporting, analytics, and visualization in collaboration with cross-functional teams.

Responsibilities

Design, develop, optimize, and maintain scalable ETL/ELT pipelines to ingest, transform, and deliver structured and unstructured data from multiple cloud-based sources

Build, manage, and optimize data storage architectures, including data warehouses, data lakes, and lakehouses, to support large-scale analytics, reporting, and business intelligence requirements

Perform backend SQL and NoSQL database testing, including data validation, post-deployment verification, troubleshooting, and resolution of data ingestion, data integrity, and schema-related issues

Collaborate with cross-functional teams, including analysts, engineers, product teams, and business stakeholders, to translate requirements into technical data models, workflows, architecture decisions, and pipeline specifications

Develop scripts and automation to ingest, parse, transform, and convert diverse data types into usable, analysis-ready formats

Scale, monitor, troubleshoot, and maintain cloud-based data platforms to ensure reliable data processing, pipeline performance, and data quality

Support end-to-end data engineering solutions that enable reporting, analytical modeling, business intelligence, and data visualization

Leverage generative AI coding tools to assist with development, improve engineering productivity, and support efficient implementation of data solutions

Preferred

Experience developing solutions within collaborative, cross-functional team environments

Experience with AWS Glue, AWS Athena, and PostgreSQL

Experience with distributed data processing and computing technologies, such as Apache Spark, Databricks, Apache Hive, AWS EMR, or Apache Kafka

Experience developing or supporting real-time data processing and streaming applications

Experience implementing NoSQL databases, including MongoDB or Apache Cassandra

Experience with data warehousing technologies, including AWS Redshift, MySQL, or Snowflake

Experience working in UNIX and Linux environments, including basic command-line operations and shell scripting

Experience working in Agile engineering environments and following Agile development practices

AWS Glue

AWS Athena

PostgreSQL

Distributed Data Processing

Apache Spark

Databricks

Apache Hive

AWS EMR

Apache Kafka

Real-Time Data Processing

Streaming Applications

NoSQL Databases

MongoDB

Apache Cassandra

Data Warehousing

AWS Redshift

MySQL

Snowflake

UNIX

Linux

Command-Line Operations

Shell Scripting

Agile Engineering

Agile Development Practices

Cross-Functional Collaboration

Teamwork

Company

Onsite IT Advisors offers IT staffing, recruitment services, and IT consulting, helping businesses enhance their applications.

Alpharetta, Georgia, USA

https://www.onsiteitadvisors.com/

Founded in 2018

11-50 employees

Qualifications

Bachelor's degree

Must be a U.S. Citizen or Lawful Permanent Resident (Green Card Holder)

Ability to obtain and maintain a Public Trust or Suitability/Fitness determination, based on client requirements

At least 3 years of experience in data warehouse development

At least 3 years of experience analyzing aggregated data from multiple sources in cloud-based environments

At least 2 years of experience developing and maintaining scalable data stores that provide large volumes of data in formats suitable for business analysis

Experience using SQL and Python to retrieve, parse, and process structured and unstructured data

Experience designing and developing scalable ETL/ELT workflows to support reporting and analytics using cloud-based technologies

Experience using Cloudera or Hadoop technologies to ingest, transform, and process data

Experience leveraging generative AI coding tools to assist with software and data engineering development

Ability to perform backend SQL and NoSQL database testing, including data validation and post-deployment testing

Experience developing solutions within collaborative, cross-functional team environments

Experience with AWS Glue, AWS Athena, and PostgreSQL

Experience with distributed data processing and computing technologies, such as Apache Spark, Databricks, Apache Hive, AWS EMR, or Apache Kafka

Experience developing or supporting real-time data processing and streaming applications

Experience implementing NoSQL databases, including MongoDB or Apache Cassandra

Experience with data warehousing technologies, including AWS Redshift, MySQL, or Snowflake

Experience working in UNIX and Linux environments, including basic command-line operations and shell scripting

Experience working in Agile engineering environments and following Agile development practices

At least 3 years of experience in data warehouse development.

At least 3 years of experience analyzing aggregated data from multiple sources in cloud-based environments.

At least 2 years of experience developing and maintaining scalable data stores that provide large volumes of data in formats suitable for business analysis.

Required skills

Represents the skills you have, click on the tags, SQL, ETL/ELT, Data Warehousing, Cloud Data Engineering, Hadoop, SQL and NoSQL Database Testing, Public Trust Clearance, AWS Glue, AWS Athena, PostgreSQL

Preferred skills

Experience with AWS Glue, AWS Athena, and PostgreSQL, Alpharetta, Georgia, USA, https://www.onsiteitadvisors.com, Leadership Team, Seth Nachiappan----- F., Founder & CEO, Company data provided by, AI Tools, Customize Your Resume, Maximize your interview chances, Build Cover Letter, Make your application stand out, Analyze How Well You Fit, Understand your strength & weakness

Experience

3+ years