Data Engineer

The New York City Department of Sanitation (DSNY) is hiring a Data Engineer to support its Operations Management Division. The role designs, develops, maintains, and documents data systems and integrations, and supports analytics, reporting, and operational initiatives.

Responsibilities

Perform data analysis, data profiling, data modeling, source-to-target mapping, and reporting activities for enterprise data and analytics projects

Develop and maintain conceptual, logical, and physical data models to support current and future operational, analytical, and reporting requirements

Analyze legacy and strategic source systems to assess data quality, identify inconsistencies, and recommend corrective actions to improve data integrity and governance

Collaborate with business stakeholders, technical teams, and external partners to design, develop, and support complex data integration pipelines using ETL tools such as Databricks, Informatica PowerCenter, Pentaho Data Integration (Kettle), and similar technologies

Develop, maintain, and enhance technical documentation, including data models, data dictionaries, source-to-target mapping documents, ETL specifications, process flows, and reporting artifacts

Develop and maintain SQL-based data solutions, including complex queries, stored procedures, database objects, and data transformation processes

Support business intelligence and reporting initiatives through the development of reports, dashboards, and analytical solutions utilizing platforms such as Oracle Business Intelligence Enterprise Edition (OBIEE), Power BI, and other enterprise reporting tools

Participate in the design and implementation of innovative data architecture and integration strategies to improve data sharing and interoperability across multiple business domains within DSNY and across City agencies

Support and maintain internal IT systems, applications, and databases, ensuring reliability, performance, and adherence to agency standards

Preferred

Additional experience with pySpark

experience with Databricks is highly desirable

Experience with Microsoft .NET technologies, including ASP.NET, C#, TypeScript, React or other modern web application framework

Experience with Java development and related technologies

Familiarity with data warehousing, data governance, cloud platforms, and enterprise data integration best practices

Python and pySpark programming

Unix/Linux environments and shell scripting

Databricks and modern cloud-based data platforms

Oracle Business Intelligence Enterprise Edition (OBIEE)

Power BI, Tableau, or similar analytics platforms

C++, Java, C#, and ASP.NET development

Data warehousing, data governance, and enterprise architecture concepts

Designing and implementing enterprise-scale data integration solutions

Developing and supporting cloud and hybrid data platforms

Building automated reporting and analytics solutions

Leading cross-functional technical projects and data modernization initiatives

Qualifying experience should demonstrate substantial involvement in the analysis, design, development, implementation, administration, or management of enterprise technology solutions

Candidates should demonstrate experience involving complex technical problem-solving, systems development, data engineering, database management, application development, or enterprise analytics solutions

PySpark

Databricks

Microsoft .NET

ASP.NET

C#

TypeScript

React

Modern Web Frameworks

Java

C++

Python

Unix/Linux

Shell Scripting

Data Warehousing

Data Governance

Cloud Platforms

Cloud Data Platforms

Hybrid Data Platforms

Enterprise Data Integration

Enterprise Architecture

Data Integration Solutions

Automated Reporting

Analytics Solutions

Data Modernization

Enterprise Technology Solutions

Systems Development

Data Engineering

Database Management

Application Development

Enterprise Analytics

Technical Project Management

Technology Solution Analysis

Technology Solution Design

Technology Solution Implementation

Technology Solution Administration

Technology Solution Management

Technical Problem-Solving

Cross-Functional Leadership

Benefits

As a prospective employee of the City of New York, you may be eligible for federal loan forgiveness programs and state repayment assistance programs.

Company

New York City public health agency

Queens, New York, USA

https://www.nyc.gov/site/doh/index.page

Founded in 1866

5001-10000 employees

Qualifications

Strong proficiency in SQL and Python programming

Experience with data modeling, data analysis, ETL development, and database design

Experience with enterprise reporting and analytics platforms such as Power BI, OBIEE, Tableau, or similar tools

Knowledge of modern data platforms and cloud-based analytics environments

Database design, development, and administration

Advanced SQL programming and query optimization

Data integration, ETL/ELT development, and data transformation processes

Enterprise reporting, analytics, and business intelligence solutions

Data modeling, data profiling, and data quality analysis

A baccalaureate degree from an accredited college in computer science, engineering or a related field and four years of satisfactory full-time experience related to datacenter engineering and operations, cloud engineering and operations, complex IT infrastructure engineering

A baccalaureate degree from an accredited college and eight years of satisfactory full-time experience related to datacenter engineering and operations, cloud engineering and operations, complex IT infrastructure engineering

Education and/or experience which is equivalent to "1" or "2" above

Additional experience with pySpark

Experience with Databricks is highly desirable

Experience with Microsoft .NET technologies, including ASP.NET, C#, TypeScript, React or other modern web application framework

Experience with Java development and related technologies

Familiarity with data warehousing, data governance, cloud platforms, and enterprise data integration best practices

Python and pySpark programming

Unix/Linux environments and shell scripting

Databricks and modern cloud-based data platforms

Oracle Business Intelligence Enterprise Edition (OBIEE)

Power BI, Tableau, or similar analytics platforms

C++, Java, C#, and ASP.NET development

Data warehousing, data governance, and enterprise architecture concepts

Designing and implementing enterprise-scale data integration solutions

Developing and supporting cloud and hybrid data platforms

Building automated reporting and analytics solutions

Leading cross-functional technical projects and data modernization initiatives

Qualifying experience should demonstrate substantial involvement in the analysis, design, development, implementation, administration, or management of enterprise technology solutions

Candidates should demonstrate experience involving complex technical problem-solving, systems development, data engineering, database management, application development, or enterprise analytics solutions

Required skills

SQL, Python, Data Modeling, ETL/ELT Development, Database Design and Administration, Data Analysis and Profiling, Databricks, Power BI, Oracle Business Intelligence Enterprise Edition (OBIEE), Tableau, PySpark

Preferred skills

Additional experience with pySpark, experience with Databricks is highly desirable, Experience with Java development and related technologies, Python and pySpark programming, Unix/Linux environments and shell scripting, Databricks and modern cloud-based data platforms, Oracle Business Intelligence Enterprise Edition (OBIEE), Power BI, Tableau, or similar analytics platforms, C++, Java, C#, and ASP.NET development, Developing and supporting cloud and hybrid data platforms, Building automated reporting and analytics solutions

Experience

4+ years