Data Engineer (remote US)

Responsibilities

• Design, develop, and maintain data pipelines using Microsoft Fabric, Azure Data Factory, and related Azure technologies
• Develop Python, PySpark, and SQL notebooks for data ingestion, transformation, and processing across Bronze, Silver, and Gold layers
• Integrate data from multiple sources, including files, databases, APIs, and enterprise applications
• Implement data transformations, enrichment, aggregations, and business rules based on approved specifications
• Optimize pipelines and SQL queries for performance, reliability, and scalability
• Develop and maintain enterprise Data Products in collaboration with Data Governance
• Maintain on-premises SQL Server databases, stored procedures, and ETL processes while supporting their migration to Azure and Microsoft Fabric
• Develop and maintain automated, rule-based processing workflows, including queue-driven processing and batch orchestration
• Manage file-based ingestion, feed tracking, processing status, and downstream data distribution
• Implement data validation, reconciliation, logging, monitoring, and exception handling
• Monitor production pipelines, troubleshoot failures and data discrepancies, and implement corrective actions
• Ensure continuity, reliability, and timely delivery of business-critical data processes
• Collaborate with business stakeholders and Data Governance to understand requirements, data definitions, business rules, and approved Data Product specifications
• Translate business requirements into technical designs, identifying source data gaps, dependencies, and process improvements
• Maintain technical documentation, including data flows, source-to-target mappings, transformation logic, and operational procedures
• Perform unit and integration testing, support QA and business sign-off, and adhere to established development, deployment, and Data Governance standards

Preferred

• Familiarity with SSIS or legacy ETL tools is an asset
• Experience modernizing legacy data platforms and relevant Microsoft certifications are considered assets
• cover letters are strongly preferred

Benefits

• A comprehensive benefits package with a variety of physical health, retirement and savings, caregiving, emotional wellbeing, and transportation benefits.
• Elective benefits employees may select to best fit their needs and personal situations.
• Glassdoor
• 3.4
• HarperCollins Publishers is the second-largest consumer book publisher in the world.
• New York, New York, USA
• https://www.harpercollins.com/
• Distribution of Different Job Fields Receiving Sponsorship
• Represents job field similar to this job
• Engineering and Development
• Trends of Total Sponsorships
• *2026
• (1)
• 2023
• 2020
• Late Stage
• Leadership Team
• Charlie Redmayne
• CEO, HarperCollins UK
• Lisa Sharkey
• SVP, Publisher Harper Influence at HarperCollins
• Recent News
• PR Newswire
• In The Way Of Wealth, Billionaire Entrepreneur Michael Lee-Chin Reveals The Strategies, Sacrifices and Struggles Behind His Success
• 2026-10-23
• Times Now
• Jeffrey Archer Is Dead: The Best-Selling Author Who Once Said Bollywood Copied His Work
• 2026-10-07
• Cyprus Mail
• UK author and politician Jeffrey Archer dies aged 86

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, Information Technology, or a related technical discipline
• 3–5+ years of hands-on experience in data engineering, database development, or enterprise data integration in building pipelines, migrating data and modernization of environments
• Strong proficiency in SQL Server, T-SQL, stored procedures, query optimization, and ETL/ELT development
• Hands-on experience with Azure Data Factory, Azure Data Lake Storage, and Microsoft Fabric, including Data Pipelines, Lakehouse, and Notebooks
• Proficiency in Python and/or PySpark, with an understanding of data warehousing, relational data modeling, and Medallion Architecture
• Experience with file-based integrations, batch processing, queue-driven workflows, data validation, monitoring, troubleshooting, and production support
• Strong communication and collaboration skills, with the ability to engage business stakeholders, Data Governance, and technical teams to translate requirements into effective solutions
• Strong analytical and problem-solving abilities, with attention to data accuracy, quality, and operational reliability
• Familiarity with SSIS or legacy ETL tools is an asset
• Experience modernizing legacy data platforms and relevant Microsoft certifications are considered assets
• Cover letters are strongly preferred
• 3–5+ years of hands-on experience in data engineering, database development, or enterprise data integration in building pipelines, migrating data and modernization of environments.

Required skills

• Azure Data Factory, Azure Data Lake Storage, Data Warehousing, ETL/ELT Development, Medallion Architecture, Microsoft Fabric, PySpark, Python, Relational Data Modeling, SQL Server, SSIS, T-SQL

Preferred skills

• Familiarity with SSIS or legacy ETL tools is an asset, cover letters are strongly preferred

Experience

• 3+ years