Data Engineer II (School of Medicine)

Emory University is a leading research university that fosters excellence and attracts world-class talent to innovate today and prepare leaders for the future. The Data Engineer II will develop and evolve a scalable Unified Data Platform on Azure Fabric, partnering with researchers, architects, and cross-functional teams. The role focuses on data pipelines, ETL/ELT processes, biomedical informatics, data governance, HIPAA compliance, and research and clinical data solutions.

Responsibilities

• Establishes collaborative relationships with subject matter experts and develops an understanding of the line of business
• Collaborate as a core member of cross-functional teams including Business Analysts, Project Managers, Data Analysts, and Architects to deliver high-quality solutions within scope and timeline
• Work directly with researchers and stakeholders to gather, analyze, and translate requirements into technical solutions
• Design, develop, and maintain scalable data pipelines, ETL/ELT processes, and data integration workflows across multiple systems
• Build and optimize data solutions that integrate data from disparate sources, ensuring data quality, integrity, and consistency
• Evaluate emerging technologies and develop proof-of-concepts to support innovation and continuous improvement
• Apply biomedical informatics standards, methodologies, and principles to research data solutions
• Ensure adherence to HIPAA and institutional data governance policies and standards
• Develop and maintain metadata, data standards, and data governance processes for complex datasets
• Create and maintain clear technical documentation, including data pipelines, workflows, and system designs
• Communicate technical concepts effectively to both technical and non-technical stakeholders
• Partner with data and system architects and demonstrate an understanding of data modeling concepts and best practices
• Manage workload effectively and provide timely updates on task progress and deliverables
• Works as a positive team member of a project that may consist of Business Analysts, Project Managers, Information Architects, Data Analysts, and/or Database Administrators to deliver quality applications and components within scope, on time, and within budget
• Manages workload effectively and report status of tasks in a timely manner
• Works directly with researchers to document, analyze, and translate their needs into technical designs and informatics solutions
• Participates in the evaluation of emerging technologies and develops proof-of-concepts
• Contributes to technical teams
• Follows standard operational procedures and HIPAA regulations
• Develops strategies for managing complex data sets through maintaining data standards and metadata
• Applies biomedical informatics technical standards, methodologies, and principles to research-specific program needs, objectives, and outcomes
• Develops complex reports, data pipelines, and ETL processes from disparate systems and ensures their accuracy
• Gathers user requirements and creates technical documentation
• Performs other related duties as required

Preferred

• Strong experience designing and maintaining data pipelines and ETL/ELT processes
• Proficiency in Python, SQL, and Apache Spark (PySpark)
• Experience with modern data platforms such as Azure Fabric (preferred), Azure Synapse Serverless, or Databricks
• Ability to work with large, complex, and distributed datasets
• Solid understanding of data modeling concepts and best practices
• Familiarity with data governance, metadata management, and data quality frameworks
• Knowledge of HIPAA and healthcare data compliance standards
• Experience with CI/CD practices and proficiency with version control systems like Git
• Familiarity with data pipeline orchestration and monitoring tools Like Airflow , Azure Data Factory or similar frameworks
• Understanding of biomedical informatics methodologies is a plus
• Strong analytical, problem-solving, and critical-thinking abilities
• Excellent communication and collaboration skills across diverse teams
• Ability to adapt quickly to new technologies and evolving data environments
• Exposure to FHIR, OMOP, or similar healthcare data models (nice-to-have)
• Data Pipeline Design
• ETL/ELT Processes
• Python
• SQL
• Apache Spark
• PySpark
• Azure Fabric
• Azure Synapse Serverless
• Databricks
• Distributed Data Processing
• Large-Scale Datasets
• Data Modeling
• Data Governance
• Metadata Management
• Data Quality Frameworks
• HIPAA Compliance
• Healthcare Data Compliance
• CI/CD
• Version Control
• Git
• Pipeline Orchestration
• Pipeline Monitoring
• Apache Airflow
• Azure Data Factory
• Biomedical Informatics
• FHIR
• OMOP
• Analytical Thinking
• Problem-Solving
• Critical Thinking
• Communication
• Collaboration
• Adaptability

Benefits

• Full Remote – Monthly
• Tasks related to this position can be performed remotely with only occasional visits to an Emory University location.

Company

• Research university and health care system
• Atlanta, Georgia, USA
• https://www.emory.edu/home/index.html
• Founded in 1836
• 10001+ employees

Qualifications

• *Applicants must be legally authorized to work in the United States. This position is not eligible for visa sponsorship now or in the future
• A bachelor's degree in a related field and three years of related experience, OR an equivalent combination of education, training, and experience
• Strong experience designing and maintaining data pipelines and ETL/ELT processes
• Proficiency in Python, SQL, and Apache Spark (PySpark)
• Experience with modern data platforms such as Azure Fabric (preferred), Azure Synapse Serverless, or Databricks
• Ability to work with large, complex, and distributed datasets
• Solid understanding of data modeling concepts and best practices
• Familiarity with data governance, metadata management, and data quality frameworks
• Knowledge of HIPAA and healthcare data compliance standards
• Experience with CI/CD practices and proficiency with version control systems like Git
• Familiarity with data pipeline orchestration and monitoring tools Like Airflow , Azure Data Factory or similar frameworks
• Understanding of biomedical informatics methodologies is a plus
• Strong analytical, problem-solving, and critical-thinking abilities
• Excellent communication and collaboration skills across diverse teams
• Ability to adapt quickly to new technologies and evolving data environments
• Exposure to FHIR, OMOP, or similar healthcare data models (nice-to-have)
• A bachelor's degree in a related field and three years of related experience, OR an equivalent combination of education, training, and experience.

Required skills

• Data Pipelines and ETL/ELT, Python, SQL, Apache Spark (PySpark), Azure Fabric, Azure Synapse, Databricks, Data Modeling, Data Governance and Metadata Management, HIPAA Compliance, Git and CI/CD, Airflow and Azure Data Factory

Preferred skills

• Proficiency in Python, SQL, and Apache Spark (PySpark), Knowledge of HIPAA and healthcare data compliance standards

Experience

• 3+ years