Database Analyst

SoTalent is recruiting for an unnamed technology services firm. The Database Analyst and Engineer role focuses on developing and maintaining automated data pipelines and applications, supporting reliable data movement and producing well-modeled datasets for informatics and reporting teams.

Responsibilities

• The primary focus of this role involves the development and maintenance of automated data pipelines and applications
• You will be responsible for building, testing, and supporting pipelines within cloud environments like Databricks to ensure the delivery of high quality, analysis ready datasets
• A significant portion of the work involves designing bidirectional data movements between enterprise warehouses and operational systems, ensuring synchronization integrity and resolving data conflicts through robust source of truth logic
• Technical responsibilities include creating extraction and transformation logic using SQL, Spark SQL, PySpark, and Python
• You will also be tasked with implementing complex patient matching and record linkage logic, harmonizing datasets from diverse sources, and establishing automated validation checks to identify data gaps or inconsistencies
• In addition to engineering tasks, you will be expected to create comprehensive technical documentation and data flow maps
• This includes developing training materials for administrators and end users, providing technical support for all developed applications, and drafting transition plans to ensure long term project continuity
• The goal is to provide well modeled data that internal informatics teams can easily leverage for Power BI and other reporting platforms

Benefits

• Fully remote role within the United States

Company

• Recruitment media and candidate acquisition for employers
• New York, US
• https://sotalent.us/
• Founded in 2020
• 2-10 employees

Qualifications

• Experience building and maintaining data pipelines in cloud workgroups like Databricks
• Proficiency in SQL, Spark SQL, PySpark, and Python
• Expertise in ETL and ELT development with a focus on reproducible and auditable code
• Knowledge of record linkage and patient matching methodologies
• Ability to harmonize large datasets and implement automated data quality checks
• Strong documentation skills for technical processes and user training
• Capacity to design data flow maps and internal control systems
• Background in delivering clean data models for business intelligence tools

Required skills

• Data Harmonization, Data Modeling, Data Quality Validation, Databricks, ETL/ELT Development, Patient Matching, PySpark, Python, Record Linkage, SQL, Spark SQL

Experience

• Mid level