← Back to jobsJob ID col-cmwf7446g0b0c89hqf6l5hpxt

CRED iQ®Posted 8 days ago

Senior Data Engineer

Full-time
Remote
5+ years
$140K/yr
Posted 8 days ago

CRED iQ is seeking a Senior Data Engineer to own and optimize the pipelines that ingest, transform, and deliver commercial real estate data. The role focuses on Airflow orchestration, PostgreSQL performance, Snowflake client data delivery, infrastructure cost ownership, and collaboration with product, engineering, and research teams.

Big Data

Posted
8 days ago
Source
linkedin.com
Sector
Information Technology & AI

Opens the employer's application page in a new tab

Responsibilities

  • Design, build, and maintain Airflow DAGs that ingest, validate, and transform large volumes of loan, property, and financial data from internal and third-party sources
  • Make pipelines incremental and idempotent, so reruns and backfills are safe and reprocess only what changed
  • Right-size scheduling, concurrency, pools, and worker resources so jobs finish on time without idle or oversized compute
  • Own monitoring, alerting, and on-call triage for pipeline failures and late data
  • Tune PostgreSQL for both speed and spend: query plans, indexing, partitioning, vacuum and bloat, and connection management
  • Find and fix the most expensive queries and jobs using pg_stat_statements, EXPLAIN plans, and database performance metrics
  • Right-size instances, storage, and read replicas, and set retention and archival policies for data that no longer needs hot storage
  • Review schema and data model changes for their performance and cost impact before they ship
  • Monitor database usage and access for audits
  • Build and operate client data deliveries in Snowflake, including secure data shares and scheduled loads
  • Control Snowflake spend: warehouse sizing, auto-suspend, clustering, incremental loads, and resource monitors with budgets
  • Guarantee delivery quality with row-count, schema, and reconciliation checks between PostgreSQL and Snowflake
  • Build cost visibility: attribute compute, storage, and warehouse spend to individual pipelines, datasets, and client deliveries
  • Set cost baselines and targets, report on them regularly, and lead the work that closes the gap
  • Weigh cost against freshness and reliability, and explain the trade-off clearly to engineering and business stakeholders
  • Mentor other engineers and set standards for cost-aware pipeline and SQL design through code review
  • Partner with product managers, engineers, and research teams to meet evolving data needs
  • Create automated data exports and views for CRE Research and external clients

Qualifications

  • 5+ years of experience
  • a hands-on data engineer who has owned production pipelines end to end and treats cost as an engineering metric alongside reliability and data freshness
  • experienced with large loan, property, and financial datasets
  • strong knowledge of PostgreSQL internals
  • a degree in CS, CE, EE or have the equivalent practical experience
  • 5+ years of data engineering experience, including ownership of production pipelines end to end
  • Deep Apache Airflow experience: DAG design, scheduling, backfills, retries, resource tuning, and debugging failed runs
  • Performance and Optimization (dbt modeling): Design, build, and maintain modular, scalable dbt models to transform raw warehouse data into reliable, analytics-ready, datasets
  • Expert SQL and strong PostgreSQL internals knowledge: query planning, indexing strategies, partitioning, locking, and vacuum behavior
  • Hands-on Snowflake experience delivering data to external consumers, including data sharing and warehouse cost management
  • A track record of cutting data infrastructure cost, with numbers: what you measured, what you changed, and what it saved
  • Strong Python for pipeline code, data validation, and tooling
  • Experience running data workloads on a major cloud platform (preferably AWS) and reading its billing and usage data
  • Sound data modeling judgment and a habit of building data quality checks into every pipeline
  • Clear written communication. You can explain a cost or performance trade-off to a non-engineer in a paragraph

Preferred

  • Experience with commercial real estate, CMBS, or other structured finance and loan-level data
  • Experience building client data feeds with delivery SLAs and audit requirements
  • Infrastructure as code (Terraform) and containerized workloads
  • FinOps practice: cost tagging, chargeback or showback reporting, and budget alerting
  • Experience working in a SOC 2 environment, including access controls and audit logging for data systems

Required skills

AWSApache AirflowData ModelingData Pipeline EngineeringData Quality ValidationPostgreSQLPythonSQLSnowflakedbt

Full job description

CRED iQ is seeking a Senior Data Engineer to own and optimize the pipelines that ingest, transform, and deliver commercial real estate data. The role focuses on Airflow orchestration, PostgreSQL performance, Snowflake client data delivery, infrastructure cost ownership, and collaboration with product, engineering, and research teams.

Big Data

  • Artificial Intelligence (AI)
  • Real Estate
  • Information Technology
  • Software
  • Analytics
  • Commercial Real Estate

Responsibilities

  • Design, build, and maintain Airflow DAGs that ingest, validate, and transform large volumes of loan, property, and financial data from internal and third-party sources
  • Make pipelines incremental and idempotent, so reruns and backfills are safe and reprocess only what changed
  • Right-size scheduling, concurrency, pools, and worker resources so jobs finish on time without idle or oversized compute
  • Own monitoring, alerting, and on-call triage for pipeline failures and late data
  • Tune PostgreSQL for both speed and spend: query plans, indexing, partitioning, vacuum and bloat, and connection management
  • Find and fix the most expensive queries and jobs using pg_stat_statements, EXPLAIN plans, and database performance metrics
  • Right-size instances, storage, and read replicas, and set retention and archival policies for data that no longer needs hot storage
  • Review schema and data model changes for their performance and cost impact before they ship
  • Monitor database usage and access for audits
  • Build and operate client data deliveries in Snowflake, including secure data shares and scheduled loads
  • Control Snowflake spend: warehouse sizing, auto-suspend, clustering, incremental loads, and resource monitors with budgets
  • Guarantee delivery quality with row-count, schema, and reconciliation checks between PostgreSQL and Snowflake
  • Build cost visibility: attribute compute, storage, and warehouse spend to individual pipelines, datasets, and client deliveries
  • Set cost baselines and targets, report on them regularly, and lead the work that closes the gap
  • Weigh cost against freshness and reliability, and explain the trade-off clearly to engineering and business stakeholders
  • Mentor other engineers and set standards for cost-aware pipeline and SQL design through code review
  • Partner with product managers, engineers, and research teams to meet evolving data needs
  • Create automated data exports and views for CRE Research and external clients

Preferred

  • Experience with commercial real estate, CMBS, or other structured finance and loan-level data
  • Experience building client data feeds with delivery SLAs and audit requirements
  • Infrastructure as code (Terraform) and containerized workloads
  • FinOps practice: cost tagging, chargeback or showback reporting, and budget alerting
  • Experience working in a SOC 2 environment, including access controls and audit logging for data systems

Benefits

  • Equity Options
  • Medical, Dental, Vision, Short Term Disability, HSA, and 401K
  • Flexible Work Schedule / Remote Work (CRED iQ is a fully remote company)
  • A work environment built on teamwork, flexibility, and respect.

Qualifications

  • 5+ years of experience
  • a hands-on data engineer who has owned production pipelines end to end and treats cost as an engineering metric alongside reliability and data freshness
  • experienced with large loan, property, and financial datasets
  • write expert SQL
  • strong knowledge of PostgreSQL internals
  • a degree in CS, CE, EE or have the equivalent practical experience
  • 5+ years of data engineering experience, including ownership of production pipelines end to end
  • Deep Apache Airflow experience: DAG design, scheduling, backfills, retries, resource tuning, and debugging failed runs
  • Performance and Optimization (dbt modeling): Design, build, and maintain modular, scalable dbt models to transform raw warehouse data into reliable, analytics-ready, datasets
  • Expert SQL and strong PostgreSQL internals knowledge: query planning, indexing strategies, partitioning, locking, and vacuum behavior
  • Hands-on Snowflake experience delivering data to external consumers, including data sharing and warehouse cost management
  • A track record of cutting data infrastructure cost, with numbers: what you measured, what you changed, and what it saved
  • Strong Python for pipeline code, data validation, and tooling
  • Experience running data workloads on a major cloud platform (preferably AWS) and reading its billing and usage data
  • Sound data modeling judgment and a habit of building data quality checks into every pipeline
  • Clear written communication. You can explain a cost or performance trade-off to a non-engineer in a paragraph
  • A hands-on data engineer who has owned production pipelines end to end and treats cost as an engineering metric alongside reliability and data freshness
  • Experienced with large loan, property, and financial datasets
  • Write expert SQL
  • Strong knowledge of PostgreSQL internals
  • A degree in CS, CE, EE or have the equivalent practical experience
  • Experience with commercial real estate, CMBS, or other structured finance and loan-level data
  • Experience building client data feeds with delivery SLAs and audit requirements
  • Infrastructure as code (Terraform) and containerized workloads
  • FinOps practice: cost tagging, chargeback or showback reporting, and budget alerting
  • Experience working in a SOC 2 environment, including access controls and audit logging for data systems
  • 5+ years of data engineering experience, including ownership of production pipelines end to end.

Required skills

  • AWS, Apache Airflow, Commercial Real Estate and Structured Finance Data, Data Modeling, Data Pipeline Engineering, Data Quality Validation, FinOps and Data Infrastructure Cost Optimization, PostgreSQL, Python, SQL, Snowflake, dbt

Experience

  • 5+ years

Company

  • Commercial real estate data and analytics platform for securitized markets.
  • Wayne, Pennsylvania, USA
  • https://www.cred-iq.com/

Job details

Employment type

Full-time

Seniority level

5+ years

Date posted

Nov 13, 2026

Work mode

Remote

Job source

linkedin.com

Company

CRED iQ®

Information Technology & AI

Commercial real estate data and analytics platform for securitized markets.

  • Wayne, Pennsylvania, USA
  • Information Technology & AI
  • cred-iq.com

Similar jobs

Matched on role, skills and location · live jobs only

See more Data Engineer jobs

First Citizens Bank

1d ago

New

Senior Data Engineer - DevOps (Remote - NC, AZ, TX, FL)

Full-timeRemoteUnited StatesCompetitive

First Citizens Bank is seeking a Senior Data Engineer - DevOps to support its data engineering platform.

Why similarSame roleAWSPython

Coinbase

1d ago

New

Senior Software Engineer, Data Engineering Platform

Full-timeRemoteUnited States$186K – $219K/yr

Coinbase is hiring a Senior Software Engineer for its Data Platform team. The role builds and maintains SDKs, self-service applications, reusable abstractions, and security and observability tooling that help internal teams create, manage, and use data…

Why similarSame roleApache AirflowPython

Airbnb

1d ago

New

Data Engineer Tech Foundations

Full-timeRemoteUnited States$162K – $186K/yr

Airbnb is hiring a Data Engineer to build and operate reliable, reusable datasets and pipelines that support AI usage and cost measurement, Unit Economics, enterprise application data, and critical financial metrics.

Why similarSame roleData ModelingPython

Get new Data Engineer jobs by email

Create job alert

Senior Data Engineer

CRED iQ® · $140K/yr

Apply on company site