← Back to jobsJob ID col-cmv8gprqs07wjoz016nfsh40c

Responsibilities

  • Lakehouse Architecture: Design and implement robust data pipelines using the Databricks Medallion Architecture (Bronze, Silver, Gold layers) to process structured and unstructured data
  • Pipeline Automation: Develop, scale, and orchestrate complex data workflows utilizing Databricks Jobs and Delta Live Tables (DLT)
  • Data Ops & CI/CD Deployment: Standardize and automate the deployment of Databricks assets, workspace configurations, and code pipelines across Dev, QA, and Production environments
  • AWS Integration: Ensure seamless data cataloging, storage, and movement across the AWS ecosystem, specifically integrating Databricks with Amazon S3, AWS Glue, and AWS IAM for secure access control
  • Performance Optimization: Tune Spark clusters, optimize Delta Lake storage (e.g., Z-Ordering, partitioning), and manage compute costs within the AWS environment
  • Data Governance & Security: Implement fine-grained data access controls, data lineage, and auditing using Unity Catalog or native cloud security controls
  • Collaboration: Partner with Data Scientists, Risk Managers, and downstream analytics teams to deliver clean, business-ready data views for reporting and AI modeling

Qualifications

  • Professional experience in architectural design and development within the Databricks platform, working in an AWS cloud environment
  • Proven proficiency in automated deployments using Databricks Asset Bundles (DABs), Terraform (specifically the Databricks and AWS providers), and standard Git pipelines (e.g., GitHub Actions, GitLab CI/CD, or AWS CodePipeline)
  • Programming skills in Python (PySpark) and SQL for complex data manipulation and transformation
  • Strong understanding of Apache Spark internals, Delta Lake mechanics, and streaming data concepts (e.g., interacting with Amazon MSK or Kafka data streams)
  • Proven experience building production-grade ETL/ELT pipelines, handling data schema validation, and cleansing raw capture feeds

Preferred

  • Databricks Certified Data Engineer Professional or AWS Certified Data Engineer – Professional is highly advantageous

Required skills

DatabricksAWSDatabricks Asset BundlesTerraformGitPythonPySparkSQLApache SparkDelta LakeKafkaETL/ELT Pipelines

Full job description

Data Engineer Job Details | Sumitomo Mitsui Banking Corporation jobs in United States

SMBC Group

  • $73K/yr - $98K/yr
  • Professional Services

Responsibilities

  • Lakehouse Architecture: Design and implement robust data pipelines using the Databricks Medallion Architecture (Bronze, Silver, Gold layers) to process structured and unstructured data
  • Pipeline Automation: Develop, scale, and orchestrate complex data workflows utilizing Databricks Jobs and Delta Live Tables (DLT)
  • Data Ops & CI/CD Deployment: Standardize and automate the deployment of Databricks assets, workspace configurations, and code pipelines across Dev, QA, and Production environments
  • AWS Integration: Ensure seamless data cataloging, storage, and movement across the AWS ecosystem, specifically integrating Databricks with Amazon S3, AWS Glue, and AWS IAM for secure access control
  • Performance Optimization: Tune Spark clusters, optimize Delta Lake storage (e.g., Z-Ordering, partitioning), and manage compute costs within the AWS environment
  • Data Governance & Security: Implement fine-grained data access controls, data lineage, and auditing using Unity Catalog or native cloud security controls
  • Collaboration: Partner with Data Scientists, Risk Managers, and downstream analytics teams to deliver clean, business-ready data views for reporting and AI modeling

Preferred

  • Databricks Certified Data Engineer Professional or AWS Certified Data Engineer – Professional is highly advantageous

Benefits

  • The role may also be eligible for an annual discretionary incentive award.
  • SMBC’s employees participate in a Hybrid workforce model that provides employees with an opportunity to work from home, as well as, from an SMBC office. Employees must live within a reasonable commuting distance of their office location.

Qualifications

  • Professional experience in architectural design and development within the Databricks platform, working in an AWS cloud environment
  • Proven proficiency in automated deployments using Databricks Asset Bundles (DABs), Terraform (specifically the Databricks and AWS providers), and standard Git pipelines (e.g., GitHub Actions, GitLab CI/CD, or AWS CodePipeline)
  • Programming skills in Python (PySpark) and SQL for complex data manipulation and transformation
  • Strong understanding of Apache Spark internals, Delta Lake mechanics, and streaming data concepts (e.g., interacting with Amazon MSK or Kafka data streams)
  • Proven experience building production-grade ETL/ELT pipelines, handling data schema validation, and cleansing raw capture feeds
  • Databricks Certified Data Engineer Professional or AWS Certified Data Engineer – Professional is highly advantageous

Required skills

  • Databricks, AWS, Databricks Asset Bundles, Terraform, Git, Python, PySpark, SQL, Apache Spark, Delta Lake, Kafka, ETL/ELT Pipelines

Experience

  • Mid level

Company

  • SMBC Group is a leading global financial group with a 400-year history and a network spanning nearly 40 countries and regions.
  • Tokyo, Tokyo, JPN
  • https://www.smbcgroup.com/americas

Job details

Employment type

Full-time

Seniority level

Mid level

Date posted

Oct 8, 2026

Work mode

Hybrid

Job source

Ethic Leaf recruiter

Company

Data Engineer Job Details

Information Technology & AI

SMBC Group is a leading global financial group with a 400-year history and a network spanning nearly 40 countries and regions.

Similar jobs

Matched on role, skills and location · live jobs only

See more Data Engineer jobs in New York

M Science

1d ago

New

Data Engineer

Full-timeOn-siteNew York, NY$110K – $150K/yr

M Science is hiring a Data Engineer to build and improve large-scale data pipelines and datasets for analytics, experimentation, and machine learning.

Why similarSame roleAWSNew York

Capgemini

1d ago

New

Senior Data Engineer - Databricks

Full-timeOn-siteNew York, NY$84K – $156K/yr

Capgemini is seeking a Senior Databricks Data Engineer to design, build, and optimize scalable data platforms and pipelines in cloud-based environments.

Why similarSame roleAWSNew York

Dice

1d ago

New

Senior Data Engineer - Databricks

Full-timeOn-siteNew York, NY$84K – $156K/yr

Capgemini is seeking a Senior Databricks Data Engineer to design, build, and optimize scalable cloud-based data platforms and pipelines.

Why similarSame roleAWSNew York

Get new Data Engineer jobs in New York by email

Create job alert

Data Engineer

Data Engineer Job Details · $73K – $98K/yr

Apply via Ethic Leaf