Data Engineer-Data Platforms-Google - 4

IBM Consulting helps organizations advance their hybrid cloud and AI journeys through collaboration, strategic partnerships, and IBM technology. IBM is seeking a Data Engineer specializing in Google's data platforms to design, build, and maintain batch and real-time data pipelines, manage data platforms, and optimize data solutions on Google Cloud.

Responsibilities

Design Data Pipelines: Design and build data pipelines using Google DataProc, DataFlow, PubSub, BigQuery, and other Google services to meet business requirements

Develop Data Engineering Solutions: Utilize Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python to develop and manage data engineering solutions

Manage Data Platforms: Schedule and manage the data platform using Google Cloud Scheduler and Cloud Composer (Airflow) to ensure seamless data pipeline operations

Ensure Data Quality: Develop and manage batch and real-time data pipelines for Data Warehouse and Datalake to ensure high-quality data delivery

Optimize Data Solutions: Collaborate with stakeholders to optimize data solutions and ensure alignment with business objectives

Preferred

Experience with open source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala in data engineering solutions, enabling the development of efficient data pipelines and scalable data architectures

Experience with utilizing various data engineering tools and technologies to develop and manage data engineering solutions on Google's Cloud ecosystem, ensuring seamless integration and data flow

Experience in optimizing data solutions and ensuring alignment with business objectives, collaborating with stakeholders to drive business outcomes through data-driven insights

Data Pipeline Development

Scalable Data Architecture

Google Cloud

Data Engineering Tools

Data Solution Optimization

Data-Driven Insights

Stakeholder Collaboration

Benefits

This position can be performed from anywhere in the United States of America.

Support for long-term career development

Company

Enterprise AI, hybrid cloud, automation, data, and consulting solutions

Armonk, New York, USA

http://www.ibm.com/

Founded in 1911

10001+ employees

Qualifications

Experience with designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem, including services such as Google DataProc, DataFlow, PubSub, BigQuery, Big Table, Cloud Spanner, CloudSQL, and AlloyDB

Experience in developing and managing batch and real-time data pipelines for Data Warehouse and Datalake using Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python

Experience with open source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala in data engineering solutions

Experience in scheduling and managing the data platform using Google Cloud Scheduler and Cloud Composer (Airflow) to ensure seamless data pipeline operations

Experience with utilizing various data engineering tools and technologies to develop and manage data engineering solutions on Google's Cloud ecosystem

Experience with open source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala in data engineering solutions, enabling the development of efficient data pipelines and scalable data architectures

Experience with utilizing various data engineering tools and technologies to develop and manage data engineering solutions on Google's Cloud ecosystem, ensuring seamless integration and data flow

Experience in optimizing data solutions and ensuring alignment with business objectives, collaborating with stakeholders to drive business outcomes through data-driven insights

Required skills

Google Cloud Platform, Data Pipeline Development, Google Cloud Dataflow, Google Cloud Dataproc, BigQuery, Google Cloud Storage, Apache Beam, Apache Spark, Apache Airflow, Python, Hadoop, dbt

Experience

Mid level