Data Engineer – Physical AI Platform

Caterpillar Inc. is developing technology that connects machines, data, and people to enable safer, more productive, and more sustainable jobsites. The Data Engineer will design, build, and maintain scalable data pipelines, microservices, and cloud-based data systems, with an emphasis on performance, reliability, and data quality.

Responsibilities

Design, build, and optimize data pipelines and microservices using Python for real-time and batch processing

Develop cloud-based data ingestion solutions using AWS services (Kinesis, S3, DynamoDB, EventBridge, etc.)

Build and maintain data integration workflows for source data pipelines aligned to CI Autonomy

Translate business requirements into reliable data workflows, mappings, and system designs

Implement automated testing and validation to ensure data integrity across distributed systems

Monitor and troubleshoot pipelines using tools like CloudWatch to maintain reliability

Preferred

Bachelor's degree in Computer Science, Computer Engineering, or related field

2+ years development experience on modern, large scale, complex data platforms

3+ years developing and deploying Java or Python solutions to a production environment

Experience building high-throughput, scalable data pipelines

Strong hands-on experience with AWS data services (Kinesis, S3, DynamoDB, EventBridge, etc.)

Proficiency in SQL, including data quality and validation practices

Proficiency in deploying software using CI/CD tools such as Azure DevOps, Jira, Jenkins, etc

Experience developing microservices that support real-time data ingestion

Experience developing software applications using relational and noSQL databases

Ability to ensure data integrity across distributed and streaming systems

Experience with API development and secure integrations (OAuth 2.0)

Proven experience with monitoring, testing, and automation in large-scale data environments

Computer Science

Computer Engineering

Large-Scale Data Platforms

Production Deployment

Scalable Data Pipelines

AWS Data Services

Amazon Kinesis

Amazon S3

Amazon DynamoDB

Amazon EventBridge

SQL

Data Quality

Data Validation

CI/CD

Azure DevOps

Jira

Jenkins

Microservices Development

Real-Time Data Ingestion

Relational Databases

NoSQL Databases

Data Integrity

Distributed Systems

Streaming Systems

API Development

Secure Integrations

OAuth 2.0

System Monitoring

Software Testing

Benefits

Visa sponsorship is available with this position.

Domestic relocation assistance is available for this position.

Medical, dental, and vision benefits

Paid time off plan (Vacation, Holidays, Volunteer, etc.)

401(k) savings plans

Health Savings Account (HSA)

Flexible Spending Accounts (FSAs)

Health Lifestyle Programs

Employee Assistance Program

Voluntary Benefits and Employee Discounts

Career Development

Incentive bonus

Disability benefits

Life Insurance

Parental leave

Adoption benefits

Tuition Reimbursement

These benefits also apply to part-time employees

Company

Construction and mining equipment, engines, turbines, locomotives

Irving, Texas, USA

https://www.caterpillar.com/

Founded in 1925

10001+ employees

Qualifications

Ability to analyze issues in distributed data systems and deliver scalable solutions

Ability to document data flows, mappings, and system behavior clearly for cross-team use

Experience building backend systems and pipelines using Python, Java, and modern frameworks

Experience delivering solutions in an agile environment

Experience integrating APIs, streaming platforms, and databases

Ability to design scalable, event-driven data systems

Strong understanding of AWS services and data engineering tools

Experience implementing testing strategies to ensure data quality and system reliability

Bachelor's degree in Computer Science, Computer Engineering, or related field

2+ years development experience on modern, large scale, complex data platforms

3+ years developing and deploying Java or Python solutions to a production environment

Experience building high-throughput, scalable data pipelines

Strong hands-on experience with AWS data services (Kinesis, S3, DynamoDB, EventBridge, etc.)

Proficiency in SQL, including data quality and validation practices

Proficiency in deploying software using CI/CD tools such as Azure DevOps, Jira, Jenkins, etc

Experience developing microservices that support real-time data ingestion

Experience developing software applications using relational and noSQL databases

Ability to ensure data integrity across distributed and streaming systems

Experience with API development and secure integrations (OAuth 2.0)

Proven experience with monitoring, testing, and automation in large-scale data environments

Required skills

Python, Java, Data Pipelines, AWS, SQL, Microservices, API Development, Relational and NoSQL Databases, CI/CD, OAuth 2.0, Automated Testing, Data Quality and Validation

Preferred skills

Experience building high-throughput, scalable data pipelines

Experience

3+ years