← Back to jobsJob ID col-cmvgydsz3009ts3ctc0zcxe13

State FarmPosted 30+ days ago

Lead Data Engineer (AWS)

Full-time
Hybrid
3+ years
Competitive
Posted 30+ days ago

State Farm is hiring a Lead Data Engineer for its Innovation Group within SAIL, focused on Agency and Digital. The role builds and operates data capabilities and platforms for experimentation, product development, and emerging technology, with technical leadership responsibilities that may include data science, machine learning, and generative AI initiatives.

Posted
30+ Days Ago
Source
jobs.statefarm.com
Sector
Information Technology & AI

Opens the employer's application page in a new tab

Responsibilities

  • Innovation Group is seeking a Data Engineer or Lead Data Engineer to join our product execution team within SAIL (State Farm Applied Innovation Lab) focused on Agency and Digital. You will work alongside software engineers, product teams, and business partners to design and build the data capabilities that enable experimentation, product development, and emerging technology solutions
  • Our teams operate in environments where the problem, data, and solution may not be fully defined at the outset. We are looking for engineers who are comfortable working through that ambiguity, quickly learning new technologies and domains, and turning complex data needs into practical, scalable solutions
  • Day to day, the work centers on three areas: writing code to move and transform data; partnering with software engineers and product teams to determine how data can be accessed, analyzed, and used; and designing and maintaining the platforms where structured and unstructured data lives. Strong engineering fundamentals are essential, but success in this role also requires flexibility, curiosity, innovative thinking, and the ability to navigate a complex organization
  • The ideal candidate understands the fundamentals of data engineering and how data solutions are designed, built, deployed, monitored, and maintained as production-quality applications rather than one-off scripts. They communicate clearly, collaborate effectively across technical and non-technical teams, seek to understand the business problem behind the request, and continuously develop their skills as technologies and needs evolve. Depending on the initiative, this role may also help lead data science, machine learning, and generative AI efforts, working across the lifecycle from data exploration and experimentation through engineering, deployment, and ongoing operation
  • At the Lead Data Engineer level, the role also includes providing technical leadership, mentoring engineers, influencing architecture and engineering practices, and helping shape the direction of the team's data capabilities and solutions

Qualifications

  • 3 plus years with python development for extracting, transforming, processing, and moving data
  • Proficiency writing SQL queries and working with relational databases
  • Experience designing, building, and maintaining ETL/ELT data pipelines
  • Experience working with structured, semi-structured, and unstructured data, including formats such as CSV, JSON, and text
  • Experience with public cloud platforms
  • Experience using Git and modern version-control practices
  • Familiarity with CI/CD practices and deploying code into shared or production environments
  • Comfort working in Linux and command-line environments
  • Understanding of software engineering principles such as testing, maintainability, observability, and reusable design
  • Ability to troubleshoot data quality, pipeline, integration, and performance issues

Preferred

  • preferably AWS, including services such as S3, Glue, and Lambda
  • Experience designing data architectures for new products, experiments, or emerging technology use cases
  • Familiarity with machine learning concepts and the end-to-end ML lifecycle, including data preparation, feature engineering, model development, evaluation, deployment, monitoring, and iteration
  • Experience partnering with data scientists or ML engineers to operationalize models and build the data pipelines and infrastructure that support machine learning solutions
  • Familiarity with generative AI, large language models (LLMs), and the data considerations associated with building AI-enabled applications
  • Experience designing or building LLM-based workflows, including agentic workflows that use tools, APIs, retrieval, orchestration, or multi-step reasoning to complete tasks
  • Experience using AI-assisted development tools as part of the engineering workflow to accelerate coding, testing, debugging, documentation, and technical exploration
  • Ability to evaluate AI-generated code and outputs critically and apply appropriate engineering, security, testing, and governance practices
  • Experience with APIs, event-driven architectures, or streaming data
  • Familiarity with object storage and modern data platforms
  • Experience working with large or complex datasets
  • Understanding of data governance, security, privacy, lineage, and access-management considerations
  • Experience working in Agile or product-oriented engineering environments
  • Ability to translate business needs into technical data requirements and solutions
  • Experience working across teams and managing technical dependencies
  • Comfort experimenting with new technologies and solving problems where the path forward is not yet fully defined
  • Emerging technology use cases

Required skills

AWSAWS GlueAWS LambdaAmazon S3CI/CDETL/ELT Data PipelinesGitLinuxPythonRelational DatabasesSQLSoftware Testing

Full job description

State Farm is hiring a Lead Data Engineer for its Innovation Group within SAIL, focused on Agency and Digital. The role builds and operates data capabilities and platforms for experimentation, product development, and emerging technology, with technical leadership responsibilities that may include data science, machine learning, and generative AI initiatives.

Responsibilities

  • Innovation Group is seeking a Data Engineer or Lead Data Engineer to join our product execution team within SAIL (State Farm Applied Innovation Lab) focused on Agency and Digital. You will work alongside software engineers, product teams, and business partners to design and build the data capabilities that enable experimentation, product development, and emerging technology solutions
  • Our teams operate in environments where the problem, data, and solution may not be fully defined at the outset. We are looking for engineers who are comfortable working through that ambiguity, quickly learning new technologies and domains, and turning complex data needs into practical, scalable solutions
  • Day to day, the work centers on three areas: writing code to move and transform data; partnering with software engineers and product teams to determine how data can be accessed, analyzed, and used; and designing and maintaining the platforms where structured and unstructured data lives. Strong engineering fundamentals are essential, but success in this role also requires flexibility, curiosity, innovative thinking, and the ability to navigate a complex organization
  • The ideal candidate understands the fundamentals of data engineering and how data solutions are designed, built, deployed, monitored, and maintained as production-quality applications rather than one-off scripts. They communicate clearly, collaborate effectively across technical and non-technical teams, seek to understand the business problem behind the request, and continuously develop their skills as technologies and needs evolve. Depending on the initiative, this role may also help lead data science, machine learning, and generative AI efforts, working across the lifecycle from data exploration and experimentation through engineering, deployment, and ongoing operation
  • At the Lead Data Engineer level, the role also includes providing technical leadership, mentoring engineers, influencing architecture and engineering practices, and helping shape the direction of the team's data capabilities and solutions

Preferred

  • preferably AWS, including services such as S3, Glue, and Lambda
  • Experience designing data architectures for new products, experiments, or emerging technology use cases
  • Familiarity with machine learning concepts and the end-to-end ML lifecycle, including data preparation, feature engineering, model development, evaluation, deployment, monitoring, and iteration
  • Experience partnering with data scientists or ML engineers to operationalize models and build the data pipelines and infrastructure that support machine learning solutions
  • Familiarity with generative AI, large language models (LLMs), and the data considerations associated with building AI-enabled applications
  • Experience designing or building LLM-based workflows, including agentic workflows that use tools, APIs, retrieval, orchestration, or multi-step reasoning to complete tasks
  • Experience using AI-assisted development tools as part of the engineering workflow to accelerate coding, testing, debugging, documentation, and technical exploration
  • Ability to evaluate AI-generated code and outputs critically and apply appropriate engineering, security, testing, and governance practices
  • Experience with APIs, event-driven architectures, or streaming data
  • Familiarity with object storage and modern data platforms
  • Experience working with large or complex datasets
  • Understanding of data governance, security, privacy, lineage, and access-management considerations
  • Experience working in Agile or product-oriented engineering environments
  • Ability to translate business needs into technical data requirements and solutions
  • Experience working across teams and managing technical dependencies
  • Comfort experimenting with new technologies and solving problems where the path forward is not yet fully defined
  • AWS
  • Amazon S3
  • AWS Glue
  • AWS Lambda
  • Data architecture design
  • Product data architecture
  • Experimental data architecture
  • Emerging technology use cases
  • Machine learning concepts
  • ML lifecycle
  • Data preparation
  • Feature engineering
  • Model development
  • Model evaluation
  • Model deployment
  • Model monitoring
  • Model iteration
  • ML model operationalization
  • Data pipelines
  • ML infrastructure
  • Generative AI
  • Large language models
  • AI application data
  • LLM workflow design
  • Agentic workflows
  • Tool integration
  • API integration
  • Retrieval workflows
  • Workflow orchestration
  • Multi-step reasoning
  • AI-assisted development tools
  • AI code evaluation
  • AI output evaluation
  • Security practices
  • Testing practices
  • Governance practices
  • APIs
  • Event-driven architecture
  • Streaming data
  • Object storage
  • Modern data platforms
  • Large-scale datasets
  • Complex datasets
  • Data governance
  • Data security
  • Data privacy
  • Data lineage
  • Access management
  • Agile environments
  • Product-oriented engineering
  • Business requirements translation
  • Technical data requirements
  • Technical solution design
  • Technical dependency management
  • Cross-functional collaboration
  • Adaptability
  • Problem-solving

Benefits

  • Hybrid work environment: spend time working from home and some time working in the office.

Company

  • Personalized insurance company offering auto, home, property, and life coverage
  • Bloomington, Illinois, USA
  • https://www.statefarm.com/
  • Founded in 1922
  • 10001+ employees

Qualifications

  • 3 plus years with python development for extracting, transforming, processing, and moving data
  • Proficiency writing SQL queries and working with relational databases
  • Experience designing, building, and maintaining ETL/ELT data pipelines
  • Experience working with structured, semi-structured, and unstructured data, including formats such as CSV, JSON, and text
  • Experience with public cloud platforms
  • Experience using Git and modern version-control practices
  • Familiarity with CI/CD practices and deploying code into shared or production environments
  • Comfort working in Linux and command-line environments
  • Understanding of software engineering principles such as testing, maintainability, observability, and reusable design
  • Ability to troubleshoot data quality, pipeline, integration, and performance issues
  • Preferably AWS, including services such as S3, Glue, and Lambda
  • Experience designing data architectures for new products, experiments, or emerging technology use cases
  • Familiarity with machine learning concepts and the end-to-end ML lifecycle, including data preparation, feature engineering, model development, evaluation, deployment, monitoring, and iteration
  • Experience partnering with data scientists or ML engineers to operationalize models and build the data pipelines and infrastructure that support machine learning solutions
  • Familiarity with generative AI, large language models (LLMs), and the data considerations associated with building AI-enabled applications
  • Experience designing or building LLM-based workflows, including agentic workflows that use tools, APIs, retrieval, orchestration, or multi-step reasoning to complete tasks
  • Experience using AI-assisted development tools as part of the engineering workflow to accelerate coding, testing, debugging, documentation, and technical exploration
  • Ability to evaluate AI-generated code and outputs critically and apply appropriate engineering, security, testing, and governance practices
  • Experience with APIs, event-driven architectures, or streaming data
  • Familiarity with object storage and modern data platforms
  • Experience working with large or complex datasets
  • Understanding of data governance, security, privacy, lineage, and access-management considerations
  • Experience working in Agile or product-oriented engineering environments
  • Ability to translate business needs into technical data requirements and solutions
  • Experience working across teams and managing technical dependencies
  • Comfort experimenting with new technologies and solving problems where the path forward is not yet fully defined

Required skills

  • AWS, AWS Glue, AWS Lambda, Amazon S3, CI/CD, ETL/ELT Data Pipelines, Git, Linux, Python, Relational Databases, SQL, Software Testing

Preferred skills

  • Experience working with large or complex datasets, Familiarity with object storage and modern data platforms

Experience

  • 3+ years

Job details

Employment type

Full-time

Seniority level

3+ years

Date posted

Oct 19, 2026

Work mode

Hybrid

Job source

jobs.statefarm.com

Company

State Farm

Information Technology & AI

Personalized insurance company offering auto, home, property, and life coverage

See all 4 jobs from State Farm →
  • Bloomington, Illinois, USA
  • Information Technology & AI
  • 10001+ employees
  • Founded 1922
  • statefarm.com

Similar jobs

Matched on role, skills and location · live jobs only

See more Data Engineer jobs in Bloomington

Woven by Toyota

Today

New

Software Engineer II, Data Labeling

Full-timeHybridAnn Arbor, MICompetitive

Woven by Toyota is enabling Toyota’s transformation into a mobility company through autonomous driving, software-defined vehicles, mobility infrastructure, and cloud and AI technologies.

Why similarSame roleAWSGit

Axtria - Ingenious Insights

Today

New

Data Engineer II – Databricks

Full-timeOn-siteChapel Hill, NCCompetitive

Axtria is seeking a Data Engineer II to build and optimize scalable data pipelines for pharmaceutical marketing and customer engagement initiatives.

Why similarSame roleAWSPython

Snorkel AI

Today

New

Senior/Staff Forward Deployed Engineer - Data as a Service

Full-timeHybridNew York, NY$180K – $320K/yr

Snorkel AI is a frontier AI data lab that develops datasets, benchmarks, evaluations, and custom solutions for AI systems.

Why similarSame rolePythonSQL

Get new Data Engineer jobs in Bloomington by email

Create job alert

Lead Data Engineer (AWS)

State Farm · Competitive

Apply on company site