Mid-Level/Lead DATA ENGINEER-Python, AWS, Spark (Hybrid-TX)

State Farm is seeking an experienced Data Engineer for its Property & Casualty Data Pipeline team. The role involves building and scaling reliable, automated data products and pipelines, developing compliant data solutions across platforms, and partnering with teams to address complex data challenges and deliver business impact.

Responsibilities

Utilizes industry-adopted languages and frameworks in coding, testing, security, DevOps, DataOps and data engineering practices

Develops and maintains reusable, scalable, and compliant data solutions across multiple platforms and compute environments

Responsible for the identification, acquisition, cleansing, profiling, and ETL (extracting, transformation, and loading) of data used in analytic discovery and production solution deployment across multiple platforms

Establishes business domain knowledge for existing State Farm data sources and investigates, recommends, and initiates acquisition of data resources, both internal and external

Identifies and consults on emerging technologies and critical core systems, including techniques, tools, data sources, and platforms in the data engineering field

Familiar with handling datasets containing mixes of structured and unstructured data

Exhibits DataOps mindset where team is accountable for ensuring data aligns to enterprise needs and leveraging automation to deliver quality data solutions

Collects and analyzes information to identify customer's technical needs, suggest solutions and develop implementation and integration plans (e.g., technical proposals)

Responsible for the analysis, design, deployment, support, and security of technology to ensure the organization is efficiently managing its technology and data-related assets in accordance with market best-practices and external regulations

Applies a wide application of complex principles, theories, and concepts in computer science for data engineering solutions

Benefits

Potential yearly incentive pay up to 15% of base salary

Annual raise and bonus

State Farm pays most of your healthcare premium

Multiple healthcare plan options, including a high deductible plan

100% coverage for in-network preventative care through all medical plans

Vision, dental, telemedicine, and 24/7 mental health professionals

Health and wellbeing programs

Industry-leading training programs

Tuition assistance programs

Employee resource groups

Mentoring

Fertility/IVF/adoption assistance

College coaching

National discount programs

Interactive monthly financial workshops

Free financial coaching

Access to a State Farm Federal Credit Union savings account or financing

Generous time off policies

Opportunity to initially earn up to 20 days annually

Parental leave

Paid holidays

Celebration day

Life leave (40 hours/year)

Bereavement leave

Community service/education support days

Matching Gift Program

Good Neighbor Grant Program

Employee Assistance Fund

Free financial advisors

401(k) plan with company contributions of up to 7% of salary

Hybrid work environment with time working from home and in the office

Company

Personalized insurance company offering auto, home, property, and life coverage

Bloomington, Illinois, USA

https://www.statefarm.com/

Founded in 1922

10001+ employees

Qualifications

Qualified candidates must live within close proximity to a hub location listed below and should plan to spend time working from home and some time working in the office as part of our hybrid work environment

Applicants for this position are required to be eligible to lawfully work in the U.S. immediately; employer will not sponsor applicants for U.S. work authorization (e.g. H-1B visa) for this opportunity

Professional experience as a Data Engineer

Proficiency in programming languages such as Python, Spark SQL (or PySpark), R, Java, Bash, etc

Hands-on experience with AWS services including ETL tools (Glue, EMR Serverless), Lambda, Step Functions, EventBridge, S3, DynamoDB, Kinesis Firehose, Redshift, Iceberg, and SageMaker

Experience with distributed data processing frameworks such as Apache Spark, Databricks

Experience with infrastructure as code tools such as OpenTofu (formerly Terraform) for managing cloud resources and deployments

Familiarity with CI/CD pipelines including automated testing, security scans, and tools like Airflow

Experience or ability to rapidly gain P&C data domain knowledge, including rating, underwriting, and/or claims

Experience with relational databases such as DB2, Postgres, Redshift, etc

Experience with version control systems such as GitHub or GitLab

Data access skills using SQL, and Athena

Experience in designing, building, and maintaining data pipelines for automated data processing

Knowledge of data modeling techniques such as star schema and snowflake schema, with an understanding of data architecture

Adaptability

Work Ethic

Critical Thinking

Strategic Business Focus

Technical/Functional Expertise

Required skills

Python, Apache Spark, AWS, ETL, Data Pipeline Development, Infrastructure as Code, OpenTofu (Terraform), CI/CD, Apache Airflow, SQL, Data Modeling, Property and Casualty Insurance Data