ALLDATA
Join to apply for the
Sr. Data Platform Engineer
role at
ALLDATA
Job Description Hybrid role 3X a week in office in Elk Grove, CA; no remote capabilities
The Senior Data Platform Engineer is a technical expert responsible for designing, implementing, and supporting scalable data solutions using modern big data technologies. In this full-time role, you will work closely with data architects, analysts, and business stakeholders to build, optimize, and maintain data pipelines and platforms that drive analytics, reporting, and machine learning. You will ensure the reliability, performance, and security of production data systems, and play a key role in troubleshooting, monitoring, and continuous improvement.
Position Responsibilities – Other Duties May Be Assigned
Design, develop, and maintain robust data pipelines for ingesting, transforming, and storing large volumes of structured, semi-structured, and unstructured data.
Implement data workflows and ETL processes using technologies such as Spark, Delta Lake, and cloud-native tools.
Support the production use of big data platforms (e.g., Databricks, Snowflake, Google Cloud Platform), ensuring high availability, scalability, and performance.
Monitor, troubleshoot, and resolve issues in production data systems, including job failures, performance bottlenecks, and data quality concerns.
Collaborate with data architects to translate business requirements into technical solutions, ensuring alignment with architectural standards and best practices.
Optimize SQL queries, Spark jobs, and data processing workloads for efficiency and cost-effectiveness.
Implement and maintain data governance, security, and compliance measures, including access controls, data masking, and audit logging.
Integrate data workflows into CI/CD pipelines, automate deployment processes, and manage source control using Git and related DevOps tools.
Document data pipelines, workflows, and operational procedures to support knowledge sharing and maintainability.
Stay current with emerging big data technologies and recommend improvements to enhance reliability, scalability, and efficiency.
Position Requirements
7+ years of experience in data engineering or related roles, with hands‑on experience building and supporting production data pipelines.
Strong proficiency with big data platforms (e.g., Databricks, Snowflake, Google Cloud Platform) and the Spark ecosystem.
Experience with data lakehouse and warehouse architectures, including Delta Lake or similar technologies.
Senior Data Engineer with deep expertise in Retrieval‑Augmented Generation (RAG) systems and advanced chunking strategies for optimizing semantic search and LLM integration.
Advanced skills in PySpark, Python, SQL, and related data processing frameworks.
Experience with cloud storage, messaging/streaming technologies (e.g., Apache Kafka, cloud Pub/Sub), and vector databases.
Proven ability to optimize ETL jobs, tune cluster configurations, and troubleshoot production issues.
Familiarity with data governance tools, catalog solutions, and security best practices.
Experience integrating data workflows into CI/CD pipelines and using DevOps tools (e.g., Git, Jenkins, Terraform).
Strong problem‑solving skills, attention to detail, and ability to work independently or as part of a team.
Excellent communication skills to collaborate with cross‑functional teams and document technical solutions.
Experience working in an Agile environment.
Tools & Technologies
Big Data Platforms: Spark, Delta Lake, SQL engines, MLflow (or similar model tracking tools), relational databases.
Data Governance: Data catalog and access control solutions.
Programming & Data Processing: PySpark, Python, SQL.
Cloud Services: Cloud storage, messaging/streaming technologies, vector databases, BigQuery.
DevOps & CI/CD: Git for version control, Jenkins, infrastructure‑as‑code tools (e.g., Terraform).
Other Tools: Project and workflow management tools (e.g., JIRA, Confluence), BI/analytics tools (e.g., Power BI, Looker Studio).
Preferred
Certifications in major big data platforms (e.g., Databricks, Snowflake, Google Cloud Platform) are advantageous.
Exposure to related big data and streaming tools such as Apache Kafka, cloud messaging services, Apache Airflow, and BI/analytics tools is a plus.
Google Professional Cloud Architect or better – Preferred, but not required.
Azure Solutions Architect Expert – Preferred, but not required.
The salary range for this position is $119,000 – $208,000. When extending an offer of employment, ALLDATA considers factors such as scope and responsibilities of the position, the candidate’s work experience, education/training, key skills, internal peer equity, federal, state, and local laws, company financials, as well as external market and organizational considerations. ALLDATA values and is committed to diversity, equity and inclusion.
About Us ALLDATA® is the industry’s #1 choice for unedited original equipment manufacturer (OEM) automotive repair and collision information. Founded in 1986, the Elk Grove, California-based company has more than 115,000 subscribers worldwide who rely on us for access to the OEM‑accurate information and procedures they need for safe and accurate repairs. More than 400,000 technicians trust ALLDATA’s industry‑leading software solutions for faster diagnostics, updated OEM information covering 95% of vehicles on the road today, and simpler shop management. With the support of our parent company, AutoZone, we are an employer of choice for those who are passionate about working for a leader in the automotive software industry.
Benefits At ALLDATA®
Competitive pay
Unrivaled company culture
Medical, dental and vision plans
Exclusive discounts and perks, including an AutoZone in‑store discount
401(k) with company match and Stock Purchase Plan
AutoZoners Living Well Program for free mental health support
Opportunities for career growth
Paid time off
Life, short‑ and long‑term disability insurance options
Health Savings and Flexible Spending Accounts with wellness rewards
Tuition reimbursement
Eligibility and waiting period requirements may apply; benefits for AutoZoners in Puerto Rico, Hawaii, or the U.S. Virgin Islands may differ. Learn more about all that AutoZone has to offer at Careers.AutoZone.com.
AutoZone Equal Opportunity Statement AutoZone, and its subsidiary, ALLDATA are equal opportunity employers. All applicants will be considered for employment without attention to age, race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status, or any other legally protected categories.
Seniority level Mid‑Senior level
Employment type Full‑time
Job function Engineering and Information Technology
Industries Automotive
#J-18808-Ljbffr
Sr. Data Platform Engineer
role at
ALLDATA
Job Description Hybrid role 3X a week in office in Elk Grove, CA; no remote capabilities
The Senior Data Platform Engineer is a technical expert responsible for designing, implementing, and supporting scalable data solutions using modern big data technologies. In this full-time role, you will work closely with data architects, analysts, and business stakeholders to build, optimize, and maintain data pipelines and platforms that drive analytics, reporting, and machine learning. You will ensure the reliability, performance, and security of production data systems, and play a key role in troubleshooting, monitoring, and continuous improvement.
Position Responsibilities – Other Duties May Be Assigned
Design, develop, and maintain robust data pipelines for ingesting, transforming, and storing large volumes of structured, semi-structured, and unstructured data.
Implement data workflows and ETL processes using technologies such as Spark, Delta Lake, and cloud-native tools.
Support the production use of big data platforms (e.g., Databricks, Snowflake, Google Cloud Platform), ensuring high availability, scalability, and performance.
Monitor, troubleshoot, and resolve issues in production data systems, including job failures, performance bottlenecks, and data quality concerns.
Collaborate with data architects to translate business requirements into technical solutions, ensuring alignment with architectural standards and best practices.
Optimize SQL queries, Spark jobs, and data processing workloads for efficiency and cost-effectiveness.
Implement and maintain data governance, security, and compliance measures, including access controls, data masking, and audit logging.
Integrate data workflows into CI/CD pipelines, automate deployment processes, and manage source control using Git and related DevOps tools.
Document data pipelines, workflows, and operational procedures to support knowledge sharing and maintainability.
Stay current with emerging big data technologies and recommend improvements to enhance reliability, scalability, and efficiency.
Position Requirements
7+ years of experience in data engineering or related roles, with hands‑on experience building and supporting production data pipelines.
Strong proficiency with big data platforms (e.g., Databricks, Snowflake, Google Cloud Platform) and the Spark ecosystem.
Experience with data lakehouse and warehouse architectures, including Delta Lake or similar technologies.
Senior Data Engineer with deep expertise in Retrieval‑Augmented Generation (RAG) systems and advanced chunking strategies for optimizing semantic search and LLM integration.
Advanced skills in PySpark, Python, SQL, and related data processing frameworks.
Experience with cloud storage, messaging/streaming technologies (e.g., Apache Kafka, cloud Pub/Sub), and vector databases.
Proven ability to optimize ETL jobs, tune cluster configurations, and troubleshoot production issues.
Familiarity with data governance tools, catalog solutions, and security best practices.
Experience integrating data workflows into CI/CD pipelines and using DevOps tools (e.g., Git, Jenkins, Terraform).
Strong problem‑solving skills, attention to detail, and ability to work independently or as part of a team.
Excellent communication skills to collaborate with cross‑functional teams and document technical solutions.
Experience working in an Agile environment.
Tools & Technologies
Big Data Platforms: Spark, Delta Lake, SQL engines, MLflow (or similar model tracking tools), relational databases.
Data Governance: Data catalog and access control solutions.
Programming & Data Processing: PySpark, Python, SQL.
Cloud Services: Cloud storage, messaging/streaming technologies, vector databases, BigQuery.
DevOps & CI/CD: Git for version control, Jenkins, infrastructure‑as‑code tools (e.g., Terraform).
Other Tools: Project and workflow management tools (e.g., JIRA, Confluence), BI/analytics tools (e.g., Power BI, Looker Studio).
Preferred
Certifications in major big data platforms (e.g., Databricks, Snowflake, Google Cloud Platform) are advantageous.
Exposure to related big data and streaming tools such as Apache Kafka, cloud messaging services, Apache Airflow, and BI/analytics tools is a plus.
Google Professional Cloud Architect or better – Preferred, but not required.
Azure Solutions Architect Expert – Preferred, but not required.
The salary range for this position is $119,000 – $208,000. When extending an offer of employment, ALLDATA considers factors such as scope and responsibilities of the position, the candidate’s work experience, education/training, key skills, internal peer equity, federal, state, and local laws, company financials, as well as external market and organizational considerations. ALLDATA values and is committed to diversity, equity and inclusion.
About Us ALLDATA® is the industry’s #1 choice for unedited original equipment manufacturer (OEM) automotive repair and collision information. Founded in 1986, the Elk Grove, California-based company has more than 115,000 subscribers worldwide who rely on us for access to the OEM‑accurate information and procedures they need for safe and accurate repairs. More than 400,000 technicians trust ALLDATA’s industry‑leading software solutions for faster diagnostics, updated OEM information covering 95% of vehicles on the road today, and simpler shop management. With the support of our parent company, AutoZone, we are an employer of choice for those who are passionate about working for a leader in the automotive software industry.
Benefits At ALLDATA®
Competitive pay
Unrivaled company culture
Medical, dental and vision plans
Exclusive discounts and perks, including an AutoZone in‑store discount
401(k) with company match and Stock Purchase Plan
AutoZoners Living Well Program for free mental health support
Opportunities for career growth
Paid time off
Life, short‑ and long‑term disability insurance options
Health Savings and Flexible Spending Accounts with wellness rewards
Tuition reimbursement
Eligibility and waiting period requirements may apply; benefits for AutoZoners in Puerto Rico, Hawaii, or the U.S. Virgin Islands may differ. Learn more about all that AutoZone has to offer at Careers.AutoZone.com.
AutoZone Equal Opportunity Statement AutoZone, and its subsidiary, ALLDATA are equal opportunity employers. All applicants will be considered for employment without attention to age, race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status, or any other legally protected categories.
Seniority level Mid‑Senior level
Employment type Full‑time
Job function Engineering and Information Technology
Industries Automotive
#J-18808-Ljbffr