Senior Data Engineer (Azure & Databricks)

Senior Data Engineer – Azure & Databricks Industry:Technology Consulting / Digital Transformation Role Level:Lead II – Software Engineering Location:Alpharetta, Georgia (USA) Employment Type:Full-Time About the Role: We are seeking an experiencedSenior Data Engineer – Azure & Databrickswith a proven track record (8+ years) of designing, building, and scaling modern data platforms within theAzure ecosystem. This role is ideal for someone who thrives in complex enterprise environments, is deeply hands-on with Databricks and Spark, and can lead end-to-end data engineering initiatives from architecture to implementation. You will be responsible for building robust, high-performance data pipelines, establishing engineering best practices, and collaborating with cross-functional teams to deliver reliable, scalable data solutions across both batch and real-time streams. Key Responsibilities: Architect, design, and implement scalable data platforms and pipelines usingAzureandDatabricks. Build and optimise ingestion, transformation, and processing workflows forbatch and real-time(streaming) use cases. Work extensively withADLS,Delta Lake, andSpark (Python/PySpark)to enable large-scale data engineering capabilities. Lead the development of complexETL/ELT pipelines, ensuring performance, reliability, and code quality. Design conceptual, logical, and physicaldata modelsto support analytics and operational workloads. Work with relational and lakehouse systems, includingPostgreSQLandDelta Lake. Define, implement, and enforce best practices fordata governance,security,quality, andarchitecture standards. Partner closely with architects, data scientists, analysts, and business teams to translate requirements into scalable technical solutions. Troubleshoot production issues, drive performance optimisation, and support continuous platform improvements. Mentor junior engineers and contribute to the creation of reusable components and engineering standards. Required Qualifications: 8+ yearsof hands-on data engineering experience in enterprise environments. Strong expertise inAzure services, particularly Azure Databricks, Azure Functions, and (preferably) Azure Data Factory. Advanced proficiency inApache Spark with Python (PySpark). Strong SQL skills, includingquery optimisationandperformance tuning. Deep experience withETL/ELT methodologies, scheduling, and data orchestration. Hands-on expertise withDelta Lake(ACID, schema evolution, performance tuning). Strong understanding ofdata modelling(normalised, dimensional, lakehouse). Proven experience withbatch and streamingtechnologies such as Kafka or Event Hub. Solid grasp ofdata architecture, distributed systems, and cloud-native design patterns. Ability to design and evaluate end-to-end technical solutions and recommend best-fit architectures. Excellent analytical, problem-solving, and communication skills. Ability to collaborate across teams and lead technical discussions. Preferred Skills: Experience withCI/CDtools such as Azure DevOps and Git. Familiarity withInfrastructure-as-Code(Terraform, ARM). Exposure todata governanceand metadata cataloguing tools (e.g., Azure Purview). Experience supportingmachine learning or BI workloadson Databricks. Benefits: For Full-Time, Regular Employees Minimum10 days paid vacationannually 6 days paid sick leave(prorated for new hires) 10 paid holidays Paid bereavement and jury duty leave Eligibility for401(k) Retirement Planwith employer match Medical, dental, and vision insurance eligibility (employee + dependents) Company-paid: Basic life insurance Accidental death & disability coverage Short- and long-term disability Access to HSA and FSA programs (healthcare, dependent care, commuting)

#J-18808-Ljbffr