CopperPoint Insurance Companies
Overview
CopperPoint Insurance Companies is seeking a Sr. Data Engineer to design, build, and optimize the enterprise data lake and analytics platforms using Databricks on AWS. This role integrates with an on-premises MS SQL Server Data Warehouse and enables data consumption through Power BI. You will drive high-performance data pipelines, ensure data quality, and develop data architecture, integration, and modeling strategies to empower the enterprise with advanced analytics and critical business insights. Responsibilities
Design and implement scalable ETL/ELT pipelines using Databricks (PySpark, Delta Lake) on AWS. Leverage AWS services (S3, Lambda, EventBridge, IAM) for data ingestion and orchestration. Build and manage data ingestion architecture (bronze, silver) in Delta Lake. Develop connectors and data movement strategies between on-prem MS SQL Server and cloud environments. Optimize data ingestion from legacy systems to the data lake with security and latency considerations. Implement data models supporting operational and analytical use cases. Ensure data quality, lineage, and governance using Unity Catalog or equivalent tools. Create efficient data views and aggregate tables for reporting performance. Optimize Spark jobs, SQL queries, and Power BI DAX queries for performance. Design, develop, and maintain end-to-end ETL processes using SSIS to load data into data warehouses and marts. Implement logging, error handling, incremental loading, package configuration, and automation via SQL Server Agent and deployment strategies. Guide onshore/offshore development teams, perform code reviews, manage resource planning, and collaborate with business analysts, DBAs, and cross-functional stakeholders. Optimize T-SQL queries, SSIS packages, and SSAS processing. Tune SSRS report execution. Implement automated testing (unit tests, test cases) for ETL workflows. Regularly conduct peer code reviews to ensure code quality, provide expert documentation, and deliver operating guidance for users of all levels; champion high-quality software using TDD and other quality control approaches. Qualifications/Competencies
Bachelor’s degree in computer science, Information Systems, Data Engineering, or a related technical field is required. 6+ years of data engineering experience with focus on cloud and hybrid data platforms. Expertise in Databricks, Delta Lake, with PySpark, Python and Java. Strong experience with AWS cloud services, especially S3, Glue, IAM, Lambda, Step Functions. Proficient in MS SQL Server – performance tuning, stored procedures, and SSIS/ETL migration. Solid knowledge of data modeling, partitioning, and optimization strategies. Experience building CI/CD pipelines for data using tools like Bit Bucket, DevOps, or Jenkins. Technical Expertise in Microsoft Tools – MSSQL, SSIS, SSAS, Batch Script, SSRS. Experience with Unity Catalog, Databricks SQL, and role-based access control (RBAC). Understanding of enterprise data warehouse architecture, slowly changing dimensions, and CDC. Exposure to DevOps or DataOps practices and infrastructure-as-code using Terraform or CloudFormation. Relevant certifications such as: Databricks Certified Data Engineer AWS Certified Data Analytics Azure Data Engineer Associate Power BI Data Analyst Associate Benefits
Employees and their families are eligible to participate in CopperPoint Insurance Companies benefit plans including medical, dental, vision, short-term disability, long-term disability, life insurance, a health savings account, a flexible spending account, and a 401k plan. We provide some benefits at no cost to the employee (Basic Life Insurance and AD&D at two times an employee’s annual salary, Short- and Long-term Disability coverage, and Employee Assistance Plan). Employees will accrue PTO and holidays per year (e.g., 22 days of PTO and 10 paid holidays), aligned with company policy. About CopperPoint
CopperPoint Insurance Companies is a leading provider of workers’ compensation and commercial insurance solutions, with a growing ten-state footprint in the western United States. CopperPoint maintains a drug-free workplace and is an equal employment opportunity employer. All qualified applicants will receive consideration without regard to race, color, sex, religion, age, national origin, disability, veteran status, sexual orientation, gender identity or expression, marital status, ancestry or citizenship status, genetic information, pregnancy status or any other characteristic protected by state, federal or local law.
#J-18808-Ljbffr
CopperPoint Insurance Companies is seeking a Sr. Data Engineer to design, build, and optimize the enterprise data lake and analytics platforms using Databricks on AWS. This role integrates with an on-premises MS SQL Server Data Warehouse and enables data consumption through Power BI. You will drive high-performance data pipelines, ensure data quality, and develop data architecture, integration, and modeling strategies to empower the enterprise with advanced analytics and critical business insights. Responsibilities
Design and implement scalable ETL/ELT pipelines using Databricks (PySpark, Delta Lake) on AWS. Leverage AWS services (S3, Lambda, EventBridge, IAM) for data ingestion and orchestration. Build and manage data ingestion architecture (bronze, silver) in Delta Lake. Develop connectors and data movement strategies between on-prem MS SQL Server and cloud environments. Optimize data ingestion from legacy systems to the data lake with security and latency considerations. Implement data models supporting operational and analytical use cases. Ensure data quality, lineage, and governance using Unity Catalog or equivalent tools. Create efficient data views and aggregate tables for reporting performance. Optimize Spark jobs, SQL queries, and Power BI DAX queries for performance. Design, develop, and maintain end-to-end ETL processes using SSIS to load data into data warehouses and marts. Implement logging, error handling, incremental loading, package configuration, and automation via SQL Server Agent and deployment strategies. Guide onshore/offshore development teams, perform code reviews, manage resource planning, and collaborate with business analysts, DBAs, and cross-functional stakeholders. Optimize T-SQL queries, SSIS packages, and SSAS processing. Tune SSRS report execution. Implement automated testing (unit tests, test cases) for ETL workflows. Regularly conduct peer code reviews to ensure code quality, provide expert documentation, and deliver operating guidance for users of all levels; champion high-quality software using TDD and other quality control approaches. Qualifications/Competencies
Bachelor’s degree in computer science, Information Systems, Data Engineering, or a related technical field is required. 6+ years of data engineering experience with focus on cloud and hybrid data platforms. Expertise in Databricks, Delta Lake, with PySpark, Python and Java. Strong experience with AWS cloud services, especially S3, Glue, IAM, Lambda, Step Functions. Proficient in MS SQL Server – performance tuning, stored procedures, and SSIS/ETL migration. Solid knowledge of data modeling, partitioning, and optimization strategies. Experience building CI/CD pipelines for data using tools like Bit Bucket, DevOps, or Jenkins. Technical Expertise in Microsoft Tools – MSSQL, SSIS, SSAS, Batch Script, SSRS. Experience with Unity Catalog, Databricks SQL, and role-based access control (RBAC). Understanding of enterprise data warehouse architecture, slowly changing dimensions, and CDC. Exposure to DevOps or DataOps practices and infrastructure-as-code using Terraform or CloudFormation. Relevant certifications such as: Databricks Certified Data Engineer AWS Certified Data Analytics Azure Data Engineer Associate Power BI Data Analyst Associate Benefits
Employees and their families are eligible to participate in CopperPoint Insurance Companies benefit plans including medical, dental, vision, short-term disability, long-term disability, life insurance, a health savings account, a flexible spending account, and a 401k plan. We provide some benefits at no cost to the employee (Basic Life Insurance and AD&D at two times an employee’s annual salary, Short- and Long-term Disability coverage, and Employee Assistance Plan). Employees will accrue PTO and holidays per year (e.g., 22 days of PTO and 10 paid holidays), aligned with company policy. About CopperPoint
CopperPoint Insurance Companies is a leading provider of workers’ compensation and commercial insurance solutions, with a growing ten-state footprint in the western United States. CopperPoint maintains a drug-free workplace and is an equal employment opportunity employer. All qualified applicants will receive consideration without regard to race, color, sex, religion, age, national origin, disability, veteran status, sexual orientation, gender identity or expression, marital status, ancestry or citizenship status, genetic information, pregnancy status or any other characteristic protected by state, federal or local law.
#J-18808-Ljbffr