Data Engineer

Job Title: Data Engineer

Location:

Corporate Headquarters - The Woodlands, TX Department:

Operations Applications Reports To:

Executive Director of Software Development Job Summary: We are seeking a highly skilled and experienced

Data Engineer

to join our data engineering team. The ideal candidate will have deep expertise in building scalable data pipelines, optimizing big data workflows, and integrating Databricks with AWS services. You will play a key role in designing and implementing cloud-native data solutions that drive business insights and innovation. Key Responsibilities: Proficient in

Databricks SQL

for executing complex ad-hoc queries on large-scale lake house datasets, enabling rapid data exploration, trend identification, and actionable insights for business decision-making. Skilled in leveraging

Databricks notebooks

to design and implement scalable data transformation workflows, integrating PySpark and SQL to cleanse, enrich, and prepare large datasets for downstream analytics and reporting. Design, develop, and maintain scalable data pipelines using

Apache Spark

on

Databricks . Architect and implement

ETL/ELT workflows

leveraging

AWS services

such as S3, Glue, Lambda, Redshift, and EMR. Optimize Spark jobs for performance and cost-efficiency in a cloud environment. Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver robust solutions. Implement CI/CD pipelines for Databricks notebooks and jobs using tools like

GitHub Actions ,

Azure DevOps , or

Jenkins . Ensure data quality, security, and governance using tools like

Unity Catalog ,

Delta Lake , and

AWS Lake Formation . Monitor and troubleshoot production data pipelines and jobs. Mentor junior engineers and contribute to best practices and standards. Required Qualifications: Bachelor's or Master's degree in Computer Science, Engineering, or a related field. 5+ years of experience in software/data engineering with at least 2 years working with

Databricks

and

Apache Spark . Strong proficiency in

Python ,

SQL , and

PySpark . Deep understanding of

AWS cloud architecture

and services (especially S3, Glue, Lambda, IAM, Redshift, and CloudWatch). Experience with

Delta Lake ,

Databricks Workflows , and

Databricks SQL . Familiarity with

data modeling ,

data warehousing , and

data lakehouse architectures . Experience with

infrastructure-as-code

tools like

Terraform

or

CloudFormation . Strong problem-solving skills and ability to work in a fast-paced, agile environment. Preferred Qualifications: Databricks Certified Data Engineer or AWS Certified Solutions Architect. Experience with

streaming data

using

Kafka ,

Kinesis , or

Structured Streaming . Knowledge of

MLflow ,

feature stores , or MLOps practices. Familiarity with

data governance

and

compliance frameworks

(e.g., GDPR, HIPAA). Soft Skills: Excellent communication and collaboration skills. Strong analytical thinking and attention to detail. Ability to mentor and lead technical discussions. What's In It for You? You will be joining a high profile team and a great company known for its Servant Leadership and "work hard, play harder" culture. Competitive compensation package includes base salary plus annual cash bonus and equity grant (restricted share units) eligibility. We also offer excellent family benefits including: medical, dental, vision, flexible spending account, long term and short term disability, life insurance, 401(k) retirement and unlimited opportunities to "Connect with Your Future." Waste Connections is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to disability or protected veteran status.

#J-18808-Ljbffr