Logo
Ccgmag

Software Engineer III - Data Engineering

Ccgmag, Chicago, Illinois, United States, 60290

Save Job

The Planning and Analysis - Data Platform Team (FPNAE) is at the forefront of managing and optimizing finance-related data. We focus on effective data lake utilization, data governance, and seamless data integration through advanced ETL processes. Our goal is to deliver high-quality, timely data in a controlled manner, reducing risk and cost. We are looking for a Senior Application Engineer skilled in AWS, event-driven architecture, RESTful API design, and modern application development, with expertise in Python frameworks, database management, and Databricks for data engineering.

Key Responsibilities

Design and Develop Cloud-Native Applications

Architect, build, and maintain scalable applications on AWS using event-driven patterns (e.g., Lambda, SNS/SQS, EventBridge).

API Development

Create robust RESTful APIs using Python frameworks such as FastAPI and Django, ensuring high performance and security.

Database Engineering

Design, implement, and optimize operational databases, primarily PostgreSQL, including schema design, indexing, and query optimization.

Data Engineering & Analytics

Collaborate with data teams to integrate applications with Databricks, supporting ETL pipelines, data processing, and analytics workflows.

Application Lifecycle Management

Participate in the full software development lifecycle, including requirements gathering, design, coding, testing, deployment, and maintenance.

DevOps & Automation:

Implement CI/CD pipelines, automate deployments, and monitor application health using AWS-native and third-party tools.

Collaboration & Documentation:

Work closely with cross-functional teams (product, data, infrastructure) and document technical solutions, architecture, and best practices.

Required Qualifications, Capabilities and Skills

AWS Expertise

Proven experience with AWS services (EC2, Lambda, S3, RDS, SNS/SQS, EventBridge, IAM, CloudFormation).

Event-Driven Architecture

Strong understanding and practical experience designing event-driven systems and microservices.

RESTful API Development

Deep knowledge of REST principles and hands-on experience building APIs with FastAPI and Django.

Python Programming

Advanced proficiency in Python, with experience in application engineering and scripting.

Database Management

Solid experience with PostgreSQL, including performance tuning and operational management.

Databricks Knowledge

Good understanding of Databricks platform, Spark, and data engineering concepts.

DevOps Practices

Experience with CI/CD, infrastructure as code, and application monitoring.

Soft Skills

Excellent problem-solving, communication, and teamwork abilities.

#J-18808-Ljbffr