Highbridge Consulting LLC
Our client, a leading Management Consulting Firm, is seeking to hire a Data Engineer on a 12-week contract starting September 29th. The consultant will lead discovery, migration, and implementation efforts across multiple PostgreSQL and Aurora environments.
Duration:
12 weeks Start Date:
September 29th
Responsibilities
Discovery & Requirements Gathering Assess five databases (Aurora cluster, Postgres clusters with 2 read replicas, and 3 standalone Postgres clusters) across the following dimensions: Availability & Resilience : Review HA/DR setup, RTO/RPO non-functional requirements (NFRs). Performance & Volume : Document instance types, storage class, IOPS, CPU/memory usage, growth trends, and latency/throughput NFRs. Observability & Monitoring : Evaluate CloudWatch metrics, logging, alerts, slow query logs, and performance insights integrations. Security & Compliance : Check encryption (at rest/in transit), IAM controls, audit logging, and compliance alignment. Connectivity & Integration : Map VPC/subnet placement, network latency, replication strategy, and application dependencies.
Migration & Implementation
Migrate a pilot database using Terraform, AWS DMS, and SCT. Develop Terraform modules and CI/CD pipelines for the pilot. Deploy and validate the database in the target account. Configure DMS and SCT for migration and change data capture (CDC) between source and target. Onboard the pilot application to the migration/CDC service. Validate data integrity and ensure no data loss. Promote databases and migrations through SIT and Production environments. Required Skills
DevOps : Terraform, CI/CD pipelines (e.g., Jenkins), Python. Data : AWS DMS, AWS SCT, Aurora, PostgreSQL, schema design/optimization, CDC, migration testing, data validation, and data loss prevention.
Duration:
12 weeks Start Date:
September 29th
Responsibilities
Discovery & Requirements Gathering Assess five databases (Aurora cluster, Postgres clusters with 2 read replicas, and 3 standalone Postgres clusters) across the following dimensions: Availability & Resilience : Review HA/DR setup, RTO/RPO non-functional requirements (NFRs). Performance & Volume : Document instance types, storage class, IOPS, CPU/memory usage, growth trends, and latency/throughput NFRs. Observability & Monitoring : Evaluate CloudWatch metrics, logging, alerts, slow query logs, and performance insights integrations. Security & Compliance : Check encryption (at rest/in transit), IAM controls, audit logging, and compliance alignment. Connectivity & Integration : Map VPC/subnet placement, network latency, replication strategy, and application dependencies.
Migration & Implementation
Migrate a pilot database using Terraform, AWS DMS, and SCT. Develop Terraform modules and CI/CD pipelines for the pilot. Deploy and validate the database in the target account. Configure DMS and SCT for migration and change data capture (CDC) between source and target. Onboard the pilot application to the migration/CDC service. Validate data integrity and ensure no data loss. Promote databases and migrations through SIT and Production environments. Required Skills
DevOps : Terraform, CI/CD pipelines (e.g., Jenkins), Python. Data : AWS DMS, AWS SCT, Aurora, PostgreSQL, schema design/optimization, CDC, migration testing, data validation, and data loss prevention.