DP Professionals Inc
Senior Cloud Data Engineer - AWS & Big Data
DP Professionals Inc, Columbia, South Carolina, us, 29228
DPP
is seeking a
Senior Cloud Data Engineer – AWS & Big Data
for an opportunity in Columbia, SC.
Work arrangement
Hybrid: Partially onsite 3 days/week (Tues., Wed., Thurs.) and as needed.
W2 position; 12 months
Required technologies
Experience in big data technologies
Experience in cloud-based technologies
AWS Services & Technologies:
Orchestration & Infrastructure: State Machines, CloudFormation, CDK
Data Processing & ETL: Glue, Spark, Python
Monitoring & Logging: CloudWatch
Compute & Serverless: Lambda
Storage: S3, Glacier Archival Storage, DataSync
Data Lake & Integration: Lake Formation, AppFlow
Databases & Querying: RDS PostgreSQL, Aurora, Athena
Streaming & Messaging: Amazon MSK
Data Formats & Optimization: Apache Iceberg
Development Languages: TypeScript, Python
Nice to have
AWS Services: Redshift, Sagemaker, Sagemaker Unified Studio, Bedrock, Data Firehose, AppFlow, Lake Formation, Glue, CloudFormation, CloudWatch, Lambda, CDK, S3, Glacier Archival Storage, DataSync, RDS PostgreSQL, Aurora, Athena
Databricks Ecosystem: Delta Lake, Unity Catalog, Data Engineering and Processing using Databricks
AI/ML Tools: Amazon Bedrock, AWS Sagemaker, R Studio / Posit Workbench, R Shiny / Posit Connect, Posit Package Manager
Streaming & Messaging: Amazon MSK, Kafka
Big Data & Processing: Apache Iceberg, Spark, Hive, Hue, Oozie, Sqoop
Development Tools: Git / Git Actions, IntelliJ, Scala
Minimum required work experience
8 years of application development, systems testing, or other job-related experience.
Required education/equivalencies
Bachelor s degree in Computer Science, Information Technology, or other job-related degree;
OR, Associates degree in CS, IT, or other job-related degree plus 2 years of work-related experience;
OR, 4 years job-related work experience (total 12 years without a degree)
Responsibilities
Review and analyze systems, streamlining processes to ensure efficient, consistent use of all resources and adjusting designs to maximize reusability and reduce costs across subsystems. Provide technology solutions and prototypes to be used as the basis or template for more complicated business/technical requirements. Research, diagnose, and develop fixes for defects in existing code. Address maintenance issues related to current development, testing, and production.
Act as an internal consultant, advocate, mentor, and change agent providing expertise and technical guidance on complex projects. Work closely with customers, business analysts, and team members to determine business requirements that drive the analysis and design of quality technical solutions. Ensure solutions are aligned with business and IT strategies and comply with the organization s architectural standards.
Provide design recommendations based on long-term IT organization strategy. Make recommendations towards the development of new code or reuse of existing code. Responsibilities may also include participation in component and data architecture design, performance monitoring, product evaluation and buy versus build recommendations.
Perform analysis, design, programming, and implementation on systems and procedures to solve complex business or scientific problems. Develop enterprise-level applications and custom integration solutions. Evaluate complex interrelationships in immediate programming area to determine how changes in one program will affect another related area.
Develop programming and development standards. Devise new sources of data and develop new approaches and techniques. Involved in the full systems life cycle and is responsible for designing, coding, testing, implementing, maintaining, and supporting application software that is delivered on time and within budget.
Provide guidance to lower-level programmers/analysts. Lead, plan, organize, and/or coordinate complex projects or phases of large projects. Determine and resolve problems with other systems analysts, programmers, and systems users.
Research application development quality control trends throughout the industry.
Interested? Learn more: Click the apply button or contact our recruiter
Kenny
at to learn more about this position (# 25-00627
).
Authorized US Worker
– Those authorized to work in the US are encouraged to apply. We are unable to sponsor at this time.
EOE/AA/V/D
DPP
offers a range of compensation and benefits packages to our employees and their eligible dependents. Call today to learn more about working with
DPP .
#J-18808-Ljbffr
is seeking a
Senior Cloud Data Engineer – AWS & Big Data
for an opportunity in Columbia, SC.
Work arrangement
Hybrid: Partially onsite 3 days/week (Tues., Wed., Thurs.) and as needed.
W2 position; 12 months
Required technologies
Experience in big data technologies
Experience in cloud-based technologies
AWS Services & Technologies:
Orchestration & Infrastructure: State Machines, CloudFormation, CDK
Data Processing & ETL: Glue, Spark, Python
Monitoring & Logging: CloudWatch
Compute & Serverless: Lambda
Storage: S3, Glacier Archival Storage, DataSync
Data Lake & Integration: Lake Formation, AppFlow
Databases & Querying: RDS PostgreSQL, Aurora, Athena
Streaming & Messaging: Amazon MSK
Data Formats & Optimization: Apache Iceberg
Development Languages: TypeScript, Python
Nice to have
AWS Services: Redshift, Sagemaker, Sagemaker Unified Studio, Bedrock, Data Firehose, AppFlow, Lake Formation, Glue, CloudFormation, CloudWatch, Lambda, CDK, S3, Glacier Archival Storage, DataSync, RDS PostgreSQL, Aurora, Athena
Databricks Ecosystem: Delta Lake, Unity Catalog, Data Engineering and Processing using Databricks
AI/ML Tools: Amazon Bedrock, AWS Sagemaker, R Studio / Posit Workbench, R Shiny / Posit Connect, Posit Package Manager
Streaming & Messaging: Amazon MSK, Kafka
Big Data & Processing: Apache Iceberg, Spark, Hive, Hue, Oozie, Sqoop
Development Tools: Git / Git Actions, IntelliJ, Scala
Minimum required work experience
8 years of application development, systems testing, or other job-related experience.
Required education/equivalencies
Bachelor s degree in Computer Science, Information Technology, or other job-related degree;
OR, Associates degree in CS, IT, or other job-related degree plus 2 years of work-related experience;
OR, 4 years job-related work experience (total 12 years without a degree)
Responsibilities
Review and analyze systems, streamlining processes to ensure efficient, consistent use of all resources and adjusting designs to maximize reusability and reduce costs across subsystems. Provide technology solutions and prototypes to be used as the basis or template for more complicated business/technical requirements. Research, diagnose, and develop fixes for defects in existing code. Address maintenance issues related to current development, testing, and production.
Act as an internal consultant, advocate, mentor, and change agent providing expertise and technical guidance on complex projects. Work closely with customers, business analysts, and team members to determine business requirements that drive the analysis and design of quality technical solutions. Ensure solutions are aligned with business and IT strategies and comply with the organization s architectural standards.
Provide design recommendations based on long-term IT organization strategy. Make recommendations towards the development of new code or reuse of existing code. Responsibilities may also include participation in component and data architecture design, performance monitoring, product evaluation and buy versus build recommendations.
Perform analysis, design, programming, and implementation on systems and procedures to solve complex business or scientific problems. Develop enterprise-level applications and custom integration solutions. Evaluate complex interrelationships in immediate programming area to determine how changes in one program will affect another related area.
Develop programming and development standards. Devise new sources of data and develop new approaches and techniques. Involved in the full systems life cycle and is responsible for designing, coding, testing, implementing, maintaining, and supporting application software that is delivered on time and within budget.
Provide guidance to lower-level programmers/analysts. Lead, plan, organize, and/or coordinate complex projects or phases of large projects. Determine and resolve problems with other systems analysts, programmers, and systems users.
Research application development quality control trends throughout the industry.
Interested? Learn more: Click the apply button or contact our recruiter
Kenny
at to learn more about this position (# 25-00627
).
Authorized US Worker
– Those authorized to work in the US are encouraged to apply. We are unable to sponsor at this time.
EOE/AA/V/D
DPP
offers a range of compensation and benefits packages to our employees and their eligible dependents. Call today to learn more about working with
DPP .
#J-18808-Ljbffr