Highbrow LLC
AWS Data Bricks Spark/Scala Engineer
Highbrow LLC - Austin, Texas, us, 78716
Work at Highbrow LLC
Overview
- View job
Overview
Job Title:
AWS Data Bricks Spark/Scala Engineer Job ID: 2023-11933 Job Location: Morris Plains, NJ (remote to start); Remote, Memphis, TN (preferred), Morris Plains, NJ, Austin, TX (preferred), St. Louis, MO Job Travel Location(s): # Positions: 1 Employment Type: W2 Candidate Constraints: Duration:Long Term # of Layers:0 Work Eligibility:All Work Authorizations are Permitted Key Technology:
DataBricks, AWS, Spark Job Responsibilities: Drive mentality of building well architected applications for AWS Cloud. Drive the mentality of quality being owned by the entire team Identify code defects and work with developers to address quality issues in product code. Finding bottlenecks and thresholds in existing code through the use of automation tools. Articulate clear business objectives aligned to technical specifications and work in an iterative agile pattern daily. Ownership over your work task and are comfortable interacting with all levels of the team and raise challenges when necessary. Skills and Experience Required: 6-8 years of strong experience in Spark, Python, Shell Scripting, PostgreSQL, Hadoop, AWS (S3, EC2, SNS, SQS, Lambda, ECS, Glue, IAM, and CloudWatch), Databricks (Delta lake, Notebooks, Pipelines, cluster management, Azure/AWS integration) In depth knowledge on Hadoop architecture and its components. Troubleshooting the Hadoop Yarn jobs is must. Implement Hadoop job orchestration using Shell scripting, and Airflow In depth understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, RDD caching, Spark MLib. Strong experience in writing complex and effective SQLs, and Stored Procedures. Strong query optimizations skills to improve performance of ETL process. Should have strong hands-on cloud experience with AWS suite Should be able to demonstrate work experience in Hadoop, Spark, hive Should have strong development experience on Databricks components Strong hands-on experience on Spark is must Must have work experience on Hadoop as data warehouse/Data Lake implementations Moderate to senior Terraform development experience needed Health care domain knowledge is a plus
#J-18808-Ljbffr
AWS Data Bricks Spark/Scala Engineer Job ID: 2023-11933 Job Location: Morris Plains, NJ (remote to start); Remote, Memphis, TN (preferred), Morris Plains, NJ, Austin, TX (preferred), St. Louis, MO Job Travel Location(s): # Positions: 1 Employment Type: W2 Candidate Constraints: Duration:Long Term # of Layers:0 Work Eligibility:All Work Authorizations are Permitted Key Technology:
DataBricks, AWS, Spark Job Responsibilities: Drive mentality of building well architected applications for AWS Cloud. Drive the mentality of quality being owned by the entire team Identify code defects and work with developers to address quality issues in product code. Finding bottlenecks and thresholds in existing code through the use of automation tools. Articulate clear business objectives aligned to technical specifications and work in an iterative agile pattern daily. Ownership over your work task and are comfortable interacting with all levels of the team and raise challenges when necessary. Skills and Experience Required: 6-8 years of strong experience in Spark, Python, Shell Scripting, PostgreSQL, Hadoop, AWS (S3, EC2, SNS, SQS, Lambda, ECS, Glue, IAM, and CloudWatch), Databricks (Delta lake, Notebooks, Pipelines, cluster management, Azure/AWS integration) In depth knowledge on Hadoop architecture and its components. Troubleshooting the Hadoop Yarn jobs is must. Implement Hadoop job orchestration using Shell scripting, and Airflow In depth understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, RDD caching, Spark MLib. Strong experience in writing complex and effective SQLs, and Stored Procedures. Strong query optimizations skills to improve performance of ETL process. Should have strong hands-on cloud experience with AWS suite Should be able to demonstrate work experience in Hadoop, Spark, hive Should have strong development experience on Databricks components Strong hands-on experience on Spark is must Must have work experience on Hadoop as data warehouse/Data Lake implementations Moderate to senior Terraform development experience needed Health care domain knowledge is a plus
#J-18808-Ljbffr