Databricks
Senior Technical Solutions Engineer (Apache Spark)
Databricks, San Francisco, California, United States, 94199
Overview
As a Senior Spark Technical Solutions Engineer, you will provide a deep dive technical and consulting related solutions for challenging Spark/ML/AI/Delta/Streaming/Lakehouse reported issues by our customers and resolve challenges involving the Databricks unified analytics platform with strong technical and customer communication skills. You will assist our customers in their Databricks journey and provide them with guidance, knowledge, and expertise to realize value and achieve strategic objectives using our products. Outcomes
Perform initial level analysis and troubleshoot issues in Spark using Spark UI metrics, DAG, and Event Logs for various customer-reported job slowness issues. Troubleshoot, resolve, and suggest deep code-level analysis of Spark to address customer issues related to Spark core internals, Spark SQL, Structured Streaming, Delta, Lakehouse, and other Databricks runtime features. Assist customers in setting up reproducible Spark problems with solutions in the areas of Spark SQL, Delta, Memory Management, Performance tuning, Streaming, Data Science, and Data Integration in Spark. Contribute to the development of tools and automation initiatives. Participate in the Designated Solutions Engineer program and drive one or two of a strategic customer’s day-to-day Spark and Cloud issues. Provide best practices guidance around Spark runtime performance and the usage of Spark core libraries and APIs for custom-built solutions developed by Databricks customers. Provide frontline support on third-party integrations with the Databricks environment. Plan and coordinate with Account Executives, Customer Success Engineers, and Resident Solution Architects for coordinating customer issues and best-practices guidelines. Participate in screen-sharing meetings, answer Slack channel conversations with internal stakeholders and customers, and help drive major Spark issues at an individual-contributor level. Review Engineering JIRA tickets and proactively inform the support leadership team for follow-up on action items. Manage assigned Spark cases daily and adhere to committed SLAs. Build an internal wiki and knowledge base with technical documentation and manuals for the support team and customers; participate in the creation and maintenance of company documentation and knowledge base articles. Achieve above and beyond expectations of the support organization KPIs. Coordinate with Engineering and Backline Support teams to assist in identifying and reporting product defects. Be a true proponent of customer advocacy. Participate in weekend and weekday on-call rotation and run escalations during Databricks runtime outages or incident situations, with the ability to multitask and plan day-to-day activities and provide escalated support for critical customer operational issues. Strengthen AWS, Azure, and Databricks platform expertise through continuous learning and internal training programs. Competencies
5+ years of experience designing, building, testing, and maintaining Python/Java/Scala-based applications in typical project delivery and consulting environments. 2+ years of hands-on experience in developing two or more of the following at production scale: Big Data, Hadoop, Spark, Machine Learning, Artificial Intelligence, Streaming, Kafka, Data Science, Data Visualization; Spark experience is mandatory. Hands-on experience in performance tuning and troubleshooting Hive and Spark-based applications at production scale. Proven experience in JVM and memory management techniques such as garbage collection and heap/thread dump analysis is preferred. Working knowledge of Data Lakes and preferably SCD types at production scale. Working hands-on experience with SQL-based databases and data warehousing/ETL technologies (e.g., Informatica, DataStage, Oracle, Teradata, SQL Server, MySQL) is preferred. Linux/Unix administration skills are a plus. Hands-on experience with AWS, Azure, or GCP is preferred. Excellent written and oral communication skills. Demonstrated analytical and problem-solving skills applicable to a distributed big data computing environment. About Databricks
Databricks is the data and AI company. More than 10,000 organizations worldwide—including Comcast, Condé Nast, Grammarly, and over 50% of the Fortune 500—rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics, and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake, and MLflow. To learn more, follow Databricks on Twitter, LinkedIn, and Facebook. Benefits: At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on benefits offered in your region, please visit the official benefits page. Our Commitment to Diversity and Inclusion
At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. We ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Databricks are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics. Compliance
If access to export-controlled technology or source code is required for performance of job duties, it is within the Employer’s discretion to apply for a U.S. government license for such positions, and the Employer may decline to proceed with an applicant on this basis alone.
#J-18808-Ljbffr
As a Senior Spark Technical Solutions Engineer, you will provide a deep dive technical and consulting related solutions for challenging Spark/ML/AI/Delta/Streaming/Lakehouse reported issues by our customers and resolve challenges involving the Databricks unified analytics platform with strong technical and customer communication skills. You will assist our customers in their Databricks journey and provide them with guidance, knowledge, and expertise to realize value and achieve strategic objectives using our products. Outcomes
Perform initial level analysis and troubleshoot issues in Spark using Spark UI metrics, DAG, and Event Logs for various customer-reported job slowness issues. Troubleshoot, resolve, and suggest deep code-level analysis of Spark to address customer issues related to Spark core internals, Spark SQL, Structured Streaming, Delta, Lakehouse, and other Databricks runtime features. Assist customers in setting up reproducible Spark problems with solutions in the areas of Spark SQL, Delta, Memory Management, Performance tuning, Streaming, Data Science, and Data Integration in Spark. Contribute to the development of tools and automation initiatives. Participate in the Designated Solutions Engineer program and drive one or two of a strategic customer’s day-to-day Spark and Cloud issues. Provide best practices guidance around Spark runtime performance and the usage of Spark core libraries and APIs for custom-built solutions developed by Databricks customers. Provide frontline support on third-party integrations with the Databricks environment. Plan and coordinate with Account Executives, Customer Success Engineers, and Resident Solution Architects for coordinating customer issues and best-practices guidelines. Participate in screen-sharing meetings, answer Slack channel conversations with internal stakeholders and customers, and help drive major Spark issues at an individual-contributor level. Review Engineering JIRA tickets and proactively inform the support leadership team for follow-up on action items. Manage assigned Spark cases daily and adhere to committed SLAs. Build an internal wiki and knowledge base with technical documentation and manuals for the support team and customers; participate in the creation and maintenance of company documentation and knowledge base articles. Achieve above and beyond expectations of the support organization KPIs. Coordinate with Engineering and Backline Support teams to assist in identifying and reporting product defects. Be a true proponent of customer advocacy. Participate in weekend and weekday on-call rotation and run escalations during Databricks runtime outages or incident situations, with the ability to multitask and plan day-to-day activities and provide escalated support for critical customer operational issues. Strengthen AWS, Azure, and Databricks platform expertise through continuous learning and internal training programs. Competencies
5+ years of experience designing, building, testing, and maintaining Python/Java/Scala-based applications in typical project delivery and consulting environments. 2+ years of hands-on experience in developing two or more of the following at production scale: Big Data, Hadoop, Spark, Machine Learning, Artificial Intelligence, Streaming, Kafka, Data Science, Data Visualization; Spark experience is mandatory. Hands-on experience in performance tuning and troubleshooting Hive and Spark-based applications at production scale. Proven experience in JVM and memory management techniques such as garbage collection and heap/thread dump analysis is preferred. Working knowledge of Data Lakes and preferably SCD types at production scale. Working hands-on experience with SQL-based databases and data warehousing/ETL technologies (e.g., Informatica, DataStage, Oracle, Teradata, SQL Server, MySQL) is preferred. Linux/Unix administration skills are a plus. Hands-on experience with AWS, Azure, or GCP is preferred. Excellent written and oral communication skills. Demonstrated analytical and problem-solving skills applicable to a distributed big data computing environment. About Databricks
Databricks is the data and AI company. More than 10,000 organizations worldwide—including Comcast, Condé Nast, Grammarly, and over 50% of the Fortune 500—rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics, and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake, and MLflow. To learn more, follow Databricks on Twitter, LinkedIn, and Facebook. Benefits: At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on benefits offered in your region, please visit the official benefits page. Our Commitment to Diversity and Inclusion
At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. We ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Databricks are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics. Compliance
If access to export-controlled technology or source code is required for performance of job duties, it is within the Employer’s discretion to apply for a U.S. government license for such positions, and the Employer may decline to proceed with an applicant on this basis alone.
#J-18808-Ljbffr