Logo
Apex Systems

Databricks Architect

Apex Systems, Houston, Texas, United States, 77001

Save Job

Senior Lead Consultant

Databricks Platform Architect

Apex Systems is seeking a Senior Lead Consultant to serve as a Databricks Architect, leading greenfield data platform modernization initiatives, helping enterprise clients design and deliver Databricks-based solutions from inception to production. This role is part of Apex Systems' Solutions Delivery Organization and will be responsible for building scalable, cloud-based data and analytics platforms that integrate seamlessly with enterprise environments. As a technical leader, you will drive solution architecture, data migration, and integration strategies, ensuring flexibility, scalability, and adherence to best practices. Additionally, you will provide subject matter expertise to sales and account teams during the scoping of new opportunities. Key Responsibilities: Serve as the technical lead for project teams, ensuring high-quality execution and alignment with best practices. Listen and seek to understand the client and meet their needs, providing consultative guidance. Stay attuned to the future needs of the client and work with internal resources to identify opportunities. Build and foster relationships with clients, stakeholders, and technical communities, representing Apex Systems at industry events. Stay current with technological trends, proactively identifying innovative solutions to improve client outcomes. Act as a mentor to junior engineers and technical team members, fostering skills development and knowledge sharing. Collaborate with internal and external stakeholders to ensure project alignment with business goals and success metrics. Contribute to the evaluation and recruitment of technical talent through professional networking and technical screenings. Lead the end-to-end development of greenfield Databricks-based solutions, from architecture and design to implementation and delivery, ensuring alignment with business and technical requirements. Develop scalable solution architectures, including data platforms, and integrations with enterprise applications and analytics platforms. Advise clients on best practices for Databricks architecture, governance, and security to ensure long-term flexibility and scalability. Drive cloud data platform modernization, leveraging Azure services (Azure Data Factory, Azure Functions, etc.) and complementary technologies. Architect and implement streaming/batch data pipelines that optimize performance, cost, and operational efficiency. Integrate Databricks solutions with enterprise data sources. Implement enterprise-grade governance and security measures, including role-based access control (RBAC), data masking, and compliance frameworks. Optimize Databricks environments, including storage and compute performance, partitioning strategies, and cluster configurations. Act as a trusted advisor to clients and internal teams, guiding them through technical decisions and trade-offs. Support pre-sales efforts, including solution scoping, architecture discussions, and client presentations. Mentor and guide teams of developers and data engineers working within Agile delivery frameworks. Required Skills & Experience: 7+ years of experience in enterprise cloud data platforms and architecture, with at least 3+ years of hands-on experience designing and deploying Databricks solutions. Proven experience leading greenfield Databricks implementations, building platforms from the ground up in enterprise settings. Deep experience in Infrastructure-as-Code technologies like Terraform, Bicep, and ARM Templates. Deep expertise in Azure services that complement Databricks (Azure Data Factory, Azure Functions, Azure Synapse, etc.). Deep expertise in Azure Cloud Networking, Security, Storage. Proficiency in governance and security best practices, including Databricks Unity Catalog, RBAC, data encryption, and compliance controls. Hands-on experience optimizing Databricks environments, including partitioning strategies, cluster sizing, and performance tuning. Familiarity with streaming architectures and real-time data processing (Delta Live Tables, Kafka, etc.) is a plus. Experience working in Agile delivery environments, leading teams through solution design, implementation, and deployment. Strong consultative skills, with the ability to engage with stakeholders, define technical roadmaps, and deliver high-impact solutions. BA/BS required, preferably in Management Information Systems, Computer Information Systems, Computer Science, or related fields (applicable education and experience may be substituted). Preferred Qualifications: Prior experience at Databricks or working directly with Databricks professional services. Databricks technical certifications. Exposure to IDMC (Informatica Intelligent Data Management Cloud) or other data integration platforms. Experience with DataOps concepts and CI/CD pipelines for Databricks deployments. Experience designing and implementing data pipelines using Python, SQL, and Spark (PySpark, Scala, or Java). EEO Employer Apex Systems is an equal opportunity employer. We do not discriminate or allow discrimination on the basis of race, color, religion, creed, sex (including pregnancy, childbirth, breastfeeding, or related medical conditions), age, sexual orientation, gender identity, national origin, ancestry, citizenship, genetic information, registered domestic partner status, marital status, disability, status as a crime victim, protected veteran status, political affiliation, union membership, or any other characteristic protected by law. Apex will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable law. If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation in using our website for a search or application, please contact our Employee Services Department at employeeservices@apexsystems.com.