Get new jobs for this search by email

Principal Data Engineer (Azure and Databricks) Be among the first 25 applicants. Get AI-powered advice on this job and more exclusive features. Tiger Analytics is a global AI and analytics consulting firm. With data and technology at the core of our solutions, we are solving problems that impact the lives of millions globally. Our culture emphasizes expertise, respect, and a team-first mindset. Headquartered in Silicon Valley, we have delivery centers worldwide, including offices across India, the US, UK, Canada, and Singapore, and a substantial remote global workforce. We are Great Place to Work-Certified. Working at Tiger Analytics means being at the forefront of an AI revolution, collaborating with teams that push boundaries and build inspiring solutions. Requirements Curious about the role? What your typical day would look like: As a Principal Data Engineer (Azure), you should have hands-on experience working with Azure cloud, Databricks, and some exposure to Data Modelling. You will build and learn about various analytics solutions & platforms, data lakes, modern data platforms, data fabric solutions, using open source, Big Data, and Cloud technologies on Microsoft Azure. Design and build scalable & metadata-driven data ingestion pipelines (Batch and Streaming) Develop high-performance data processing for structured and unstructured data, including data harmonization Schedule, orchestrate, and validate pipelines Implement exception handling and log monitoring for debugging Collaborate with peers on tech stack and tools decisions Work with multiple teams (Consulting/Data Science & App Dev) and stakeholders to deliver analytical solutions What do we expect? Experience with Data Lake technologies like Azure Data Factory (ADF), PySpark, Databricks, ADLS, Azure SQL Database Knowledge of Azure Synapse Analytics, Event Hub & Streaming Analytics, Cosmos DB, and Purview Passion for writing high-quality, modular, scalable, bug-free code (SQL, Python, Scala/Java) Ability to collaborate across organization and take ownership of deliverables Experience with big data technologies like Hadoop, Spark, Airflow, NiFi, Kafka, Hive, Neo4J, Elastic Search Understanding of file formats like Delta Lake, Avro, Parquet, JSON, CSV Experience building and designing REST APIs, especially on Data Lake or Lakehouse projects Supporting BI and Data Science teams in secure, governed data consumption Certifications such as Data Engineering on Microsoft Azure (DP-203) or Databricks Certified Developer (DE) are a plus Note: Designation depends on expertise and experience. Compensation is among the best in the industry. Job Requirements Mandatory: Azure Data Factory (ADF), PySpark, Databricks, ADLS, Azure SQL Database Optional: Azure Synapse Analytics, Event Hub & Streaming Analytics, Cosmos DB, Purview Strong programming, unit testing & debugging skills in SQL, Python, or Scala/Java Experience with big data technologies like Hadoop, Spark, Airflow, NiFi, Kafka, Hive, Neo4J, Elastic Search Good understanding of file formats like Delta Lake, Avro, Parquet, JSON, CSV Experience working in Agile projects with DevOps tools like Git, Jenkins, Azure DevOps Nice to have: Data Lake & Lakehouse projects experience, REST services, secure data support for BI/Data Science Benefits This role offers significant career growth opportunities in a fast-paced, entrepreneurial environment with high responsibility. Seniority level Associate Employment type Full-time Job function Information Technology Industries IT Services and IT Consulting #J-18808-Ljbffr