Papigen
Get AI-powered advice on this job and more exclusive features.
Job Description
We are looking for a highly skilled Job Description
We are looking for a highly skilled
Azure Data Engineer
to support enterprise-scale data management and analytics transformation initiatives. This role focuses on delivering data engineering pipelines, platform integrations, and supporting advanced analytics, governance, and data products across Azure-based environments. The engineer will work closely with data architects, governance leads, and analytics teams to develop secure, scalable, and reusable data infrastructure.
You’ll be instrumental in the development of integrated data pipelines, cloud-native services, metadata management, and supporting self-service data products using platforms like
Databricks ,
Microsoft Purview ,
Collibra ,
Informatica IDMC , and
Power BI .
Key Responsibilities
Develop ETL/ELT pipelines using Azure Data Factory, Databricks, Synapse, etc. Build data lake/warehouse architectures for structured and unstructured data. Perform data curation, transformation, validation, and enrichment at scale. Implement monitoring, logging, and data quality checks within data flows. Enable self-service analytics by curating certified datasets and exposing them securely. Deliver reusable, secure, and well-documented APIs for downstream applications. Collaborate with analytics teams to support real-time and batch analytics needs. Integrate data pipelines with visualization platforms such as Power BI or Tableau. Configure Microsoft Purview, Collibra, or Informatica IDMC for data lineage, classification, and policy enforcement. Enable metadata management, data catalogs, and automated lineage capture. Support access control mechanisms (RBAC) and sensitive data tagging. Collaborate with governance and compliance teams to implement regulatory controls. Provide platform operations support including pipeline performance optimization.
Qualifications
6–10 years of experience in data engineering, with at least 3+ years on Azure. Proven hands-on experience in Azure Data Factory, Azure Synapse, Azure SQL, Databricks. Experience with metadata management tools like Microsoft Purview, Collibra, or Informatica IDMC. Strong knowledge of data governance, data catalogs, and compliance frameworks. Experience building scalable data lakes, warehouses, and lakehouses. Familiarity with DevOps practices and CI/CD pipelines using Azure DevOps or GitHub Actions. Proficiency in Python, SQL, Spark (PySpark or Scala). Understanding of security practices (data masking, RBAC, PII handling). Good communication and documentation skills in agile team environments.
Skills: spark (pyspark or scala),sql,problem solving,python,data governance,pyspark,microsoft purview,etl,databricks,semantic layers,adf,data engineering,collibra,azure synapse,informatica,agile,azure databricks,azure devops,informatica idmc,power bi,azure data factory,github actions,ci/cd pipelines,azure sql,data catalogs Seniority level
Seniority level Mid-Senior level Employment type
Employment type Contract Job function
Job function Information Technology Industries IT Services and IT Consulting Referrals increase your chances of interviewing at Papigen by 2x Get notified about new Data Engineer jobs in
Washington, DC . Washington, DC $101,000.00-$135,000.00 2 weeks ago Arlington, VA $118,900.00-$205,600.00 1 day ago Washington, DC $120,000.00-$170,000.00 2 months ago Arlington, VA $80,000.00-$89,000.00 2 months ago North Bethesda, MD $55,000.00-$125,000.00 1 month ago Rockville, MD $112,597.33-$152,810.66 2 days ago Washington DC-Baltimore Area $30.00-$34.00 1 week ago Washington, DC $112,597.33-$152,810.66 2 days ago Fairfax, VA $112,597.33-$152,810.66 2 days ago Silver Spring, MD $95,000.00-$125,000.00 1 week ago Arlington, VA $112,597.33-$152,810.66 2 days ago Washington, DC $80,000.00-$130,000.00 2 days ago Washington, DC $120,000.00-$190,000.00 1 month ago Suitland, MD $85,150.00-$153,925.00 1 week ago Arlington, VA $95,000.00-$120,000.00 7 months ago Washington, DC $145,000.00-$204,000.00 1 week ago We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr
We are looking for a highly skilled Job Description
We are looking for a highly skilled
Azure Data Engineer
to support enterprise-scale data management and analytics transformation initiatives. This role focuses on delivering data engineering pipelines, platform integrations, and supporting advanced analytics, governance, and data products across Azure-based environments. The engineer will work closely with data architects, governance leads, and analytics teams to develop secure, scalable, and reusable data infrastructure.
You’ll be instrumental in the development of integrated data pipelines, cloud-native services, metadata management, and supporting self-service data products using platforms like
Databricks ,
Microsoft Purview ,
Collibra ,
Informatica IDMC , and
Power BI .
Key Responsibilities
Develop ETL/ELT pipelines using Azure Data Factory, Databricks, Synapse, etc. Build data lake/warehouse architectures for structured and unstructured data. Perform data curation, transformation, validation, and enrichment at scale. Implement monitoring, logging, and data quality checks within data flows. Enable self-service analytics by curating certified datasets and exposing them securely. Deliver reusable, secure, and well-documented APIs for downstream applications. Collaborate with analytics teams to support real-time and batch analytics needs. Integrate data pipelines with visualization platforms such as Power BI or Tableau. Configure Microsoft Purview, Collibra, or Informatica IDMC for data lineage, classification, and policy enforcement. Enable metadata management, data catalogs, and automated lineage capture. Support access control mechanisms (RBAC) and sensitive data tagging. Collaborate with governance and compliance teams to implement regulatory controls. Provide platform operations support including pipeline performance optimization.
Qualifications
6–10 years of experience in data engineering, with at least 3+ years on Azure. Proven hands-on experience in Azure Data Factory, Azure Synapse, Azure SQL, Databricks. Experience with metadata management tools like Microsoft Purview, Collibra, or Informatica IDMC. Strong knowledge of data governance, data catalogs, and compliance frameworks. Experience building scalable data lakes, warehouses, and lakehouses. Familiarity with DevOps practices and CI/CD pipelines using Azure DevOps or GitHub Actions. Proficiency in Python, SQL, Spark (PySpark or Scala). Understanding of security practices (data masking, RBAC, PII handling). Good communication and documentation skills in agile team environments.
Skills: spark (pyspark or scala),sql,problem solving,python,data governance,pyspark,microsoft purview,etl,databricks,semantic layers,adf,data engineering,collibra,azure synapse,informatica,agile,azure databricks,azure devops,informatica idmc,power bi,azure data factory,github actions,ci/cd pipelines,azure sql,data catalogs Seniority level
Seniority level Mid-Senior level Employment type
Employment type Contract Job function
Job function Information Technology Industries IT Services and IT Consulting Referrals increase your chances of interviewing at Papigen by 2x Get notified about new Data Engineer jobs in
Washington, DC . Washington, DC $101,000.00-$135,000.00 2 weeks ago Arlington, VA $118,900.00-$205,600.00 1 day ago Washington, DC $120,000.00-$170,000.00 2 months ago Arlington, VA $80,000.00-$89,000.00 2 months ago North Bethesda, MD $55,000.00-$125,000.00 1 month ago Rockville, MD $112,597.33-$152,810.66 2 days ago Washington DC-Baltimore Area $30.00-$34.00 1 week ago Washington, DC $112,597.33-$152,810.66 2 days ago Fairfax, VA $112,597.33-$152,810.66 2 days ago Silver Spring, MD $95,000.00-$125,000.00 1 week ago Arlington, VA $112,597.33-$152,810.66 2 days ago Washington, DC $80,000.00-$130,000.00 2 days ago Washington, DC $120,000.00-$190,000.00 1 month ago Suitland, MD $85,150.00-$153,925.00 1 week ago Arlington, VA $95,000.00-$120,000.00 7 months ago Washington, DC $145,000.00-$204,000.00 1 week ago We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr