TechDigital Group
Mandatory Skills: Azure, SQL Server, SSIS
Data Engineering & ETL Development:
- Design, develop, and maintain ETL (Extract, Transform, Load) processes using SQL Server Integration Services (SSIS).
- Build and optimize data pipelines for efficient data extraction, transformation, and loading from various sources.
- Implement incremental and full load ETL strategies to support real-time and batch data processing.
- Ensure data integrity, data quality, and performance optimization across ETL workflows.
Database Development & Optimization:
- Develop and maintain SQL Server databases, stored procedures, functions, and triggers.
- Optimize SQL queries and indexing strategies to improve database performance.
- Implement data partitioning, indexing, and caching strategies for high-performance data processing.
Azure Data Services & Cloud Integration:
- Develop data solutions on Azure, leveraging services such as Azure Data Factory (ADF), Azure SQL Database, Azure Data Lake, and Azure Synapse Analytics.
- Integrate on-premises and cloud-based data sources using Azure Hybrid Data Solutions.
- Automate data workflows using Azure Logic Apps, Functions, and Power Automate.
- Implement data security best practices using Azure Key Vault, encryption, and role-based access control (RBAC).
Data Governance & Reporting:
- Ensure compliance with data governance, security, and privacy regulations.
- Work closely with business intelligence (BI) teams to provide data models and support analytics solutions.
- Create and maintain documentation for data pipelines, ETL workflows, and database schemas.
Collaboration & Continuous Improvement:
- Collaborate with cross-functional teams (Data Analysts, BI Developers, and Software Engineers) to support data-driven decision-making.
- Identify and implement automation strategies to improve data engineering efficiency.
- Stay up-to-date with the latest technologies and best practices in Azure, SQL Server, and ETL development.