Purple Drive
Senior Data Engineer NiFi & Microservices
Purple Drive, Phoenix, Arizona, United States, 85003
Description: Design and Implementationo Creating and implementing data flow solutions using NiFi, including defining data sources, destinations, and transformation processes.
Ingestion and Transformationo Managing data ingestion from various sources (e.g., files, databases, APIs) and transforming data into desired formats for downstream consumption.
Routing and EnrichmentoConfiguring NiFi to route data to appropriate destinations and enriching data with additional information (e.g., timestamps, location).
Scalability and PerformanceoEnsuring data flows are scalable and performant to handle varying data volumes and velocity.
NiFi Cluster Managemento Maintaining and improving NiFi clusters, including configuration, optimization, and troubleshooting.
Microservices Integrationo Developing and integrating microservices to support data conditioning, format validation, and transformation processes.
Troubleshooting and Issue Resolution identifying and resolving issues related to data flows, NiFi configurations, and data integration processesTechnical SkillsStrong NiFi Expertise Deep understanding of NiFi architecture, processors, controllers, and data flow management capabilities.
Data Integration and ETL Experience with data integration, ETL processes, and data modeling concepts.
Programming Languages Proficiency in Java, Python, or other relevant scripting languages.
Database and SQL Familiarity with databases, SQL, and data manipulation techniques.
Cloud Technologies Experience with cloud platforms (e.g., AWS, Azure, GCP) and their integration with NiFi. Big Data Technologies Knowledge of big data technologies like Kafka.
Ingestion and Transformationo Managing data ingestion from various sources (e.g., files, databases, APIs) and transforming data into desired formats for downstream consumption.
Routing and EnrichmentoConfiguring NiFi to route data to appropriate destinations and enriching data with additional information (e.g., timestamps, location).
Scalability and PerformanceoEnsuring data flows are scalable and performant to handle varying data volumes and velocity.
NiFi Cluster Managemento Maintaining and improving NiFi clusters, including configuration, optimization, and troubleshooting.
Microservices Integrationo Developing and integrating microservices to support data conditioning, format validation, and transformation processes.
Troubleshooting and Issue Resolution identifying and resolving issues related to data flows, NiFi configurations, and data integration processesTechnical SkillsStrong NiFi Expertise Deep understanding of NiFi architecture, processors, controllers, and data flow management capabilities.
Data Integration and ETL Experience with data integration, ETL processes, and data modeling concepts.
Programming Languages Proficiency in Java, Python, or other relevant scripting languages.
Database and SQL Familiarity with databases, SQL, and data manipulation techniques.
Cloud Technologies Experience with cloud platforms (e.g., AWS, Azure, GCP) and their integration with NiFi. Big Data Technologies Knowledge of big data technologies like Kafka.