Echo IT Solutions, Inc.
Senior Data Engineer
Remote
Contract
Key Responsibilities:
Data Pipeline Development: Design, implement, and maintain ETL processes to extract, transform, and load data from diverse sources into data storage systems. Data Integration: Integrate data from various structured and unstructured sources, including databases, APIs, logs, and external data feeds. Data Modeling: Develop and maintain data models that align with business requirements, ensuring data is stored efficiently and accurately. Data Quality Assurance: Implement data quality checks and validation procedures to ensure data accuracy, consistency, and completeness. Data Warehousing: Build and maintain data warehouses or data lakes, optimizing them for query performance and scalability. Documentation: Create and maintain documentation for data pipelines, data models, and infrastructure configurations. Collaboration: Collaborate with analysts, and other stakeholders to understand data requirements and provide data support for analytics and reporting. Monitoring and Troubleshooting: Set up monitoring systems to proactively identify issues and troubleshoot data pipeline failures or performance bottlenecks. Qualifications:
Bachelor's or higher degree in computer science engineering, information technology, or a related field. Strong programming skills Python and associated libraries: Pandas, SQL Alchemy, Flask, Fast API etc. Proficiency in SQL and experience with relational and NoSQL databases. Knowledge of data warehousing technologies like Amazon Redshift, Google BigQuery, Snowflake, or similar is a Plus Experience with data pipeline orchestration tools like Apache Airflow, Luigi, or similar. Familiarity with cloud platform AWS is a plus. Understanding of data integration techniques, data modeling, and data architecture best practices. Excellent problem-solving and troubleshooting skills. Strong communication and collaboration skills. Knowledge of data security and compliance standards is a plus. Experience with big data technologies (e.g., Hadoop, Spark) is a bonus
Key Responsibilities:
Data Pipeline Development: Design, implement, and maintain ETL processes to extract, transform, and load data from diverse sources into data storage systems. Data Integration: Integrate data from various structured and unstructured sources, including databases, APIs, logs, and external data feeds. Data Modeling: Develop and maintain data models that align with business requirements, ensuring data is stored efficiently and accurately. Data Quality Assurance: Implement data quality checks and validation procedures to ensure data accuracy, consistency, and completeness. Data Warehousing: Build and maintain data warehouses or data lakes, optimizing them for query performance and scalability. Documentation: Create and maintain documentation for data pipelines, data models, and infrastructure configurations. Collaboration: Collaborate with analysts, and other stakeholders to understand data requirements and provide data support for analytics and reporting. Monitoring and Troubleshooting: Set up monitoring systems to proactively identify issues and troubleshoot data pipeline failures or performance bottlenecks. Qualifications:
Bachelor's or higher degree in computer science engineering, information technology, or a related field. Strong programming skills Python and associated libraries: Pandas, SQL Alchemy, Flask, Fast API etc. Proficiency in SQL and experience with relational and NoSQL databases. Knowledge of data warehousing technologies like Amazon Redshift, Google BigQuery, Snowflake, or similar is a Plus Experience with data pipeline orchestration tools like Apache Airflow, Luigi, or similar. Familiarity with cloud platform AWS is a plus. Understanding of data integration techniques, data modeling, and data architecture best practices. Excellent problem-solving and troubleshooting skills. Strong communication and collaboration skills. Knowledge of data security and compliance standards is a plus. Experience with big data technologies (e.g., Hadoop, Spark) is a bonus