Infinity Consulting Solutions
We have partnered with our client in their search for a Data Engineer in Plano, TX.
Responsibilities Design, build, and maintain scalable data pipelines
to support real-time, batch, and streaming data processing using tools such as Python, Java, Spark, and Kafka.
Develop and optimize ETL/ELT workflows
for ingesting, transforming, and integrating large datasets from multiple internal and external sources.
Create and maintain data models
(logical and physical) to support analytics, reporting, and application development.
Implement robust data quality processes
including validation rules, anomaly detection, monitoring, and automated data checks.
Ensure data compliance and governance
by adhering to Company standards for security, data privacy, metadata management, data lineage, and audit requirements.
Collaborate with cross-functional teams
including data scientists, analysts, software engineers, business stakeholders, and architects to deliver high-quality data solutions.
Optimize performance
of data pipelines, distributed systems, and storage solutions to ensure reliability and scalability.
Contribute to modern data platform initiatives , including cloud migration, big data architecture, and adoption of new data technologies.
Develop automation and CI/CD processes
for deployments, testing, and monitoring of data engineering solutions using tools such as Jenkins, Git, and Terraform.
Troubleshoot and resolve pipeline issues , ensuring stability, accuracy, and timely delivery of data across the organization.
Skills Required
Experience on AWS technologies like S3, AWS Glue, EMR, and IAM roles/permissions Experience with one or more query language (e.g., SQL, PL/SQL, DDL, SparkSQL, Scala) Experience working with relational database such as Teradata and handling both structured and unstructured datasets
Preferred / Ideal to have - Proficiency in SQL, Python Experience with NoSQL, non-relational databases / data stores (e.g., object storage, document or key-value stores, graph databases, column-family databases) Experience with Snowflake and Databricks
Good to have - Proficiency in tools such as SQL, Python, Tableau or Power BI Understanding of financial industry metrics and business processes AWS Cloud Practitioner or Developer certifications
Education & Work Experience 5+ years of experience with data modeling, warehousing, analysis & data profiling experience and ability to identify trends and anomalies in the data
Client Industry Fintech
#J-18808-Ljbffr
Responsibilities Design, build, and maintain scalable data pipelines
to support real-time, batch, and streaming data processing using tools such as Python, Java, Spark, and Kafka.
Develop and optimize ETL/ELT workflows
for ingesting, transforming, and integrating large datasets from multiple internal and external sources.
Create and maintain data models
(logical and physical) to support analytics, reporting, and application development.
Implement robust data quality processes
including validation rules, anomaly detection, monitoring, and automated data checks.
Ensure data compliance and governance
by adhering to Company standards for security, data privacy, metadata management, data lineage, and audit requirements.
Collaborate with cross-functional teams
including data scientists, analysts, software engineers, business stakeholders, and architects to deliver high-quality data solutions.
Optimize performance
of data pipelines, distributed systems, and storage solutions to ensure reliability and scalability.
Contribute to modern data platform initiatives , including cloud migration, big data architecture, and adoption of new data technologies.
Develop automation and CI/CD processes
for deployments, testing, and monitoring of data engineering solutions using tools such as Jenkins, Git, and Terraform.
Troubleshoot and resolve pipeline issues , ensuring stability, accuracy, and timely delivery of data across the organization.
Skills Required
Experience on AWS technologies like S3, AWS Glue, EMR, and IAM roles/permissions Experience with one or more query language (e.g., SQL, PL/SQL, DDL, SparkSQL, Scala) Experience working with relational database such as Teradata and handling both structured and unstructured datasets
Preferred / Ideal to have - Proficiency in SQL, Python Experience with NoSQL, non-relational databases / data stores (e.g., object storage, document or key-value stores, graph databases, column-family databases) Experience with Snowflake and Databricks
Good to have - Proficiency in tools such as SQL, Python, Tableau or Power BI Understanding of financial industry metrics and business processes AWS Cloud Practitioner or Developer certifications
Education & Work Experience 5+ years of experience with data modeling, warehousing, analysis & data profiling experience and ability to identify trends and anomalies in the data
Client Industry Fintech
#J-18808-Ljbffr