RSM Solutions, Inc
Base pay range
$110,000.00/yr - $130,000.00/yr
Direct message the job poster from RSM Solutions, Inc.
Data Integration Engineer – Onsite in Irvine, California.
I am Tom Welke, Partner & VP at RSM Solutions, Inc. I have been recruiting technical talent for more than 23 years and have been in the tech space since the 1990s. I write my own job descriptions without relying on AI or bots. I focus on clear, realistic expectations. Technical fit is the highest priority; social fit is also important. The hiring manager is a longtime friend who values continuous learning – a key culture of this environment.
This role requires U.S. Citizenship or Green Card Holder status only. All other visa categories are not eligible.
Responsibilities
Design and implement batch and streaming pipelines in Apache Spark running on Kubernetes and Kubeflow Pipelines to hydrate feature stores and training datasets.
Build high throughput ETL/ELT jobs with SSIS, SSAS, and T SQL against MS SQL Server, applying Data Vault style modeling patterns for auditability.
Integrate source control, build, and release automation using GitHub Actions and Azure DevOps for every pipeline component.
Instrument pipelines with Prometheus exporters and visualize SLA, latency, and error budget metrics to enable proactive alerting.
Create automated data quality and schema drift checks; surface anomalies to support a rapid incident response process.
Use MLflow Tracking and Model Registry to version artifacts, parameters, and metrics for reproducible experiments and safe rollbacks.
Work with data scientists to automate model retraining and deployment triggers within Kubeflow based on data freshness or concept drift signals.
Develop PowerShell and .NET utilities to orchestrate job dependencies, manage secrets, and publish telemetry to Azure Monitor.
Optimize Spark and SQL workloads through indexing, partitioning, and cluster sizing strategies, benchmarking performance in CI pipelines.
Document lineage, ownership, and retention policies; ensure pipelines con‑form to PCI/SOX and internal data governance standards.
Qualifications
At least 6 years of experience building data pipelines in Spark or equivalent.
At least 2 years deploying workloads on Kubernetes/Kubeflow.
At least 2 years of experience with MLflow or similar experiment‑tracking tools.
At least 6 years of experience in T‑SQL, Python/Scala for Spark.
At least 6 years of PowerShell/.NET scripting.
At least 6 years of experience with GitHub, Azure DevOps, Prometheus, Grafana, SSIS/SSAS.
Certifications such as Kubernetes CKA/CKAD, Azure Data Engineer (DP‑203), or MLOps‑focused certifications (e.g., Kubeflow or MLflow) are desirable.
Experience mentoring engineers on best practices in containerized data engineering and MLOps.
Seniority level Mid–Senior level
Employment type Full‑time
Job function Information Technology – Manufacturing and Retail Apparel & Fashion
Benefits
Medical insurance
Vision insurance
401(k)
Referrals increase your chances of interviewing at RSM Solutions, Inc by 2x.
#J-18808-Ljbffr
Direct message the job poster from RSM Solutions, Inc.
Data Integration Engineer – Onsite in Irvine, California.
I am Tom Welke, Partner & VP at RSM Solutions, Inc. I have been recruiting technical talent for more than 23 years and have been in the tech space since the 1990s. I write my own job descriptions without relying on AI or bots. I focus on clear, realistic expectations. Technical fit is the highest priority; social fit is also important. The hiring manager is a longtime friend who values continuous learning – a key culture of this environment.
This role requires U.S. Citizenship or Green Card Holder status only. All other visa categories are not eligible.
Responsibilities
Design and implement batch and streaming pipelines in Apache Spark running on Kubernetes and Kubeflow Pipelines to hydrate feature stores and training datasets.
Build high throughput ETL/ELT jobs with SSIS, SSAS, and T SQL against MS SQL Server, applying Data Vault style modeling patterns for auditability.
Integrate source control, build, and release automation using GitHub Actions and Azure DevOps for every pipeline component.
Instrument pipelines with Prometheus exporters and visualize SLA, latency, and error budget metrics to enable proactive alerting.
Create automated data quality and schema drift checks; surface anomalies to support a rapid incident response process.
Use MLflow Tracking and Model Registry to version artifacts, parameters, and metrics for reproducible experiments and safe rollbacks.
Work with data scientists to automate model retraining and deployment triggers within Kubeflow based on data freshness or concept drift signals.
Develop PowerShell and .NET utilities to orchestrate job dependencies, manage secrets, and publish telemetry to Azure Monitor.
Optimize Spark and SQL workloads through indexing, partitioning, and cluster sizing strategies, benchmarking performance in CI pipelines.
Document lineage, ownership, and retention policies; ensure pipelines con‑form to PCI/SOX and internal data governance standards.
Qualifications
At least 6 years of experience building data pipelines in Spark or equivalent.
At least 2 years deploying workloads on Kubernetes/Kubeflow.
At least 2 years of experience with MLflow or similar experiment‑tracking tools.
At least 6 years of experience in T‑SQL, Python/Scala for Spark.
At least 6 years of PowerShell/.NET scripting.
At least 6 years of experience with GitHub, Azure DevOps, Prometheus, Grafana, SSIS/SSAS.
Certifications such as Kubernetes CKA/CKAD, Azure Data Engineer (DP‑203), or MLOps‑focused certifications (e.g., Kubeflow or MLflow) are desirable.
Experience mentoring engineers on best practices in containerized data engineering and MLOps.
Seniority level Mid–Senior level
Employment type Full‑time
Job function Information Technology – Manufacturing and Retail Apparel & Fashion
Benefits
Medical insurance
Vision insurance
401(k)
Referrals increase your chances of interviewing at RSM Solutions, Inc by 2x.
#J-18808-Ljbffr