Andiamo
Big Data Software Engineer - Intelligent DataOps Software Leader
Andiamo, San Francisco, California, United States, 94199
Big Data Software Engineer – Intelligent DataOps Software Leader
Big Data Software Engineer — Batch & OLAP
About The Role
Build robust batch pipelines and analytical stores for large-scale reporting, experimentation, and ML feature generation.
Responsibilities
Develop cost‑efficient batch processing (Spark/Dataproc/EMR) and orchestration (Airflow/Dagster).
Design lakehouse tables (Iceberg/Delta/Hudi) with compaction, partitioning, and Z‑ordering.
Implement semantic/metrics layers and OLAP acceleration (BigQuery/Snowflake/ClickHouse).
Enforce governance: lineage, access controls, PII handling, and retention policies.
Requirements
4+ years in big data; expert SQL and performance tuning.
Strong understanding of storage formats, file layout, and query engines.
About Andiamo
Talent Partners for the AI Revolution. As a globally recognized staffing and consulting firm, we specialize in placing the top 2% of technology and go‑to‑market professionals with the world’s largest and most well‑known companies. For over 20 years, we've maintained the status of tier‑one vendor for firms such as Palantir, Amazon, Fluidstack, Bloomberg, Relativity Space, Firefly, MasterCard, Visa, Two Sigma, Citadel, as well as other major financial services firms, elite hedge funds, Google‑backed tech start‑ups, and major software firms. Our talent solutions include Permanent Placement, Contract Staffing, Executive Search, and Dedicated Recruiting Services (RPO).
Seniority level: Mid‑Senior level
Employment type: Full‑time
Job function: Engineering and Information Technology; Industries: Staffing and Recruiting
#J-18808-Ljbffr
About The Role
Build robust batch pipelines and analytical stores for large-scale reporting, experimentation, and ML feature generation.
Responsibilities
Develop cost‑efficient batch processing (Spark/Dataproc/EMR) and orchestration (Airflow/Dagster).
Design lakehouse tables (Iceberg/Delta/Hudi) with compaction, partitioning, and Z‑ordering.
Implement semantic/metrics layers and OLAP acceleration (BigQuery/Snowflake/ClickHouse).
Enforce governance: lineage, access controls, PII handling, and retention policies.
Requirements
4+ years in big data; expert SQL and performance tuning.
Strong understanding of storage formats, file layout, and query engines.
About Andiamo
Talent Partners for the AI Revolution. As a globally recognized staffing and consulting firm, we specialize in placing the top 2% of technology and go‑to‑market professionals with the world’s largest and most well‑known companies. For over 20 years, we've maintained the status of tier‑one vendor for firms such as Palantir, Amazon, Fluidstack, Bloomberg, Relativity Space, Firefly, MasterCard, Visa, Two Sigma, Citadel, as well as other major financial services firms, elite hedge funds, Google‑backed tech start‑ups, and major software firms. Our talent solutions include Permanent Placement, Contract Staffing, Executive Search, and Dedicated Recruiting Services (RPO).
Seniority level: Mid‑Senior level
Employment type: Full‑time
Job function: Engineering and Information Technology; Industries: Staffing and Recruiting
#J-18808-Ljbffr