Alivia Analytics
Job Title
Senior Data Engineer
Company Alivia Analytics
Location Newtown Square, PA
Job Description We are seeking a highly skilled Senior Data Engineer with at least 5 years of experience in designing, developing, documenting, and integrating applications using Spark-based big data platforms like Databricks. The successful candidate will have expertise in deploying these pipelines to cloud infrastructure hosted in AWS or Azure.
Responsibilities
Gather requirements from business/user groups to analyze, design, develop, and implement data pipelines according to customer requirements
Process data from Azure/AWS data storage using Spark-based ETL tooling
Optimize table design and indexing for end‑user ease of use as well as workload performance
Work with various input file‑formats including delimited text files, log files, Parquet files, JSON files, XML files, Excel files and others
Develop automated ETL procedures to load data from various sources into our application’s data warehouse
Ensure pipeline structure is standardized across different customers, each with its own unique input data format
Configure monitoring systems to detect failure and performance degradation of ETL pipelines
Work with DevOps team to design CI/CD pipelines to conduct ETL upgrades
Deploy and leverage cloud infrastructure and services to assist in ETL pipeline definition and automation
Understand data modeling (Dimensional & Relational) concepts such as Star‑Schema, Fact, Dimension tables
Have strong knowledge of both SQL and NoSQL databases
Collaborate with business partners, operations, senior management, etc. on day‑to‑day operational support
Work with high volumes of data with stringent performance requirements
Use programming languages like Python to clean raw data before processing (e.g., removing new line characters/delimiters within fields)
Define data quality and validation checks to preemptively detect potential issues
Ensure ETL pipelines are HIPAA‑compliant, run with minimal permissions, and securely manage any passwords and secrets used for authentication
Document ETL pipeline logic, structure, and field lineage for review by both technical and non‑technical audiences
Requirements
5+ years of experience in designing, developing, documenting, and integrating applications using Spark‑based ETL platforms like Databricks
Extensive experience working on both Azure and AWS
Strong experience in cleaning, pipelining, and analyzing large data sets
Adept in programming languages such as Python and Scala
Experience with git for version control
Excellent problem‑solving skills and ability to work independently and as part of a team
Strong communication and collaboration skills, with ability to work with stakeholders from different backgrounds and levels of expertise
Company Description Alivia Analytics is helping customers achieve Healthcare Payment Integrity by turning mountains of data into actionable answers, delivering the accuracy, confidence and speed needed to solve payment integrity challenges. Through the Alivia Analytics Healthcare Payment Integrity Suite TM, we help private and public healthcare payers achieve payment integrity globally. In the US alone, up to 10% of every dollar spent is attributed to Fraud, Waste or Abuse, amounting to up to $370 Billion dollars lost annually. If your ambition is to grow your responsibilities and career while building world‑class analytic SaaS systems and fixing a huge problem for social good, we invite you to join us.
Seniority Level
Mid‑Senior level
Employment Type
Full‑time
Job Function
Information Technology
Hospitals and Health Care
#J-18808-Ljbffr
Company Alivia Analytics
Location Newtown Square, PA
Job Description We are seeking a highly skilled Senior Data Engineer with at least 5 years of experience in designing, developing, documenting, and integrating applications using Spark-based big data platforms like Databricks. The successful candidate will have expertise in deploying these pipelines to cloud infrastructure hosted in AWS or Azure.
Responsibilities
Gather requirements from business/user groups to analyze, design, develop, and implement data pipelines according to customer requirements
Process data from Azure/AWS data storage using Spark-based ETL tooling
Optimize table design and indexing for end‑user ease of use as well as workload performance
Work with various input file‑formats including delimited text files, log files, Parquet files, JSON files, XML files, Excel files and others
Develop automated ETL procedures to load data from various sources into our application’s data warehouse
Ensure pipeline structure is standardized across different customers, each with its own unique input data format
Configure monitoring systems to detect failure and performance degradation of ETL pipelines
Work with DevOps team to design CI/CD pipelines to conduct ETL upgrades
Deploy and leverage cloud infrastructure and services to assist in ETL pipeline definition and automation
Understand data modeling (Dimensional & Relational) concepts such as Star‑Schema, Fact, Dimension tables
Have strong knowledge of both SQL and NoSQL databases
Collaborate with business partners, operations, senior management, etc. on day‑to‑day operational support
Work with high volumes of data with stringent performance requirements
Use programming languages like Python to clean raw data before processing (e.g., removing new line characters/delimiters within fields)
Define data quality and validation checks to preemptively detect potential issues
Ensure ETL pipelines are HIPAA‑compliant, run with minimal permissions, and securely manage any passwords and secrets used for authentication
Document ETL pipeline logic, structure, and field lineage for review by both technical and non‑technical audiences
Requirements
5+ years of experience in designing, developing, documenting, and integrating applications using Spark‑based ETL platforms like Databricks
Extensive experience working on both Azure and AWS
Strong experience in cleaning, pipelining, and analyzing large data sets
Adept in programming languages such as Python and Scala
Experience with git for version control
Excellent problem‑solving skills and ability to work independently and as part of a team
Strong communication and collaboration skills, with ability to work with stakeholders from different backgrounds and levels of expertise
Company Description Alivia Analytics is helping customers achieve Healthcare Payment Integrity by turning mountains of data into actionable answers, delivering the accuracy, confidence and speed needed to solve payment integrity challenges. Through the Alivia Analytics Healthcare Payment Integrity Suite TM, we help private and public healthcare payers achieve payment integrity globally. In the US alone, up to 10% of every dollar spent is attributed to Fraud, Waste or Abuse, amounting to up to $370 Billion dollars lost annually. If your ambition is to grow your responsibilities and career while building world‑class analytic SaaS systems and fixing a huge problem for social good, we invite you to join us.
Seniority Level
Mid‑Senior level
Employment Type
Full‑time
Job Function
Information Technology
Hospitals and Health Care
#J-18808-Ljbffr