Innovya Technologies
Application Architect – data pipeline
Innovya Technologies, Pasadena, California, United States, 91122
We are looking for an experienced developer to lead the design and development of high-performance and scalable enterprise solutions using PySpark, Java, Databricks, etc. The ideal candidate will have over 10 years of experience in software development, designing enterprise‑grade, fault‑tolerant, event‑driven applications. You will work closely with cross‑functional teams to design, build, and deploy complex systems, providing technical direction and expertise to ensure the delivery of robust, efficient, and scalable solutions.
Key Responsibilities
Architect, design, develop and deploy high-performance and scalable data pipe‑line based applications that meet business complex requirements.
Familiarity with streaming architectures and patterns such as event‑driven pipelines, near real‑time scoring, and anomaly monitoring.
Experience working with high‑volume, sensitive data while adhering to security, compliance, and privacy guidelines.
Proficiency in Python for data processing, automation, API integration, anomaly‑detection scripts, and model‑ready dataset preparation.
Lead a team of engineers and work with cross‑functional teams to deliver high‑volume data pipeline and streaming solutions in a timely manner.
Provide technical leadership across all aspects of the software development lifecycle, from initial design through production deployment.
Design and implement data pipelines using PySpark, Databricks, Java, and related tech stack.
Ensure high availability and scalability of systems using Kubernetes, containerization, and cloud infrastructure.
Implement and manage schedulers, event‑driven architecture, and asynchronous processes.
Collaborate with DevOps and infrastructure teams to automate deployment, scaling, and monitoring of applications.
Drive the adoption of best practices in coding, design, testing, and deployment to improve team productivity.
Strong SQL skills, including query optimization, performance tuning, and working with both relational and non‑relational stores.
Required Skills
10+ years of total experience in software development, with at least 5 years in a design lead role.
Deep experience with
PySpark
for distributed data processing, data quality validation, data enrichment, and feature engineering.
Excellent problem‑solving, analytical, and interpersonal skills.
Extensive hands‑on experience with spring‑boot, Kafka, and API development activities.
Experience in designing scalable, distributed systems and microservices architecture.
Familiarity with schedulers, event‑driven architecture, and messaging systems (e.g., Kafka, RabbitMQ).
Proficiency in working with cloud platforms such as AWS and Azure.
Hands‑on experience with caching strategies (ECH), performance tuning, and security best practices.
Experience with version control systems (Git), CI/CD pipelines, and Agile methodologies.
Experience with relational and NoSQL databases.
Soft Skills
Strong communication and collaboration abilities.
Ability to work independently and as part of a team.
A proactive mindset focused on quality and continuous improvement.
Strong analytical and debugging skills.
What We Offer
Competitive salary and benefits plan.
Opportunity to work with modern test automation tools and cloud technologies.
Collaborative and innovative work environment.
Opportunities for professional growth and advancement.
Job Type: Full‑time
Pay: $100,000 - $175,215.97 per year
Paid time off
#J-18808-Ljbffr
Key Responsibilities
Architect, design, develop and deploy high-performance and scalable data pipe‑line based applications that meet business complex requirements.
Familiarity with streaming architectures and patterns such as event‑driven pipelines, near real‑time scoring, and anomaly monitoring.
Experience working with high‑volume, sensitive data while adhering to security, compliance, and privacy guidelines.
Proficiency in Python for data processing, automation, API integration, anomaly‑detection scripts, and model‑ready dataset preparation.
Lead a team of engineers and work with cross‑functional teams to deliver high‑volume data pipeline and streaming solutions in a timely manner.
Provide technical leadership across all aspects of the software development lifecycle, from initial design through production deployment.
Design and implement data pipelines using PySpark, Databricks, Java, and related tech stack.
Ensure high availability and scalability of systems using Kubernetes, containerization, and cloud infrastructure.
Implement and manage schedulers, event‑driven architecture, and asynchronous processes.
Collaborate with DevOps and infrastructure teams to automate deployment, scaling, and monitoring of applications.
Drive the adoption of best practices in coding, design, testing, and deployment to improve team productivity.
Strong SQL skills, including query optimization, performance tuning, and working with both relational and non‑relational stores.
Required Skills
10+ years of total experience in software development, with at least 5 years in a design lead role.
Deep experience with
PySpark
for distributed data processing, data quality validation, data enrichment, and feature engineering.
Excellent problem‑solving, analytical, and interpersonal skills.
Extensive hands‑on experience with spring‑boot, Kafka, and API development activities.
Experience in designing scalable, distributed systems and microservices architecture.
Familiarity with schedulers, event‑driven architecture, and messaging systems (e.g., Kafka, RabbitMQ).
Proficiency in working with cloud platforms such as AWS and Azure.
Hands‑on experience with caching strategies (ECH), performance tuning, and security best practices.
Experience with version control systems (Git), CI/CD pipelines, and Agile methodologies.
Experience with relational and NoSQL databases.
Soft Skills
Strong communication and collaboration abilities.
Ability to work independently and as part of a team.
A proactive mindset focused on quality and continuous improvement.
Strong analytical and debugging skills.
What We Offer
Competitive salary and benefits plan.
Opportunity to work with modern test automation tools and cloud technologies.
Collaborative and innovative work environment.
Opportunities for professional growth and advancement.
Job Type: Full‑time
Pay: $100,000 - $175,215.97 per year
Paid time off
#J-18808-Ljbffr