PortPro
Director of Software Engineering (Node.js & Web Scraping Expert)
PortPro, Los Angeles, California, United States, 90079
Director of Software Engineering (Node.js & Web Scraping Expert)
Join to apply for the
Director of Software Engineering (Node.js & Web Scraping Expert)
role at
PortPro Continue with Google Continue with Google Director of Software Engineering (Node.js & Web Scraping Expert)
Join to apply for the
Director of Software Engineering (Node.js & Web Scraping Expert)
role at
PortPro We are seeking a Director of Software Engineering with deep expertise in Node.js development and large-scale web scraping. This role will lead the engineering team, designing and optimizing high-performance, distributed web scraping systems. The ideal candidate has extensive experience in handling anti-bot measures, data pipeline optimization, and scalable cloud-based architectures.
Key Responsibilities- Software Engineering & Web Scraping Leadership:
Architect, develop, and maintain scalable and distributed web scraping systems using Node.js Design and implement data extraction pipelines to process large volumes of structured and unstructured data Develop solutions to bypass anti-bot mechanisms, including CAPTCHA handling, session management, fingerprinting, and IP rotation Optimize scraping processes for performance, reliability, and efficiency while managing proxy services(residential, datacenter, rotating).Oversee data storage and processing strategies, ensuring high availability and consistency Collaborate with Product, DevOps, and Data Science teams to integrate extracted data into analytics and business applications Implement best practices for microservices, API integrations, and real-time data streaming
Key Responsibilities- Scalability, Security & DevOps:
Lead the transition to cloud-native, containerized, and serverless architectures for web scraping Ensure compliance with legal and ethical standards (robots.txt, GDPR, CCPA, etc.).Optimize cloud resources (AWS, GCP, or Azure) to support high-throughput scraping Manage real-time monitoring and alerting systems to detect scraping failures, IP bans, or performance bottlenecks Work closely with DevOps teams to optimize CI/CD pipelines, automated deployments, and system scalability
Key Repsonsibilities- Engineering Team Management & Strategy:
Lead, mentor, and grow a high-performance engineering team Define and execute the technology roadmap, aligning with business objectives Foster a culture of continuous learning, collaboration, and innovation Implement agile development methodologies (Scrum, Kanban) to optimize project execution Ensure code quality, security, and best practices across all engineering efforts
Qualifications & Experience- Technical Expertise:
10+ years of experience in software engineering, with at least 5+ years in web scraping and large-scale data extraction Strong hands-on expertise in Node.js, Puppeteer, Playwright, Cheerio, Selenium, and headless browser automation Extensive experience in handling CAPTCHAs, IP rotation, session management, and anti-bot evasion techniques Deep knowledge of proxy management (residential, datacenter, rotating, and VPNs).Experience with NoSQL/SQL databases (MongoDB, PostgreSQL, Redis, Elasticsearch, etc.) Familiarity with data processing frameworks (Kafka, RabbitMQ, Spark, Airflow, etc.).Strong experience with CI/CD, containerization (Docker, Kubernetes), and cloud deployment (AWS/GCP/Azure)
Qualifications & Experience- Leadership & Soft Skills:
Proven track record of scaling engineering teams and leading complex projects Strong problem-solving and debugging skills, especially for scraping challenges and performance bottlenecks Excellent communication and stakeholder management skills Passion for mentorship, team development, and continuous learning
Preferred Qualifications:
Experience with machine learning for data extraction and NLP Knowledge of browser fingerprinting and bot detection mechanisms Familiarity with enterprise-scale web crawling frameworks (Scrapy, Colly, Apify, etc.) Prior leadership experience in data-driven businesses or web scraping startups
Seniority level
Seniority level Not Applicable Employment type
Employment type Full-time Job function
Job function Engineering and Information Technology Industries Truck Transportation Referrals increase your chances of interviewing at PortPro by 2x Sign in to set job alerts for “Director of Software Engineering” roles.
Continue with Google Continue with Google Continue with Google Continue with Google Director of Engineering, Operations Tools Suite
Director of Engineering, Fleet Management Delivery Platform
Los Angeles Metropolitan Area $210,000 - $250,000 17 hours ago Sr. Director , Security Software Engineering
Beverly Hills, CA $280,000 - $300,000 1 day ago Los Angeles Metropolitan Area 2 weeks ago Los Angeles, CA $119,000.00 - $195,000.00 2 weeks ago Senior Software Engineering Manager - REMOTE
Los Angeles, CA $160,000 - $180,000 11 hours ago Sr. Manager, IT Software Implementation - USA Remote
Sr. Manager, IT Software Implementation - USA Remote
Sr. Director , Security Software Engineering
Beverly Hills, CA $280,000 - $300,000 2 days ago We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr
Join to apply for the
Director of Software Engineering (Node.js & Web Scraping Expert)
role at
PortPro Continue with Google Continue with Google Director of Software Engineering (Node.js & Web Scraping Expert)
Join to apply for the
Director of Software Engineering (Node.js & Web Scraping Expert)
role at
PortPro We are seeking a Director of Software Engineering with deep expertise in Node.js development and large-scale web scraping. This role will lead the engineering team, designing and optimizing high-performance, distributed web scraping systems. The ideal candidate has extensive experience in handling anti-bot measures, data pipeline optimization, and scalable cloud-based architectures.
Key Responsibilities- Software Engineering & Web Scraping Leadership:
Architect, develop, and maintain scalable and distributed web scraping systems using Node.js Design and implement data extraction pipelines to process large volumes of structured and unstructured data Develop solutions to bypass anti-bot mechanisms, including CAPTCHA handling, session management, fingerprinting, and IP rotation Optimize scraping processes for performance, reliability, and efficiency while managing proxy services(residential, datacenter, rotating).Oversee data storage and processing strategies, ensuring high availability and consistency Collaborate with Product, DevOps, and Data Science teams to integrate extracted data into analytics and business applications Implement best practices for microservices, API integrations, and real-time data streaming
Key Responsibilities- Scalability, Security & DevOps:
Lead the transition to cloud-native, containerized, and serverless architectures for web scraping Ensure compliance with legal and ethical standards (robots.txt, GDPR, CCPA, etc.).Optimize cloud resources (AWS, GCP, or Azure) to support high-throughput scraping Manage real-time monitoring and alerting systems to detect scraping failures, IP bans, or performance bottlenecks Work closely with DevOps teams to optimize CI/CD pipelines, automated deployments, and system scalability
Key Repsonsibilities- Engineering Team Management & Strategy:
Lead, mentor, and grow a high-performance engineering team Define and execute the technology roadmap, aligning with business objectives Foster a culture of continuous learning, collaboration, and innovation Implement agile development methodologies (Scrum, Kanban) to optimize project execution Ensure code quality, security, and best practices across all engineering efforts
Qualifications & Experience- Technical Expertise:
10+ years of experience in software engineering, with at least 5+ years in web scraping and large-scale data extraction Strong hands-on expertise in Node.js, Puppeteer, Playwright, Cheerio, Selenium, and headless browser automation Extensive experience in handling CAPTCHAs, IP rotation, session management, and anti-bot evasion techniques Deep knowledge of proxy management (residential, datacenter, rotating, and VPNs).Experience with NoSQL/SQL databases (MongoDB, PostgreSQL, Redis, Elasticsearch, etc.) Familiarity with data processing frameworks (Kafka, RabbitMQ, Spark, Airflow, etc.).Strong experience with CI/CD, containerization (Docker, Kubernetes), and cloud deployment (AWS/GCP/Azure)
Qualifications & Experience- Leadership & Soft Skills:
Proven track record of scaling engineering teams and leading complex projects Strong problem-solving and debugging skills, especially for scraping challenges and performance bottlenecks Excellent communication and stakeholder management skills Passion for mentorship, team development, and continuous learning
Preferred Qualifications:
Experience with machine learning for data extraction and NLP Knowledge of browser fingerprinting and bot detection mechanisms Familiarity with enterprise-scale web crawling frameworks (Scrapy, Colly, Apify, etc.) Prior leadership experience in data-driven businesses or web scraping startups
Seniority level
Seniority level Not Applicable Employment type
Employment type Full-time Job function
Job function Engineering and Information Technology Industries Truck Transportation Referrals increase your chances of interviewing at PortPro by 2x Sign in to set job alerts for “Director of Software Engineering” roles.
Continue with Google Continue with Google Continue with Google Continue with Google Director of Engineering, Operations Tools Suite
Director of Engineering, Fleet Management Delivery Platform
Los Angeles Metropolitan Area $210,000 - $250,000 17 hours ago Sr. Director , Security Software Engineering
Beverly Hills, CA $280,000 - $300,000 1 day ago Los Angeles Metropolitan Area 2 weeks ago Los Angeles, CA $119,000.00 - $195,000.00 2 weeks ago Senior Software Engineering Manager - REMOTE
Los Angeles, CA $160,000 - $180,000 11 hours ago Sr. Manager, IT Software Implementation - USA Remote
Sr. Manager, IT Software Implementation - USA Remote
Sr. Director , Security Software Engineering
Beverly Hills, CA $280,000 - $300,000 2 days ago We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr