NewsBreak
Location: Mountain View, California, United States (Onsite 5 days in office is required)
About NewsBreak Founded in 2015, NewsBreak is the Content Intelligence platform shaping the future content economy. With over 40 million monthly active users, our flagship platform delivers highly personalized local news and information powered by advanced AI, recommendation systems, and adtech.
NewsBreak is Great Place to Work-certified and recognized by Fast Company as #32 on the Top Workplaces for Innovators. We reached unicorn status in 2021 and continue to grow to fulfill our mission: building the infrastructure layer for content intelligence.
For more information, visit www.newsbreak.com/about
About the Role As a Software Engineer-Web, you will play a pivotal role in building and optimizing our data acquisition infrastructure, ensuring the seamless collection and processing of web data to support critical business needs. You will tackle complex technical challenges in web crawling, enhance the performance of large-scale distributed systems, and contribute to the development of innovative solutions that drive our data-driven decision-making processes.
Responsibilities
Design, develop, and maintain distributed web crawler systems, ensuring efficient data scheduling, scraping, parsing, and storage.
Collect and process data from the internet and partner sources in compliance with website policies and legal regulations.
Optimize crawler performance to handle large-scale data extraction with high efficiency and reliability.
Solve complex technical challenges related to web crawling, including anti-crawling mechanisms, dynamic content rendering, and data quality assurance.
Collaborate with cross-functional teams, including data scientists, backend engineers, and product managers, to meet diverse business data requirements.
Stay updated with the latest web technologies and crawling frameworks to continuously improve system capabilities.
Requirements
Bachelor’s degree or higher in Computer Science, Engineering, or a related field, with at least 2 years of experience in web crawling and data collection.
Proficiency in mainstream web scraping technologies and frameworks/tools such as Scrapy, Selenium, and Puppeteer.
Strong coding skills in at least one programming language, such as Python, Java, Go, or C++.
Experience with distributed systems, data pipelines, and storage solutions is a plus.
Strong problem-solving skills, attention to detail, and the ability to work independently or as part of a team.
Familiarity with frontend development and dynamic content rendering techniques is preferred.
We offer a competitive benefits package:
Health, dental, and vision care for you and your family (100% coverage for employee)
Top-tier 401(K) plan with company matching
Paid time off and paid holidays
FSA, HSA and commuter benefits programs
Team activity budget
We are an equal employment opportunity employer. We do not discriminate on the basis of protected status.
#J-18808-Ljbffr
About NewsBreak Founded in 2015, NewsBreak is the Content Intelligence platform shaping the future content economy. With over 40 million monthly active users, our flagship platform delivers highly personalized local news and information powered by advanced AI, recommendation systems, and adtech.
NewsBreak is Great Place to Work-certified and recognized by Fast Company as #32 on the Top Workplaces for Innovators. We reached unicorn status in 2021 and continue to grow to fulfill our mission: building the infrastructure layer for content intelligence.
For more information, visit www.newsbreak.com/about
About the Role As a Software Engineer-Web, you will play a pivotal role in building and optimizing our data acquisition infrastructure, ensuring the seamless collection and processing of web data to support critical business needs. You will tackle complex technical challenges in web crawling, enhance the performance of large-scale distributed systems, and contribute to the development of innovative solutions that drive our data-driven decision-making processes.
Responsibilities
Design, develop, and maintain distributed web crawler systems, ensuring efficient data scheduling, scraping, parsing, and storage.
Collect and process data from the internet and partner sources in compliance with website policies and legal regulations.
Optimize crawler performance to handle large-scale data extraction with high efficiency and reliability.
Solve complex technical challenges related to web crawling, including anti-crawling mechanisms, dynamic content rendering, and data quality assurance.
Collaborate with cross-functional teams, including data scientists, backend engineers, and product managers, to meet diverse business data requirements.
Stay updated with the latest web technologies and crawling frameworks to continuously improve system capabilities.
Requirements
Bachelor’s degree or higher in Computer Science, Engineering, or a related field, with at least 2 years of experience in web crawling and data collection.
Proficiency in mainstream web scraping technologies and frameworks/tools such as Scrapy, Selenium, and Puppeteer.
Strong coding skills in at least one programming language, such as Python, Java, Go, or C++.
Experience with distributed systems, data pipelines, and storage solutions is a plus.
Strong problem-solving skills, attention to detail, and the ability to work independently or as part of a team.
Familiarity with frontend development and dynamic content rendering techniques is preferred.
We offer a competitive benefits package:
Health, dental, and vision care for you and your family (100% coverage for employee)
Top-tier 401(K) plan with company matching
Paid time off and paid holidays
FSA, HSA and commuter benefits programs
Team activity budget
We are an equal employment opportunity employer. We do not discriminate on the basis of protected status.
#J-18808-Ljbffr