Web Scraping Specialist

About the Company

Wynd Labs is an early-stage startup on a mission to make public web data accessible for AI through contributions to Grass. Grass is a network sharing application that allows users to share their unused bandwidth, creating a residential proxy network that rewards individual residential IPs.

Responsibilities

• Write, test, and refine code that extracts data from various online sources
• Perform data retrieval tasks handling pagination and dynamic AJAX content
• Clean and format extracted data to meet quality standards
• Store and manage scraped data in databases, optimizing for speed and integrity
• Monitor scraping processes and resolve issues to maintain continuous data flow

Requirements

• Proficiency in Python or JavaScript
• Experience with BeautifulSoup, Scrapy, or Selenium
• Knowledge of asynchronous programming, multithreading, and distributed scraping
• In-depth knowledge of HTML, CSS, JavaScript, and the DOM
• Experience with NoSQL databases like MongoDB or Cassandra
• Experience with cloud services such as AWS, Google Cloud, or Azure
• Demonstrated ability to extract data from complex websites with minimal supervision

Preferred Qualifications

• Ability to apply machine learning algorithms for data cleaning or categorization
• Active participation in open-source projects related to web scraping or data processing