Web Scraping Specialist

About the Company

Wynd Labs is an early-stage startup on a mission to make public web data accessible for AI through contributions to Grass. Grass is a network sharing application that allows users to share their unused bandwidth, creating a residential proxy network that rewards individual residential IPs.

Responsibilities

• Write, test, and refine code that extracts data from various online sources
• Perform data retrieval tasks, handling pagination and dynamic content loaded with AJAX
• Clean and format extracted data to meet quality standards
• Store and manage scraped data in appropriate databases, optimizing for access speed and integrity
• Monitor scraping processes and resolve issues to maintain continuous data flow

Requirements

• Demonstrated ability to extract data from complex websites with minimal supervision
• Proficiency in Python or JavaScript
• Strong skills in BeautifulSoup, Scrapy, or Selenium
• Knowledge of asynchronous programming, multithreading, and distributed scraping
• In-depth knowledge of HTML, CSS, JavaScript, and the DOM
• Experience with NoSQL databases such as MongoDB or Cassandra
• Experience with cloud services like AWS, Google Cloud, or Azure

Preferred Qualifications

• Ability to apply machine learning algorithms for data cleaning or categorization
• Active participation in open-source projects related to web scraping or data processing