ScrapeIt

The client ordered a dataset on flight tickets from the booking website Opodo.com. Websites for Scraping: opodo.com An additional requirement was the application of multiple custom filters, so that the web server itself would handle the filtering and return only the flights with the unique parameters the client needed for their analytics pipeline. This project implemented what was likely the most complex automation flow of user interactions we’ve built so far, including slider control—something rarely seen in automation projects. Search and Filtering Rules: Search for all flight tickets between specified departure and arrival airports for each of the next N days. Filter results to include only flights where both the outbound and return trips are operated by the same airline. Select the best-priced option for each airline on each departure date. To optimize the number of requests, the scraper manipulated user filters on the webpage using an automated browser. Setup & Launch Time: 4 working days Data Volume: The daily dataset contains approximately 13,000 rows.