#Octoparse list detail page update#

When a task built using 'Lists of URLs' is set to run in the Cloud, the task will be split up into sub-tasks which are then set to run on various cloud servers simultaneously. As a result, the speed of extraction will be faster, especially for Cloud Extraction. Octoparse will load the URLs one by one and scrape the data from each page.īy creating a 'List of URLs' loop mode, Octoparse has no need to deal with extra steps like 'Click to paginate' or 'Click Item' to enter the item page. To scrape by using a list of URLs, we'll simply set up a loop of all the URLs we need to scrape from then add a data extraction action right after it to get the data we need. And another example, if you are scraping news articles from any particular website, most likely the article page will share the same page structure.

For example, when you scrape listings from Yelp, you may need to paginate through the search results. Questions: When should you consider scraping by using a list of URLs?Īnswer: When the desired data spans through multiple pages sharing the same page structure. In this tutorial, we will introduce an easy and powerful way to extract data from multiple web pages by using a list of URLs. Depending on how the webpage is structured, there are usually multiple approaches you can try. Sometimes there isn’t just one way to scrape a webpage. Upgrade and check the updated version for this tutorial now!

#Octoparse list detail page update#

Psst! You are reading a tutorial for Octoparse version 7.3, which is slowly on its way out. We strongly recommend that you update Octoparse to the latest version 8.4 because the new version is more automated with a brand new auto-detect algorithm.