ROCKPRO64 for SCRAPING
#3
Thanks very much EvilBunny  Heart

Can you try again with this Python, I modify just with com.au

---------------PYTHON CODE------------
from selenium import webdriver
import time
from urllib.request import urlopen
from bs4 import BeautifulSoup
 
#---CHROME OPTION AND OPEN DRIVER---
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.managed_default_content_settings.images": 2}
chrome_options.add_experimental_option("prefs", prefs)
#driver = webdriver.Chrome('/usr/lib/chromium-browser/chromedriver',chrome_options=chrome_options)
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.set_window_size(2000, 1000)
 
#---LIST of WEBPAGE---
list=['https://www.amazon.com.au/dp/B07HPKWGJH','https://www.amazon.com.au/dp/B07FTN21JL','https://www.amazon.com.au/dp/B07HPCDHQS','https://www.amazon.com.au/dp/B07FTN21JL','https://www.amazon.com.au/dp/B079DQ7JK6','https://www.amazon.com.au/dp/B07CBPS16T','https://www.amazon.com.au/dp/B07CBP38HS','https://www.amazon.com.au/dp/B073R3MJ87','https://www.amazon.com.au/dp/B01LZG4KPC','https://www.amazon.com.au/dp/B077DX1NFH']
 
#---START TIME---
start = time.time()
 
#---Show WEBPAGE---
h='url'
for h in list:
          driver.get(h)
          driver.execute_script('window.scrollBy(0,10000);')
          print('-----------> '+str(h))
          time.sleep(1)
          html = driver.page_source
          soup = BeautifulSoup(html, 'html.parser')
          print('-----> BS4 OK')
          
#---END TIME---
end = time.time()
 
#---ELAPSED---
elapsed = end - start
 
print (elapsed)


-----
Thanks you again Smile


Messages In This Thread
ROCKPRO64 for SCRAPING - by Ulthor_31 - 07-30-2019, 06:47 AM
RE: ROCKPRO64 for SCRAPING - by evilbunny - 07-31-2019, 09:44 PM
RE: ROCKPRO64 for SCRAPING - by Ulthor_31 - 08-01-2019, 03:51 AM
RE: ROCKPRO64 for SCRAPING - by evilbunny - 08-01-2019, 04:01 AM
RE: ROCKPRO64 for SCRAPING - by Ulthor_31 - 08-01-2019, 05:19 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
Brick Logical Extensible Gizmo Organizing RockPro64 Enclosure hoarfrosty 0 506 08-06-2023, 09:32 PM
Last Post: hoarfrosty
  RockPro64 programing GPIO by Rust yanagawa3 0 1,523 11-24-2021, 08:43 PM
Last Post: yanagawa3
Photo RockPro64 programing GPIO, I2C,UART and SPI madhuks 3 6,552 08-06-2020, 09:07 AM
Last Post: Gienek
  RockPro64 as PATA/SATA bridge? Count Omega 1 4,316 12-07-2019, 02:25 PM
Last Post: Count Omega
Question HOW TO fix Wifi on RockPro64 - Recalbox OS and upload roms on SSD External Gouki 6 8,884 08-25-2019, 05:13 AM
Last Post: Gouki
  RockPro64 as PVR/DVR bm_00 1 3,901 02-27-2019, 04:46 AM
Last Post: mabs

Forum Jump:


Users browsing this thread: 1 Guest(s)