セレンでウェブスクレイピングを使用しているときにエラーが発生しました

Question

Google Playホームページからレビューを募集しています。

私はよく出て行き、途中で止まります。 私は次のエラーを取得する：

selenium.common.exceptions.ElementNotVisibleException: Message: {"errorMessage":"Element is not currently visible and may not be manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:51812","User-Agent":"Python http auth"},"httpVersion":"1.1","method":"POST","post":"{\"id\": \":wdc:1505887578605\", \"sessionId\": \"cca93cc0-9dc9-11e7-a685-bd84ddef3ed2\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/cca93cc0-9dc9-11e7-a685-bd84ddef3ed2/element/:wdc:1505887578605/click"}} 
Screenshot: available via screen 

は、私が検索し、多くの修正をしましたが、私はそれを修正することができませんでした。 私は数日間停止しています。コードをアップロードします。

このエラーはなぜ発生しますか？ どうすれば修正できますか？

#python 3.6 

from selenium import webdriver 
from time import sleep 
from bs4 import BeautifulSoup, Comment 
import pandas as pd 


#Setting up Chrome webdriver Options 
#chrome_options = webdriver.ChromeOptions() 

#setting up local path of chrome binary file 
#chrome_options.binary_location = 
"/Users/Norefly/chromedriver2/chromedriver.exec" 

#creating Chrome webdriver instance with the set chrome_options 
driver = webdriver.PhantomJS("C:/Python/phantomjs-2.1.1- 
windows/bin/phantomjs.exe") 
link = "https://play.google.com/store/apps/details? 
id=com.supercell.clashofclans&hl=en" 
driver.get(link) 
#driver.execute_script("window.scrollTo(0, document.body.scrollHeight)") 
Ptitle = driver.find_element_by_class_name('id-app-title').text.replace(' ','') 
print(Ptitle) 
#driver.find_element_by_xpath('//*[@id="body-content"]/div/div/div[1]/div[2]/div[2]/div[1]/div[4]/button[2]/div[2]').click() 

sleep(1) 
driver.find_element_by_xpath('//*[@id="body-content"]/div/div/div[1]/div[2]/div[2]/div[1]/div[4]/button[2]/div[2]/div/div'). 
click() 
#select_newest.select_by_visible_text('Newest') 
#driver.find_element_by_xpath('//*[@id="body-content"]/div/div/div[1]/div[2]/div[2]/div[1]/div[4]/button[2]/div[2]/div/div'). 
click() 
sleep(2) 
#driver.find_element_by_css_selector('.review-filter.id-review-sort-filter.dropdown-menu-container').click() 
driver.find_element_by_css_selector('.displayed-child').click() 
#driver.find_element_by_xpath("//button[@data-dropdown-value='1']").click() 
driver.execute_script("document.querySelectorAll('button.dropdown-child')[0].click()") 
reviews_df = [] 
for i in range(1,10000): 
    try: 
     for elem in driver.find_elements_by_class_name('single-review'): 
      print(str(i)) 
      content = elem.get_attribute('outerHTML') 
      soup = BeautifulSoup(content, "html.parser") 
      #print(soup.prettify()) 
      date = soup.find('span',class_='review-date').get_text() 
      rating = soup.find('div',class_='tiny-star')['aria-label'][6:7] 
      title = soup.find('span',class_='review-title').get_text() 
      txt = soup.find('div',class_='review-body').get_text().replace('Full Review','')[len(title)+1:] 
      print(soup.get_text()) 
      temp = pd.DataFrame({'Date':date,'Rating':rating,'Review Title':title,'Review Text':txt},index=[0]) 
      print('-'*10) 
      reviews_df.append(temp) 
      #print(elem) 

    except: 
     print('s') 
    driver.find_element_by_xpath('//*[@id="body-content"]/div/div/div[1]/div[2]/div[2]/div[1]/div[4]/button[2]/div[2]/div/div'). 
click() 
reviews_df = pd.concat(reviews_df,ignore_index=True) 

reviews_df.to_csv(Ptitle+'review_google.csv', encoding='utf-8') 

#driver.close() 

私にはわからないので、エラーのフレーズとパスを上げます。

Traceback (most recent call last): 
    File "C:/Users/lobyp/Downloads/reviewex.py", line 51, in <module> 
    driver.find_element_by_xpath('//*[@id="body-content"]/div/div/div[1]/div[2]/div[2]/div[1]/div[4]/button[2]/div[2]/div/div'). 
click() 
    File "C:\Users\lobyp\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webelement.py", line 78, in click 
self._execute(Command.CLICK_ELEMENT) 
    File "C:\Users\lobyp\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webelement.py", line 499, in _execute 
return self._parent.execute(command, params) 
    File "C:\Users\lobyp\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 297, in execute 
self.error_handler.check_response(response) 
    File "C:\Users\lobyp\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 194, in check_response 
raise exception_class(message, screen, stacktrace) 
selenium.common.exceptions.ElementNotVisibleException: Message: {"errorMessage":"Element is not currently visible and may not be manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:53283","User-Agent":"Python http auth"},"httpVersion":"1.1","method":"POST","post":"{\"id\": \":wdc:1505888277040\", \"sessionId\": \"6b69b0a0-9dcb-11e7-bc02-87f5b92766da\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/6b69b0a0-9dcb-11e7-bc02-87f5b92766da/element/:wdc:1505888277040/click"}} 
Screenshot: available via screen 

Answer 1

私は専門家ではないんだけど、私は、要素が隠されたHTML属性を持っている場合は、この例外が表示されることを信じています。サイトのドロップダウンメニューで同じ問題が発生しましたが、要素を表示させるためにSeleniumでスクロールイベントを使用する必要がありました。