BeautifulSoupタグは、タイプbs4.element.NavigableStringとbs4.element.Tagです

Question

私はウィキペディアの記事でテーブルを掻き集めようとしていますが、各テーブル要素のタイプは<class 'bs4.element.Tag'>と<class 'bs4.element.NavigableString'>のように見えます。

import requests 
import bs4 
import lxml 


resp = requests.get('https://en.wikipedia.org/wiki/List_of_municipalities_in_Massachusetts') 

soup = bs4.BeautifulSoup(resp.text, 'lxml') 

munis = soup.find(id='mw-content-text')('table')[1] 

for muni in munis: 
    print type(muni) 
    print '============' 

次ouputをを生成します。私はmuni.contentsを取得しようとすると

<class 'bs4.element.Tag'> 
============ 
<class 'bs4.element.NavigableString'> 
============ 
<class 'bs4.element.Tag'> 
============ 
<class 'bs4.element.NavigableString'> 
============ 
<class 'bs4.element.Tag'> 
============ 
<class 'bs4.element.NavigableString'> 
... 

私はAttributeError: 'NavigableString' object has no attribute 'contents'エラーを取得します。

私は間違っていますか？それぞれmuniのbs4.element.Tagオブジェクトを取得するにはどうすればよいですか？

（Python 2.7を使用）。

Answer 1

ノード間にマークアップのスペースがある場合、BeautifulSoupはそれらをNavigableStringに変換します。ただ、トライキャッチを入れて、あなたがそれらを望むよう内容がフェッチなっているかどうかを確認 -

アウト

for muni in munis: 
    #print type(muni) 
    try: 
     print muni.contents 
    except AttributeError: 
     pass 
    print '============' 

Answer 2

from bs4 import BeautifulSoup 
import requests 

r = requests.get('https://en.wikipedia.org/wiki/List_of_municipalities_in_Massachusetts') 
soup = BeautifulSoup(r.text, 'lxml') 
rows = soup.find(class_="wikitable sortable").find_all('tr')[1:] 

for row in rows: 
    cell = [i.text for i in row.find_all('td')] 
    print(cell) 

：

['Abington', 'Town', 'Plymouth', 'Open town meeting', '15,985', '1712'] 
['Acton', 'Town', 'Middlesex', 'Open town meeting', '21,924', '1735'] 
['Acushnet', 'Town', 'Bristol', 'Open town meeting', '10,303', '1860'] 
['Adams', 'Town', 'Berkshire', 'Representative town meeting', '8,485', '1778'] 
['Agawam', 'City[4]', 'Hampden', 'Mayor-council', '28,438', '1855'] 
['Alford', 'Town', 'Berkshire', 'Open town meeting', '494', '1773'] 
['Amesbury', 'City', 'Essex', 'Mayor-council', '16,283', '1668'] 
['Amherst', 'Town', 'Hampshire', 'Representative town meeting', '37,819', '1775'] 
['Andover', 'Town', 'Essex', 'Open town meeting', '33,201', '1646'] 
['Aquinnah', 'Town', 'Dukes', 'Open town meeting', '311', '1870'] 
['Arlington', 'Town', 'Middlesex', 'Representative town meeting', '42,844', '1807'] 
['Ashburnham', 'Town', 'Worcester', 'Open town meeting', '6,081', '1765'] 
['Ashby', 'Town', 'Middlesex', 'Open town meeting', '3,074', '1767'] 
['Ashfield', 'Town', 'Franklin', 'Open town meeting', '1,737', '1765'] 
['Ashland', 'Town', 'Middlesex', 'Open town meeting', '16,593', '1846'] 

Answer 3

#!/usr/bin/env python 
# coding:utf-8 
'''黄哥Python''' 

import requests 
import bs4 
from bs4 import BeautifulSoup 
# from urllib.request import urlopen 

html = requests.get('https://en.wikipedia.org/wiki/List_of_S%26P_500_companies') 
soup = BeautifulSoup(html.text, 'lxml') 

symbolslist = soup.find('table').tr.next_siblings 
for sec in symbolslist: 
    # print(type(sec)) 
    if type(sec) is not bs4.element.NavigableString: 
     print(sec.get_text())