Webから解析した後にリストをDataFrameに変換する

Question

私はpythonを初心者です。下のCMEウェブサイトのテーブルを再作成したいのですが、私が作成したリストをデータフレームに変換することはできません。どのような助けが非常に感謝！前もって感謝します！

url = "http://www.cmegroup.com/trading/energy/crude-oil/light-sweet-crude_product_calendar_futures.html" 
user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)' 
headers = { 'User-Agent' : user_agent } 

req = urllib2.Request(url, headers=headers) 

response = urllib2.urlopen(req) 

soup = BeautifulSoup(response) 

header = soup.findAll('th',limit = 8) 
column_header = [] 
for j in header: 
    column_header.append(j.getText()) 




data_rows = soup.findAll('tr')[2:] 
dates = [] 
for i in range(len(data_rows)): 
    for td in data_rows[i].findAll('td'): 
     dates.append(td.getText()) 

Answer 1

from bs4 import BeautifulSoup 
import requests 

r = requests.get("http://www.cmegroup.com/trading/energy/crude-oil/light-sweet-crude_product_calendar_futures.html") 

soup = BeautifulSoup(r.content, "lxml") 

headers = [th.text for th in soup.thead.find_all('th')] # use thead to narrow the scope 
print(headers) 
for tr in soup.tbody.find_all('tr'): 
    row = [i.get_text(strip=True) for i in tr(['th', 'td'])] 
    print(row) 

アウト：

['Contract Month', 'Product Code', 'First TradeLast Trade', 'Settlement', 'First HoldingLast Holding', 'First PositionLast Position', 'First NoticeLast Notice', 'First DeliveryLast Delivery'] 
['Feb 2017', 'CLG17', '21 Nov 201120 Jan 2017', '20 Jan 2017', '--', '23 Jan 201723 Jan 2017', '24 Jan 201724 Jan 2017', '01 Feb 201728 Feb 2017'] 
['Mar 2017', 'CLH17', '21 Nov 201121 Feb 2017', '21 Feb 2017', '--', '22 Feb 201722 Feb 2017', '23 Feb 201723 Feb 2017', '01 Mar 201731 Mar 2017'] 
['Apr 2017', 'CLJ17', '21 Nov 201121 Mar 2017', '21 Mar 2017', '--', '22 Mar 201722 Mar 2017', '23 Mar 201723 Mar 2017', '01 Apr 201730 Apr 2017'] 
['May 2017', 'CLK17', '21 Nov 201120 Apr 2017', '20 Apr 2017', '--', '21 Apr 201721 Apr 2017', '24 Apr 201724 Apr 2017', '01 May 201731 May 2017'] 

1. 使用.tbodyまたは.theadlimitを使用しない、範囲を絞り込むために。
2. 使用リストの内包はあまり使用にappend