私のリストの一部だけがcsvファイルに書き込むのはなぜですか？

Question

私は最初のスクリプトをビルドしようとしているpythonには新しいです。私は、URLのリストをスクラップし、csvファイルでエクスポートしたい。

私のスクリプトはうまく実行されますが、csvファイルを開くときにわずか数行のデータしか書き込まれません。 （sharelistとsharelist1）を書き込もうとしているリストを印刷しているとき、印刷は完了ですが、csvファイルは完了していません。私はここで共有しなければならない私のコードのどの部分

for url in urllist[10:1000]: 
       # query the website and return the html to the variable 'page' 
     try: 
      page = urllib2.urlopen(url) 
     except urllib2.HTTPError as e: 
       if e.getcode() == 404: # eheck the return code 
        continue 
     soup = BeautifulSoup(page, 'html.parser') 

        # Take out the <div> of name and get its value 
     name_box = soup.find(attrs={'class': 'nb-shares'}) 
     if name_box is None: 
      continue 
     share = name_box.text.strip() # strip() is used to remove starting and trailing 

     # save the data in tuple 
     sharelist.append(url) 
     sharelist1.append(share) 

    # open a file for writing. 
     csv_out = open('mycsv.csv', 'wb') 

    # create the csv writer object. 
     mywriter = csv.writer(csv_out) 

    # writerow - one row of data at a time. 
     for row in zip(sharelist, sharelist1): 
      mywriter.writerow(row) 

    # always make sure that you close the file. 
    # otherwise you might find that it is empty. 
     csv_out.close() 

わからない：ここで

は、私のコードの一部です。十分でない場合は教えてください！

Answer 1

問題は、ループを実行するたびにファイルを開いていることです。これは基本的に前のファイルを上書きします。

# open a file for writing. 
    csv_out = open('mycsv.csv', 'wb') 

# create the csv writer object. 
    mywriter = csv.writer(csv_out) 

# writerow - one row of data at a time. 
    for row in zip(sharelist, sharelist1): 
     mywriter.writerow(row) 

# always make sure that you close the file. 
# otherwise you might find that it is empty. 
    csv_out.close() 

ループの前にファイルを開くか、appendオプションを使用してファイルを開きます。

これはオプション1（インデントに注意してください）です。

# open a file for writing. 
csv_out = open('mycsv.csv', 'wb') 

# create the csv writer object. 
mywriter = csv.writer(csv_out) 
for url in urllist[10:1000]: 
    try: 
     page = urllib2.urlopen(url) 
    except urllib2.HTTPError as e: 
      if e.getcode() == 404: # eheck the return code 
       continue 
    soup = BeautifulSoup(page, 'html.parser') 

    name_box = soup.find(attrs={'class': 'nb-shares'}) 
    if name_box is None: 
     continue 
    share = name_box.text.strip() 

    # save the data in tuple 
    sharelist.append(url) 
    sharelist1.append(share) 

# writerow - one row of data at a time. 
    for row in zip(sharelist, sharelist1): 
     mywriter.writerow(row) 

# always make sure that you close the file. 
# otherwise you might find that it is empty. 
csv_out.close() 

これはオプション2：

for url in urllist[10:1000]: 
      # query the website and return the html to the variable 'page' 
    try: 
     page = urllib2.urlopen(url) 
    except urllib2.HTTPError as e: 
      if e.getcode() == 404: # eheck the return code 
       continue 
    soup = BeautifulSoup(page, 'html.parser') 

       # Take out the <div> of name and get its value 
    name_box = soup.find(attrs={'class': 'nb-shares'}) 
    if name_box is None: 
     continue 
    share = name_box.text.strip() # strip() is used to remove starting and trailing 

    # save the data in tuple 
    sharelist.append(url) 
    sharelist1.append(share) 

# open a file for writing. 
    csv_out = open('mycsv.csv', 'ab') 

# create the csv writer object. 
    mywriter = csv.writer(csv_out) 

# writerow - one row of data at a time. 
    for row in zip(sharelist, sharelist1): 
     mywriter.writerow(row) 

# always make sure that you close the file. 
# otherwise you might find that it is empty. 
    csv_out.close() 

Answer 2

問題が発見されたとファイルの最適なソリューションがどのwithキーワードを使用することですファイルを最後に自動的に閉じることができます：

with open('mycsv.csv', 'wb') as csv_out: 
    mywriter = csv.writer(csv_out) 
    for url in urllist[10:1000]: 

     try: 
      page = urllib2.urlopen(url) 
     except urllib2.HTTPError as e: 
       if e.getcode() == 404: 
        continue 
     soup = BeautifulSoup(page, 'html.parser') 

     name_box = soup.find(attrs={'class': 'nb-shares'}) 
     if name_box is None: 
      continue 
     share = name_box.text.strip() 

     # save the data in tuple 
     sharelist.append(url) 
     sharelist1.append(share) 

     for row in zip(sharelist, sharelist1): 
      mywriter.writerow(row) 

Answer 3

書き込み用ファイルを開く、usiコンテキストマネージャを使用すると、明示的にファイルを閉じる必要はありません。

with open('mycsv.csv', 'w') as file_obj: 
    mywriter = csv.writer(file_obj) 
    for url in urllist[10:1000]: 
     try: 
      page = urllib2.urlopen(url) 
     except urllib2.HTTPError as e: 
       if e.getcode() == 404: # check the return code 
        continue 
     soup = BeautifulSoup(page, 'html.parser') 

     name_box = soup.find(attrs={'class': 'nb-shares'}) 
     if name_box is None: 
      continue 
     share = name_box.text.strip() 
     # no need to use zip, and append in 2 lists as they're really expensive calls, 
     # and by the looks of it, I think it'll create duplicate rows in your file 
     mywriter.writerow((url, share))