beautifulsoupの抽出文にキーワードが含まれている場合

Question

htmlのWebサイト（例：this：http://www.uni-bremen.de/mscmarbiol/）を処理し、「research」という文字列を含む各文を保存したいと考えています。

これは、私がウェブサイトからすべてのテキストを取得したコードの単なる例です。言葉の研究」を含む文のみをエクスポートするタスクを実行するための最良の方法だろう何

from bs4 import BeautifulSoup 
from zipfile import ZipFile 
import os 
html_page = "example.html" #i saved this page as example locally 

data = [] 
with open(html_page, "r") as html: 
    soup = BeautifulSoup(html, "lxml") 
    text_group = soup.get_text() 

print text_group 

？

文字列に.splitとseperatorsを使用するよりもエレガントな方法がありますか？ "re"で何かできますか？

このトピックは非常に新しいので、大変ありがとうございます。あなたはスープを持っていたら

敬具、

Trgovec

Answer 1

"文章"がドキュメント内で厳密に定義されているわけではないので、平文を文章に分割するツールを使用する必要があるようです。

NLTKパッケージはこの種のものに最適です。あなたは、（それはあなたの文書で「修士」は、インスタンスのための独立した文ではないことを理解していない）それは完璧ではない

import nltk 
sentences = nltk.sent_tokenize(text) 
result = [sentence for sentence in sentences if "research" in sentence] 

ような何かをしたいと思うでしょうが、文のセグメンテーションは一見複雑ですこれはあなたが得るほど良いものです。

Answer 2

In [65]: soup.find_all(name=['p', 'li'], text=re.compile(r'research')) 
Out[65]: 
[<p class="bodytext">The M.Sc. programme Marine Biology is strongly research-orientated. The graduates are trained to develop hypotheses-driven research concepts and to design appropriate experimental approaches in order to answer profound questions related to the large field of marine ecosystem and organism functioning and of potential impacts of local, regional and global environmental change. 
</p>, 
<p class="bodytext">Many courses are actually taught in the laboratories and facilities of the institutes benefiting from cutting-edge research infrastructure and first-hand contact to leading experts. This unique context sets the scene for direct links from current state of research to academic training.</p>, 
<li>Training in state-of-the-art methodologies by leading research teams.</li>, 
<li>Advanced courses in different university departments and associated research institutions.</li>, 
<li>Field trips, excursions or even the opportunity to participate in research expeditions. </li>, 
<p class="bodytext">The University of Bremen and the associated research institutions offer a variety of opportunities to continue an academic career as Ph.D. candidate. 
</p>, 
<p class="bodytext">Employment opportunities for Marine Biologists exist worldwide at institutions committed to research and development, in the fishing and aquaculture industry as well as in the environmental conservation and management sector at governmental agencies or within NGOs and IGOs. Marine biologists also work at museums, zoological gardens, and aquaria. Additional employment opportunities for marine biologists include adjacent fields such as media (i.e. scientific journalism), eco-consulting, environmental impact assessments, and eco-tourism business. Marine biologists are also employed in the commercial and industrial sector, for instance for "Blue Biotechnology", coastal zone management and the sustainable use of marine resources.</p>] 

Answer 3

、あなたが試すことがあります。

for tag in soup.descendants: 
    if tag.string and 'research' in tag.string: 
     print(tag.string) 

あなたは lxmlがインストールされていることから、XPathを使用して

高速化の代替：

from lxml import etree 
with open(html_page, "r") as html: 
    tree = etree.parse(html, parser=etree.HTMLParser()) 
[e.text for e in tree.xpath("//*[contains(text(), 'research')]")]