2016-05-25 10 views
0

Googleカスタム検索APIを使用して米国のニュース・アウトレットを検索しようとしています。 Googleが提供するコード例を使用すると、多数の他の辞書やリストを含むPython辞書が完成します。メタ機能の「res」の横に表示されているタグは、各記事でアクセスしようとしている値です。JSONから変換されたPythonリストの解析

import os.path 
import csv 
from lxml import html 
from googleapiclient.discovery import build 


def newslist(): 
    ''' 
    Uses google custom search to search 20 US news sources for gun control articles, 
    and converts info into python dictionary. 

    in - none 

    out - res: JSON formatted search results 

    ''' 

    service = build("customsearch", "v1", 
      developerKey="key") 
    res = service.cse().list(
     q='query', 
     cx='searchid', 
    ).execute() 
    return res 


def meta(res, doc_count): 
    ''' 
    Finds necessary meta-data of all articles. Avoids collections, such as those found on Huffington Post and New York Times. 

    in - res: defined above 

    out - meta_csv: csv file with article meta-data 

    ''' 

    row1 = ['doc_id', 'url', 'title', 'publisher', 'date'] 
    if res['context']['items']['pagemap']['metatags']['applicationname'] is not 'collection': 
    for art in res['context']['items']: 
     url = res['context']['items']['link'] 
     title = res['context']['items']['pagemap']['article']['newsarticle']['headline'] 
     publisher = res['context']['items']['displayLink'].split('www.' and '.com') 
     date = res['context']['items']['pagemap']['newsarticle']['datepublished'] 
     row2 = [doc_count, url, title, publisher, date] 
     with open('meta.csv', 'w', encoding = 'utf-8') as meta: 
     csv_file = csv.writer(meta, delimiter = ',', quotechar = '|', 
      quoting = csv.QUOTE_MINIMAL) 
     if doc_count == 1: 
      csv_file.writerow(row1) 
     csv_file.writerow(row2) 
    doc_count += 1 

はここだと検索クエリからの印刷出力の例:私は基本的にループのために書くことができることを理解

{'context': {'title': 'Gun Control articles'}, 
'items': [{'displayLink': 'www.washingtonpost.com', 
    'formattedUrl': 'https://www.washingtonpost.com/.../white-resentment-is-fueling-opposition- ' 
        'to-gun-control-researchers-say/', 
    'htmlFormattedUrl': 'https://www.washingtonpost.com/.../white-resentment-is-fueling-opposition- ' 
         'to-<b>gun</b>-<b>control</b>-researchers-say/', 
    'htmlSnippet': 'Apr 4, 2016 <b>...</b> Racial prejudice could play ' 
        'a significant role in white Americans&#39; ' 
        'opposition to <br>\n' 
        '<b>gun control</b>, according to new research from ' 
        'political scientists at&nbsp;...', 
    'htmlTitle': 'White resentment is fueling opposition to <b>gun ' 
       'control</b>, researchers say', 
    'kind': 'customsearch#result', 
    'link': 'https://www.washingtonpost.com/news/wonk/wp/2016/04/04/white-resentment-is-fueling-opposition-to-gun-control-researchers-say/', 
    'pagemap': {'cse_image': [{'src': 'https://img.washingtonpost.com/rf/image_1484w/2010-2019/WashingtonPost/2015/10/03/Others/Images/2015-10-03/Botsford_gunshow1004_15_10_03_41831443897980.jpg'}], 
       'cse_thumbnail': [{'height': '183', 
            'src': 'https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcSXtMnfm_GHkX3d2dOWgmto3rFjmhzxV8InoPao1tBuiBrEWsDMz4WDKcPB', 
            'width': '275'}], 
       'metatags': [{'apple-itunes-app': 'app-id=938922398, ' 
                'app-argument=https://www.washingtonpost.com/news/wonk/wp/2016/04/04/white-resentment-is-fueling-opposition-to-gun-control-researchers-say/', 
           'article:author': 'https://www.facebook.com/chrisingraham', 
           'article:publisher': 'https://www.facebook.com/washingtonpost', 
           'author': 'https://www.facebook.com/chrisingraham', 
           'fb:admins': '492', 
           'fb:app_id': '41245586762', 
           'news_keywords': 'guns, gun control, ' 
               'racial resentment, ' 
               'white people', 
           'og:description': 'Some white gun owners ' 
               '"understand ' 
               "'freedom' in a very " 
               'particular way."', 
           'og:image': 'https://img.washingtonpost.com/rf/image_1484w/2010-2019/WashingtonPost/2015/10/03/Others/Images/2015-10-03/Botsford_gunshow1004_15_10_03_41831443897980.jpg', 
           'og:site_name': 'Washington Post', 
           'og:title': 'White resentment is fueling ' 
              'opposition to gun control, ' 
              'researchers say', 
           'og:type': 'article', 
           'og:url': 'https://www.washingtonpost.com/news/wonk/wp/2016/04/04/white-resentment-is-fueling-opposition-to-gun-control-researchers-say/', 
           'referrer': 'unsafe-url', 
           'twitter:card': 'summary_large_image', 
           'twitter:creator': '@_cingraham', 
           'viewport': 'width=device-width, ' 
              'initial-scale=1.0, ' 
              'user-scalable=yes, ' 
              'minimum-scale=0.5, ' 
              'maximum-scale=2.0'}], 
       'newsarticle': [{'articlebody': 'People look at ' 
               'handguns during the ' 
               "Nation's Gun Show in " 
               'Chantilly, Va. in ' 
               'October 2015. (Photo ' 
               'by Jabin Botsford/The ' 
               'Washington Post) ' 
               'Racial prejudice ' 
               'could play a ' 
               'significant role in ' 
               'white...', 
           'datepublished': '2016-04-04T11:46-500', 
           'description': 'Some white gun owners ' 
               '"understand ' 
               "'freedom' in a very " 
               'particular way."', 
           'headline': 'White resentment is ' 
              'fueling opposition to ' 
              'gun control, researchers ' 
              'say', 
           'image': 'https://img.washingtonpost.com/rf/image_1484w/2010-2019/WashingtonPost/2015/10/03/Others/Images/2015-10-03/Botsford_gunshow1004_15_10_03_41831443897980.jpg', 
           'mainentityofpage': 'True', 
           'url': 'https://www.washingtonpost.com/news/wonk/wp/2016/04/04/white-resentment-is-fueling-opposition-to-gun-control-researchers-say/'}], 
       'person': [{'name': 'Christopher Ingraham'}]}, 
    'snippet': 'Apr 4, 2016 ... Racial prejudice could play a ' 
       "significant role in white Americans' opposition to \n" 
       'gun control, according to new research from political ' 
       'scientists at\xa0...', 
    'title': 'White resentment is fueling opposition to gun control, ' 
      'researchers say'}, 

が、より簡単に、より少ないコードがある場合、私は思ったんだけどURL、タイトル、パブリッシャー、日付の各値について、このデータにアクセスする集中的な方法です。

答えて

0

jsonモジュールを使用しないのはなぜですか?

import json 
s = ... # Your JSON text 
result = json.loads(s) 

resultあなたのJSONに応じて、dictまたはlistになります。

+1

私はすでにこれを試していましたが、json.loadsで辞書を解析できないというエラーが表示されました。私は、Googleのpython apiが、元のjson出力を上記のようにリストと辞書の辞書に変換することを理解しました。 pprintのおかげでjsonという形式になっていますが、上記の出力は、pythonのリストと辞書で構成されています。 – JJ47

+0

json.load(s)は辞書を解析できます。 json.loads()のエラーで質問を更新してください。 – AMS

関連する問題