Rを使用してaspxウェブサイトから掻き集める

Question

私はRを使用してウェブサイト上のデータをスクラップするタスクを達成しようとしています。

1. 私は、次のページで、各リンクを通過したいと思います： http://capitol.hawaii.gov/advreports/advreport.aspx?year=2013&report=deadline&rpt_type=&measuretype=hb&title=House手形

2. 「知事に伝え、」現状の上映との項目のみを選択します。例えば、http://capitol.hawaii.gov/measure_indiv.aspx?billtype=HB&billnumber=17&year=2013

3. STATUS TEXT内のセルを「Passed Final Reading」の句で廃棄します。例：SD2で修正された合格者の最終レディング（ヨルダン、ツジの予約制）代表者Cabanilla、Morikawa、Oshiro、Tokioka（4）、none excused（0）。

私は以前の例をRcurlとXML（R）パッケージで使用しようとしましたが、aspxサイトで正しく使用する方法がわかりません。だから私が持っていたいと思うものは次のようなものです：1.そのようなコードを作る方法に関するいくつかの提案。 2.そのような作業を行うために必要な知識をどのように学ぶべきかについての勧告。任意の助け

おかげで、

トム

Answer 1

require(httr) 
require(XML) 

basePage <- "http://capitol.hawaii.gov" 

h <- handle(basePage) 

GET(handle = h) 

res <- GET(handle = h, path = "/advreports/advreport.aspx?year=2013&report=deadline&rpt_type=&measuretype=hb&title=House") 

# parse content for "Transmitted to Governor" text 
resXML <- htmlParse(content(res, as = "text")) 
resTable <- getNodeSet(resXML, '//*/table[@id ="GridViewReports"]/tr/td[3]') 
appRows <-sapply(resTable, xmlValue) 
include <- grepl("Transmitted to Governor", appRows) 
resUrls <- xpathSApply(resXML, '//*/table[@id ="GridViewReports"]/tr/td[2]//@href') 

appUrls <- resUrls[include] 

# look at just the first 

res <- GET(handle = h, path = appUrls[1]) 

resXML <- htmlParse(content(res, as = "text")) 


xpathSApply(resXML, '//*[text()[contains(.,"Passed Final Reading")]]', xmlValue) 

[1] "Passed Final Reading as amended in SD 2 with Representative(s) Fale, Jordan, 
Tsuji voting aye with reservations; Representative(s) Cabanilla, Morikawa, Oshiro, 
Tokioka voting no (4) and none excused (0)." 

レッツパッケージはhandleを設定することにより、すべてのバックグラウンド作業を処理httr。

あなたはすべての92個のリンク上で実行する場合：

# get all the links returned as a list (will take sometime) 
# print statement included for sanity 
res <- lapply(appUrls, function(x){print(sprintf("Got url no. %d",which(appUrls%in%x))); 
            GET(handle = h, path = x)}) 
resXML <- lapply(res, function(x){htmlParse(content(x, as = "text"))}) 
appString <- sapply(resXML, function(x){ 
        xpathSApply(x, '//*[text()[contains(.,"Passed Final Reading")]]', xmlValue) 
         }) 


head(appString) 

> head(appString) 
$href 
[1] "Passed Final Reading as amended in SD 2 with Representative(s) Fale, Jordan, Tsuji voting aye with reservations; Representative(s) Cabanilla, Morikawa, Oshiro, Tokioka voting no (4) and none excused (0)." 

$href 
[1] "Passed Final Reading, as amended (CD 1). 25 Aye(s); Aye(s) with reservations: none . 0 No(es): none. 0 Excused: none."             
[2] "Passed Final Reading as amended in CD 1 with Representative(s) Cullen, Har voting aye with reservations; Representative(s) McDermott voting no (1) and none excused (0)." 

$href 
[1] "Passed Final Reading, as amended (CD 1). 25 Aye(s); Aye(s) with reservations: none . 0 No(es): none. 0 Excused: none."         
[2] "Passed Final Reading as amended in CD 1 with none voting aye with reservations; Representative(s) Hashem, McDermott voting no (2) and none excused (0)." 

$href 
[1] "Passed Final Reading, as amended (CD 1). 24 Aye(s); Aye(s) with reservations: none . 0 No(es): none. 1 Excused: Ige."      
[2] "Passed Final Reading as amended in CD 1 with none voting aye with reservations; none voting no (0) and Representative(s) Say excused (1)." 

$href 
[1] "Passed Final Reading, as amended (CD 1). 25 Aye(s); Aye(s) with reservations: none . 0 No(es): none. 0 Excused: none."       
[2] "Passed Final Reading as amended in CD 1 with Representative(s) Johanson voting aye with reservations; none voting no (0) and none excused (0)." 

$href 
[1] "Passed Final Reading, as amended (CD 1). 25 Aye(s); Aye(s) with reservations: none . 0 No(es): none. 0 Excused: none." 
[2] "Passed Final Reading as amended in CD 1 with none voting aye with reservations; none voting no (0) and none excused (0)."