以下の入力が与えられるとJSON

Question

から値を割り当てることができないLogstash：

111.111.111.111、11.11.11.11 - 11.11.11.11 [06 /月/ 2016：08：26：10 +0000]「POST /幾分サービス/ GetSomething HTTP/1.1" 499 0 " - " "ジャカルタコモンズ-のHttpClient/3.1" "7979798797979799" 59.370 - "{\ X0A \ x22correlationId \ X22：\ x22TestCorr1 \ X22 \ X0A}" Logstash構成で

。

：

input { stdin {} } 
output { stdout { codec => "rubydebug" } } 

filter { 
    grok { 
    match => { "message" => "%{COMBINEDAPACHELOG} %{QS:partner_id} %{NUMBER:req_time} %{GREEDYDATA:extra_fields}" } 
    add_field => [ "received_at", "%{@timestamp}" ] 
    add_field => [ "received_from", "%{host}" ] 
    } 


    mutate { 
    gsub => ["extra_fields", "\\x0A","","extra_fields", "\\x22",'\"'] 
    } 

    json { 
    source => "extra_fields" 
    target => "extra_fields_json" 
    } 


    mutate { 
     add_field => { 
     "correlationId" => "%{[extra_fields_json][correlationId]}" 
     } 
    } 
} 

は私たちに次のような出力を与えました

 
Logstash startup completed 
{ 
        "message" => "111.111.111.111, 11.11.11.11 - 11.11.11.11 [06/May/2016:08:26:10 +0000] "POST /some-service/GetSomething HTTP/1.1" 499 0 "-" "Jakarta Commons-HttpClient/3.1" "8888888" 59.370 - "{\x0A\x22correlationId\x22 : \x22TestCorr1\x22\x0A}", 
        "@version" => "1", 
       "@timestamp" => "2016-06-23T18:00:28.831Z",    
        "clientip" => "11.11.11.11", 
        "ident" => "-", 
         "auth" => "10.0.12.205", 
       "timestamp" => "06/May/2016:08:26:10 +0000", 
         "verb" => "POST", 
        "request" => "/some-service/GetSomething", 
       "httpversion" => "1.1", 
        "response" => "499", 
        "bytes" => "0", 
        "referrer" => "\"-\"", 
        "agent" => "\"Jakarta Commons-HttpClient/3.1\"", 
       "partner_id" => "\"8888888\"", 
        "req_time" => [ 
         [0] "59.370", 
         [1] "1" 
        ], 
       "extra_fields" => "\"{\\\"correlationId\\\" : \\\"TestCorr_1\\\"}\"", 
      "extra_fields_json" => "{\"correlationId\" : \"TestCorr_1\"}", 
       "correlationId" => "correlationId" 

} 
Logstash shutdown completed 

なぜ "correlationId"の値が "TestCorr_1 \"ではなく "correlationId"であるのですか？

あなたのお手伝いをお待ちしております。ありがとう！

Answer 1

jsonが正しく解析されるためには、"と\をextra_fieldsから削除する必要があります。 あなたはにあなたのmutateフィルタを変更する必要があります：それが動作するかどう

は

mutate { 
     gsub => ["extra_fields", "\"","", 
      "extra_fields", "\\x0A","", 
      "extra_fields", "\\x22",'\"', 
      "extra_fields", "(\\)","" 
     ] 
} 

は、それが動作する...