Logstash csvparsefailureとdateparsefailure

Question

私はこのフィルタを使用して、私がPHPファイルから生成しているいくつかのcsvデータを解析しています。私は.hmlファイルを出力するmsi afterburnerと呼ばれるGPU監視ツールから出力を取り出しています。私のPHPファイルが取り除いてコンマ区切りの値を出力する、1トンの空白と無関係のヘッダがあります。

filter 
    { 
     csv 
     { 
      columns => ["somename","@timestamp","cpu.avg.temp","gpu.temp","fan.speed","gpu.usage","bus.usage","fan.tachometer","clock.core","framerate.hz","framerate.ms","cpu.temp.1","cpu.temp.2","cpu.temp.3","cpu.temp.4"] 
      separator => "," 
      skip_empty_columns => "true" 
     } 
     mutate 
     { 
      convert => ["somename","integer"] 
      convert => ["cpu.avg.temp","float"] 
      convert => ["gpu.temp","float"] 
      convert => ["fan.speed","float"] 
      convert => ["gpu.usage","float"] 
      convert => ["bus.usage","float"] 
      convert => ["fan.tachometer","float"] 
      convert => ["clock.core", "float"] 
      convert => ["framerate.hz","float"] 
      convert => ["framerate.ms","float"] 
      convert => ["cpu.temp.1","float"] 
      convert => ["cpu.temp.2","float"] 
      convert => ["cpu.temp.3","float"] 
      convert => ["cpu.temp.4","float"] 
     } 
     date 
     { 
      match => ["@timestamp", "dd-MM-yyyyHH:mm:ss"] 
     } 
    } 

これは出力されているlogstashが私に投げかけています。私はこれが私の日付形式が悪いという事実によるか、私のメッセージの終わりに特殊文字 '\ r'があるように見えるかどうか疑問に思っています。私はlogstashがdd-MM-yyyyHH：mm：ss形式を読むことができるかどうか疑問に思っています。

{ 
      "path" => "C:\\Users\\Public\\Documents\\gpumetrics.csv", 
     "somename" => 80, 
    "@timestamp" => 2017-02-20T02:33:10.764Z, 
     "@version" => "1", 
      "host" => "DESKTOP-Q8UEATO", 
     "message" => "80,19-02-201721:33:10,32.000,41.000,0.000,0.000,0.000,0.000,215.000,0.000,0.000,31.000,32.000,30.000,31.000\r", 
      "type" => "csv", 
      "tags" => [ 
     [0] "_csvparsefailure", 
     [1] "_dateparsefailure" 
    ] 
} 

ここに、私のログファイルのサンプルラインがいくつかあります。気づいているように、タイムスタンプの前にフィールドがあります。私はこれが許されているのだろうかと思っています。

80,19-02-201713:20:32,44.000,43.000,0.000,0.000,0.000,0.000,215.000,,,37.000,42.000,41.000,38.000 
80,19-02-201713:20:33,47.000,43.000,0.000,0.000,0.000,0.000,215.000,,,46.000,47.000,45.000,44.000 
80,19-02-201713:20:34,53.000,43.000,0.000,0.000,0.000,0.000,215.000,,,35.000,50.000,36.000,37.000 
80,19-02-201713:20:35,37.000,43.000,0.000,0.000,0.000,0.000,215.000,,,37.000,37.000,37.000,34.000 
80,19-02-201713:20:36,34.000,44.000,0.000,0.000,0.000,0.000,1582.000,0.000,0.000,39.000,34.000,33.000,36.000 
80,19-02-201713:20:37,46.000,44.000,0.000,0.000,0.000,0.000,1582.000,0.000,0.000,45.000,37.000,43.000,37.000 

Answer 1

あなたのラインが解析される前@timestampが内部的に作成されるので、あなたの問題は、あなたのタイムスタンプ変数の名前を変更することにより、非常に簡単に解決することができます。

filter 
    { 
     csv 
     { 
           remove the @ 
            | 
            v 
      columns => ["somename","timestamp","cpu.avg.temp","gpu.temp","fan.speed","gpu.usage","bus.usage","fan.tachometer","clock.core","framerate.hz","framerate.ms","cpu.temp.1","cpu.temp.2","cpu.temp.3","cpu.temp.4"] 
      separator => "," 
      skip_empty_columns => "true" 
     } 
     ... 
     date 
     { 
      match => ["timestamp", "dd-MM-yyyyHH:mm:ss"] 
        ^
         | 
        remove the @ 
     } 
    }