grokを使用したActive Directoryログの解析が遅い

Question

私はgrokを初めて使用しています。 私はMicrosoft active directory logsをgrokパーサを使って解析しようとしています。 私はjava grokライブラリを使用しています。

ログは

<13> 10.200.3.7 10.20.211.15 07/04/2017 15:34:00 PM SERVER01 07/04/2017 15:34:00 PM LogName=Security SourceName=Microsoft Windows security auditing. EventCode=4624 EventType=0 Type=Information ComputerName=SERVER01.network.local TaskCategory=Logon OpCode=Info RecordNumber=1809490942 Keywords=Audit Success Message=An account was successfully logged on.  Subject:  Security ID:  S-1-0-0  Account Name:  User-330 Account Domain:  -  Logon ID:  0x0  Logon Type:   3  New Logon:  Security ID:  S-1-5-18 Account Name:  SERVER01$  Account Domain:  DOMAIN  Logon ID:  0x12393ab39  Logon GUID:  \{C893D0A2-6498-BBE3-560D-0A1088FA4D9E\}  Process Information:  Process ID:  0x0  Process Name:  -  Network Information:  Workstation Name:  Source Network Address: 1.68.4.213  Source Port:  57261  Detailed Authentication Information:  Logon Process:  Kerberos Authentication Package: Kerberos Transited Services: -  Package Name (NTLM only): -  Key Length:  0  This event is generated when a logon session is created. It is generated on the computer that was accessed.  The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.  The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).  The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.  The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.  The authentication information fields provide detailed information about this specific logon request.  - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.  - Transited services indicate which intermediate services have participated in this logon request.  - Package name\ 
<13> 10.200.3.7 10.20.211.15 07/04/2017 15:34:00 PM SERVER01 07/04/2017 15:34:00 PM LogName=Security SourceName=Microsoft Windows security auditing. EventCode=4624 EventType=0 Type=Information ComputerName=SERVER01.network.local TaskCategory=Logon OpCode=Info RecordNumber=1809490942 Keywords=Audit Success Message=An account was successfully logged on.  Subject:  Security ID:  S-1-0-0  Account Name:  User-331 Account Domain:  -  Logon ID:  0x0  Logon Type:   3  New Logon:  Security ID:  S-1-5-18 Account Name:  SERVER01$  Account Domain:  DOMAIN  Logon ID:  0x12393ab39  Logon GUID:  \{C893D0A2-6498-BBE3-560D-0A1088FA4D9E\}  Process Information:  Process ID:  0x0  Process Name:  -  Network Information:  Workstation Name:  Source Network Address: 1.68.4.214  Source Port:  57261  Detailed Authentication Information:  Logon Process:  Kerberos Authentication Package: Kerberos Transited Services: -  Package Name (NTLM only): -  Key Length:  0  This event is generated when a logon session is created. It is generated on the computer that was accessed.  The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.  The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).  The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.  The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.  The authentication information fields provide detailed information about this specific logon request.  - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.  - Transited services indicate which intermediate services have participated in this logon request.  - Package name\ 

のように見える私のGROKパターンは、問題は私のカスタムJavaのパーサと比較して、それは非常に遅いことである

\<%{USER:hField1}\> %{IPV4:hIp1} %{IPV4:hIp2} %{DATESTAMP_12HOUR:hTime1;date;dd/MM/yyyy hh:mm:ss a} %{USER:hField2} %{DATESTAMP_12HOUR:hTime2;date;dd/MM/yyyy hh:mm:ss a} LogName=%{USER:logname}%{SPACE}SourceName=%{GREEDYDATA:sourceName}%{SPACE}EventCode=%{GREEDYDATA:eventCode}%{SPACE}EventType=%{GREEDYDATA:eventType}%{SPACE}Type=%{GREEDYDATA:typeField}%{SPACE} ComputerName=%{GREEDYDATA:computerName}%{SPACE}TaskCategory=%{GREEDYDATA:taskCategory}%{SPACE}OpCode=%{GREEDYDATA:opCode}%{SPACE}RecordNumber=%{GREEDYDATA:recordNumber}%{SPACE}Keywords=%{GREEDYDATA:keywords}%{SPACE}Message=%{NON_DOT_DELIMITER:message}.%{SPACE}%{GREEDYDATA:jsonData} 

です。 私のカスタムjavaパーサーは、grokパターンを使った解析が同じデータに対して60秒かかるうちに、50Kレコードを解析するのに2.5秒かかる。

パーサーに問題はありますか？

Answer 1

正規表現の場合と同様に、正規表現のインタプリタが推測する必要がない場合は、スピードアップが得られます。だから、あなたのgrokパターンの周りにアンカー^（行の先頭）と$（行末）を使用すると、良い利益を得るかもしれません。