PDIがリモートファイルと圧縮ファイルの読み込みに失敗する

Question

私の問題は、リモートで圧縮されたファイルをS3から読み込むときです。ファイルが破損していない場合があり、例外が発生することもあります。

私はこの古いバグで同様の問題がまだこれが起こっているエラーですhttp://jira.pentaho.com/browse/PDI-1800

を開きました。私はこの変換をカートサーバー上で実行しています。このエラーは再現しにくいですが、残念ながら再現する方法はありません。

org.pentaho.di.core.exception.KettleFileException: 
Exception reading line: java.io.EOFException: Unexpected end of ZLIB input stream 
Unexpected end of ZLIB input stream 
Unexpected end of ZLIB input stream 
at org.pentaho.di.trans.steps.fileinput.text.TextFileInputUtils.getLine(TextFileInputUtils.java:326) 
at org.pentaho.di.trans.steps.fileinput.text.TextFileInputReader.tryToReadLine(TextFileInputReader.java:420) 
at org.pentaho.di.trans.steps.fileinput.text.TextFileInputReader.readRow(TextFileInputReader.java:167) 
at org.pentaho.di.trans.steps.fileinput.BaseFileInputStep.processRow(BaseFileInputStep.java:205) 
at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62) 
at java.lang.Thread.run(Thread.java:745) 
Caused by: java.io.EOFException: Unexpected end of ZLIB input stream 
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240) 
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158) 
at java.util.zip.ZipInputStream.read(ZipInputStream.java:194) 
at org.pentaho.di.core.compress.CompressionInputStream.read(CompressionInputStream.java:68) 
at java.io.BufferedInputStream.read1(BufferedInputStream.java:284) 
at java.io.BufferedInputStream.read(BufferedInputStream.java:345) 
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) 
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) 
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) 
at sun.nio.cs.StreamDecoder.read0(StreamDecoder.java:127) 
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:112) 
at java.io.InputStreamReader.read(InputStreamReader.java:168) 
at org.pentaho.di.trans.steps.fileinput.text.TextFileInputUtils.getLine(TextFileInputUtils.java:294) 
… 5 more 
2017/12/13 12:07:40 – S3CsvInput.0 – ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Unexpected error 
2017/12/13 12:07:40 – S3CsvInput.0 – ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : org.pentaho.di.core.exception.KettleFileException: 
2017/12/13 12:07:40 – S3CsvInput.0 – 
2017/12/13 12:07:40 – S3CsvInput.0 – Exception reading line: java.io.EOFException: Unexpected end of ZLIB input stream 
2017/12/13 12:07:40 – S3CsvInput.0 – Unexpected end of ZLIB input stream 
2017/12/13 12:07:40 – S3CsvInput.0 – 
2017/12/13 12:07:40 – S3CsvInput.0 – Unexpected end of ZLIB input stream 
2017/12/13 12:07:40 – S3CsvInput.0 – 
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.steps.fileinput.text.TextFileInputUtils.getLine(TextFileInputUtils.java:326) 
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.steps.fileinput.text.TextFileInputReader.tryToReadLine(TextFileInputReader.java:420) 
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.steps.fileinput.text.TextFileInputReader.readRow(TextFileInputReader.java:167) 
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.steps.fileinput.BaseFileInputStep.processRow(BaseFileInputStep.java:205) 
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62) 
2017/12/13 12:07:40 – S3CsvInput.0 – at java.lang.Thread.run(Thread.java:745) 
2017/12/13 12:07:40 – S3CsvInput.0 – Caused by: java.io.EOFException: Unexpected end of ZLIB input stream 
2017/12/13 12:07:40 – S3CsvInput.0 – at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240) 
2017/12/13 12:07:40 – S3CsvInput.0 – at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158) 
2017/12/13 12:07:40 – S3CsvInput.0 – at java.util.zip.ZipInputStream.read(ZipInputStream.java:194) 
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.core.compress.CompressionInputStream.read(CompressionInputStream.java:68) 
2017/12/13 12:07:40 – S3CsvInput.0 – at java.io.BufferedInputStream.read1(BufferedInputStream.java:284) 
2017/12/13 12:07:40 – S3CsvInput.0 – at java.io.BufferedInputStream.read(BufferedInputStream.java:345) 
2017/12/13 12:07:40 – S3CsvInput.0 – at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) 
2017/12/13 12:07:40 – S3CsvInput.0 – at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) 
2017/12/13 12:07:40 – S3CsvInput.0 – at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) 
2017/12/13 12:07:40 – S3CsvInput.0 – at sun.nio.cs.StreamDecoder.read0(StreamDecoder.java:127) 
2017/12/13 12:07:40 – S3CsvInput.0 – at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:112) 
2017/12/13 12:07:40 – S3CsvInput.0 – at java.io.InputStreamReader.read(InputStreamReader.java:168) 
2017/12/13 12:07:40 – S3CsvInput.0 – at org.pentaho.di.trans.steps.fileinput.text.TextFileInputUtils.getLine(TextFileInputUtils.java:294) 
2017/12/13 12:07:40 – S3CsvInput.0 – … 5 more 
child index = 56, logging object : org.pentaho.di.core.logging.LoggingObject@46345a23 parent=1ff32099-5cbe-47b3-b32c-34f1291f6c09 
2017/12/13 12:07:40 – md5_field12308.0 – Finished processing (I=0, O=0, R=391170, W=782340, U=0, E=0) 
2017/12/13 12:07:40 – Filter Rows field12308.0 – Finished processing (I=0, O=0, R=3, W=3, U=0, E=0) 
2017/12/13 12:07:40 – Filter Rows field12315.0 – Finished processing (I=0, O=0, R=13, W=13, U=0, E=0) 
2017/12/13 12:07:40 – TransLoad_cube5403_data137170 – ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Errors detected! 

Answer 1

ネットワーク上の問題である可能性があります。ファイルが完全にダウンロードされていないため、正常に終了しません。

まずローカルファイルからダウンロードし、問題が解決するかどうかを確認してください。