2017-09-12 18 views
0

Hadoop 2.8.1に付属のmapreduceジョブの例を実行していますHadoop Mapreduceジョブ:入力パスが存在しません:hdfs:// localhost:9000/user/******/grep-temp-1208171489

bin/hdfs dfs -copyFromLocal etc/hadoop/core-site.xml . 

bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar grep ./core-site.xml output ‘configuration’ 

しかし、私はこれを実行すると、ジョブ終了(メインエラーがタイトルに表示されます):

***********s-mbp-2:hadoop-2.8.1 ***********$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.1.jar grep ./core-site.xml output ‘configuration’ 
17/09/12 14:30:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
17/09/12 14:30:29 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 
17/09/12 14:30:29 INFO input.FileInputFormat: Total input files to process : 1 
17/09/12 14:30:30 INFO mapreduce.JobSubmitter: number of splits:1 
17/09/12 14:30:30 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1505251091124_0001 
17/09/12 14:30:30 INFO impl.YarnClientImpl: Submitted application application_1505251091124_0001 
17/09/12 14:30:30 INFO mapreduce.Job: The url to track the job: http://***********s-mbp-2.lan:8088/proxy/application_1505251091124_0001/ 
17/09/12 14:30:30 INFO mapreduce.Job: Running job: job_1505251091124_0001 
17/09/12 14:30:33 INFO mapreduce.Job: Job job_1505251091124_0001 running in uber mode : false 
17/09/12 14:30:33 INFO mapreduce.Job: map 0% reduce 0% 
17/09/12 14:30:33 INFO mapreduce.Job: Job job_1505251091124_0001 failed with state FAILED due to: Application application_1505251091124_0001 failed 2 times due to AM Container for appattempt_1505251091124_0001_000002 exited with exitCode: 127 
Failing this attempt.Diagnostics: Exception from container-launch. 
Container id: container_1505251091124_0001_02_000001 
Exit code: 127 
Stack trace: ExitCodeException exitCode=127: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:972) 
    at org.apache.hadoop.util.Shell.run(Shell.java:869) 
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1170) 
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:236) 
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:305) 
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84) 
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
    at java.lang.Thread.run(Thread.java:748) 


Container exited with a non-zero exit code 127 
For more detailed output, check the application tracking page: http://***********s-mbp-2.lan:8088/cluster/app/application_1505251091124_0001 Then click on links to logs of each attempt. 
. Failing the application. 
17/09/12 14:30:33 INFO mapreduce.Job: Counters: 0 
17/09/12 14:30:33 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 
17/09/12 14:30:33 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/***********/.staging/job_1505251091124_0002 
org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://localhost:9000/user/***********/grep-temp-576807334 
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:329) 
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:271) 
    at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:59) 
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:393) 
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:303) 
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:320) 
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:198) 
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1341) 
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1338) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:422) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807) 
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1338) 
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1359) 
    at org.apache.hadoop.examples.Grep.run(Grep.java:94) 
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) 
    at org.apache.hadoop.examples.Grep.main(Grep.java:103) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:498) 
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71) 
    at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144) 
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:498) 
    at org.apache.hadoop.util.RunJar.run(RunJar.java:234) 
    at org.apache.hadoop.util.RunJar.main(RunJar.java:148) 
間違って起こっている

とどのように私は、これらのコマンドを使用しています

私が直します?ありがとう。

答えて

0

プログラムが例外のパスにアクセスできないようです。コードを実行する前に、hdfsシステムでmkdirを使用してそのディレクトリを手動で作成してみてください。それが助けてくれることを願って。

関連する問題