2016-10-17 20 views
0

Hueを使用してOozieでsqoopジョブを実行する際に問題が発生しました。私はHortonworks HDPに基づいて4つのノードクラスタを持っています。OozieのSqoopジョブがHiveのインポートを実行できません

import 
--options-file ./dss_conn_parms.txt 
--table BD.BD_TABLE 
--target-dir /user/user_1/DMS 
--m 1 
--hive-import 
--hive-table BD.BD_TABLE_HIVE 

Oracleデータベースからのデータが正常にダウンロードされ、HDFSに挿入されました:

マイSqoopジョブは、以下のようになります。残念ながら、Hiveインポートは機能しません。エラーが許可に関連付けられている:Sqoopジョブはハイブimportコマンドを実行することはできませんなぜ

73167 [main] INFO org.apache.sqoop.hive.HiveImport - Loading uploaded data into Hive 
2016-10-17 09:42:55,203 INFO [main] hive.HiveImport (HiveImport.java:importTable(195)) - Loading uploaded data into Hive 
73180 [main] ERROR org.apache.sqoop.tool.ImportTool - Encountered IOException running import job: java.io.IOException: Cannot run program "hive": error=13, Permission denied 
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) 
at java.lang.Runtime.exec(Runtime.java:620) 
at java.lang.Runtime.exec(Runtime.java:528) 
at org.apache.sqoop.util.Executor.exec(Executor.java:76) 
at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:391) 
at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:344) 
at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:245) 
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:514) 
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605) 
at org.apache.sqoop.Sqoop.run(Sqoop.java:148) 
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) 
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184) 
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226) 
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235) 
at org.apache.sqoop.Sqoop.main(Sqoop.java:244) 
at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:197) 
at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:177) 
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47) 
at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:46) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:498) 
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:241) 
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) 
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) 
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) 
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) 
at java.security.AccessController.doPrivileged(Native Method) 
at javax.security.auth.Subject.doAs(Subject.java:422) 
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709) 
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) 
Caused by: java.io.IOException: error=13, Permission denied 
at java.lang.UNIXProcess.forkAndExec(Native Method) 
at java.lang.UNIXProcess.<init>(UNIXProcess.java:248) 
at java.lang.ProcessImpl.start(ProcessImpl.java:134) 
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) 
... 31 more 

2016-10-17 09:42:55,216 ERROR [main] tool.ImportTool (ImportTool.java:run(613)) - Encountered IOException running import job: java.io.IOException: Cannot run program "hive": error=13, Permission denied 
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) 
at java.lang.Runtime.exec(Runtime.java:620) 
at java.lang.Runtime.exec(Runtime.java:528) 
at org.apache.sqoop.util.Executor.exec(Executor.java:76) 
at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:391) 
at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:344) 
at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:245) 
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:514) 
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605) 
at org.apache.sqoop.Sqoop.run(Sqoop.java:148) 
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) 
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184) 
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226) 
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235) 
at org.apache.sqoop.Sqoop.main(Sqoop.java:244) 
at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:197) 
at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:177) 
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47) 
at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:46) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:498) 
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:241) 
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) 
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) 
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) 
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) 
at java.security.AccessController.doPrivileged(Native Method) 
at javax.security.auth.Subject.doAs(Subject.java:422) 
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709) 
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) 
Caused by: java.io.IOException: error=13, Permission denied 
at java.lang.UNIXProcess.forkAndExec(Native Method) 
at java.lang.UNIXProcess.<init>(UNIXProcess.java:248) 
at java.lang.ProcessImpl.start(ProcessImpl.java:134) 
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) 
... 31 more 

Intercepting System.exit(1) 

<<< Invocation of Main class completed <<< 

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1] 

あなたが任意のアイデアを持っていますか?

UPDATE

私は、コマンドラインでのハイブインポートオプションでSqoopジョブを実行し、私は問題が何であるかを知っています。コマンドラインでは、私がこの情報を確認できます。

Logging initialized using configuration in jar:file:/usr/hdp/2.4.2.0-258 /hive/lib/hive-common-1.2.1000.2.4.2.0-jar!/hive-log4j.properties 
OK 

を問題は、ローカルファイルシステム上に配置されているハイブ-共通1.2.1000.2.4.2.0ジャーへのアクセス権を持つです。私は何をすべきですか?

+0

ログにアクセスが拒否されました。あなたのアクセス権を確認してくださいハイブインポート –

+0

コマンドラインからこのsqoopコマンドを実行できるため、ハイブインポートを作成するためのアクセス権があります。私は問題がOozieかHueだと思う。 –

+0

あなたはcore-site.xmlを投稿できますか? –

答えて

0

1)

<property> 
     <name>hadoop.proxyuser.oozie.hosts</name> 
     <value>*</value> 
    </property> 
    <property> 
     <name>hadoop.proxyuser.oozie.groups</name> 
     <value>*</value> 
    </property> 
    <property> 
     <name>hadoop.proxyuser.YOUR_USERNAME.hosts</name> 
     <value>*</value> 
    </property> 
    <property> 
     <name>hadoop.proxyuser.YOUR_USERNAME.groups</name> 
     <value>*</value> 
    </property> 

を追加し、すべての再起動コア-site.xmlのでworkflow.xml

2)であなたのジョブ定義の<shell xmlns="uri:oozie:shell-action:0.2">セクション内

<env-var>HADOOP_USER_NAME=YOUR_USERNAME</env-var> 

を追加してみてくださいサービスを再試行してからもう一度お試しください

関連する問題