2016-12-28 4 views
1

sqoop javaを使用してMySQLデータをHDFSにインポートする際にエラーが発生しました。しかし、私はhdfsにすべての権限を与え、同じ問題が発生しています。誰もこの問題を解決することができますか? Hadoopバージョン:2.6.0 Sqoop:1.4.6 ローカルファイルシステムに与えるべきアクセス権を正確に示唆しています。 ありがとうございます。ここでSqoop import java io例外:Mkdirがファイルの作成に失敗しました

16/12/28 13:14:54 WARN sqoop.ConnFactory: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration. 
16/12/28 13:14:54 INFO manager.SqlManager: Using default fetchSize of 1000 
16/12/28 13:14:54 INFO tool.CodeGenTool: Beginning code generation 
16/12/28 13:14:55 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM SHIPPER AS t WHERE 1=0 
16/12/28 13:14:56 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM SHIPPER AS t WHERE 1=0 
16/12/28 13:14:56 INFO orm.CompilationManager: HADOOP_HOME is /usr/local/hadoop 
Note: /tmp/sqoop-hadoop/compile/779c01454836e023b4d99d8ebb4e3add/SHIPPER.java uses or overrides a deprecated API. 
Note: Recompile with -Xlint:deprecation for details. 
16/12/28 13:14:57 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/779c01454836e023b4d99d8ebb4e3add/SHIPPER.jar 
16/12/28 13:14:57 INFO mapreduce.ImportJobBase: Beginning import of SHIPPER 
16/12/28 13:14:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
16/12/28 13:14:58 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 
16/12/28 13:14:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM SHIPPER AS t WHERE 1=0 
16/12/28 13:14:58 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 
16/12/28 13:14:58 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 
16/12/28 13:14:58 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 
16/12/28 13:14:58 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 
16/12/28 13:14:59 INFO mapreduce.JobSubmitter: number of splits:1 
16/12/28 13:14:59 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1242973550_0001 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099728/avro-mapred-1.7.5-hadoop2.jar <- /home/hadoop/Workspaces/BigData/avro-mapred-1.7.5-hadoop2.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/avro-mapred-1.7.5-hadoop2.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099728/avro-mapred-1.7.5-hadoop2.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099729/kite-data-hive-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-data-hive-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-data-hive-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099729/kite-data-hive-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099730/parquet-format-2.0.0.jar <- /home/hadoop/Workspaces/BigData/parquet-format-2.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-format-2.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099730/parquet-format-2.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099731/sqoop-1.4.6-hadoop200.jar <- /home/hadoop/Workspaces/BigData/sqoop-1.4.6-hadoop200.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/sqoop-1.4.6-hadoop200.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099731/sqoop-1.4.6-hadoop200.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099732/parquet-common-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-common-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-common-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099732/parquet-common-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099733/sqoop-1.4.1-incubating.jar <- /home/hadoop/Workspaces/BigData/sqoop-1.4.1-incubating.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/home/hadoop/.m2/repository/org/apache/sqoop/sqoop/1.4.1-incubating/sqoop-1.4.1-incubating.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099733/sqoop-1.4.1-incubating.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099734/mysql-connector-java-5.1.40-bin.jar <- /home/hadoop/Workspaces/BigData/mysql-connector-java-5.1.40-bin.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/mysql-connector-java-5.1.40-bin.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099734/mysql-connector-java-5.1.40-bin.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099735/ojdbc6-11.jar <- /home/hadoop/Workspaces/BigData/ojdbc6-11.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/ojdbc6-11.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099735/ojdbc6-11.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099736/commons-logging-1.1.1.jar <- /home/hadoop/Workspaces/BigData/commons-logging-1.1.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-logging-1.1.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099736/commons-logging-1.1.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099737/parquet-encoding-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-encoding-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-encoding-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099737/parquet-encoding-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099738/mysql-connector-java-5.1.40.jar <- /home/hadoop/Workspaces/BigData/mysql-connector-java-5.1.40.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/home/hadoop/.m2/repository/mysql/mysql-connector-java/5.1.40/mysql-connector-java-5.1.40.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099738/mysql-connector-java-5.1.40.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099739/opencsv-2.3.jar <- /home/hadoop/Workspaces/BigData/opencsv-2.3.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/opencsv-2.3.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099739/opencsv-2.3.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099740/snappy-java-1.0.5.jar <- /home/hadoop/Workspaces/BigData/snappy-java-1.0.5.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/snappy-java-1.0.5.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099740/snappy-java-1.0.5.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099741/commons-codec-1.4.jar <- /home/hadoop/Workspaces/BigData/commons-codec-1.4.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-codec-1.4.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099741/commons-codec-1.4.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099742/parquet-column-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-column-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-column-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099742/parquet-column-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099743/jackson-mapper-asl-1.9.13.jar <- /home/hadoop/Workspaces/BigData/jackson-mapper-asl-1.9.13.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-mapper-asl-1.9.13.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099743/jackson-mapper-asl-1.9.13.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099744/jackson-databind-2.3.1.jar <- /home/hadoop/Workspaces/BigData/jackson-databind-2.3.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-databind-2.3.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099744/jackson-databind-2.3.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099745/parquet-generator-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-generator-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-generator-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099745/parquet-generator-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099746/commons-io-2.4.jar <- /home/hadoop/Workspaces/BigData/commons-io-2.4.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-io-2.4.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099746/commons-io-2.4.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099747/commons-jexl-2.1.1.jar <- /home/hadoop/Workspaces/BigData/commons-jexl-2.1.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-jexl-2.1.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099747/commons-jexl-2.1.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099748/jackson-core-2.3.1.jar <- /home/hadoop/Workspaces/BigData/jackson-core-2.3.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-core-2.3.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099748/jackson-core-2.3.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099749/parquet-avro-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-avro-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-avro-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099749/parquet-avro-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099750/kite-data-mapreduce-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-data-mapreduce-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-data-mapreduce-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099750/kite-data-mapreduce-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099751/avro-1.7.5.jar <- /home/hadoop/Workspaces/BigData/avro-1.7.5.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/avro-1.7.5.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099751/avro-1.7.5.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099752/ant-contrib-1.0b3.jar <- /home/hadoop/Workspaces/BigData/ant-contrib-1.0b3.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/ant-contrib-1.0b3.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099752/ant-contrib-1.0b3.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099753/paranamer-2.3.jar <- /home/hadoop/Workspaces/BigData/paranamer-2.3.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/paranamer-2.3.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099753/paranamer-2.3.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099754/slf4j-api-1.6.1.jar <- /home/hadoop/Workspaces/BigData/slf4j-api-1.6.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/slf4j-api-1.6.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099754/slf4j-api-1.6.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099755/parquet-jackson-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-jackson-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-jackson-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099755/parquet-jackson-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099756/xz-1.0.jar <- /home/hadoop/Workspaces/BigData/xz-1.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/xz-1.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099756/xz-1.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099757/parquet-hadoop-1.4.1.jar <- /home/hadoop/Workspaces/BigData/parquet-hadoop-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/parquet-hadoop-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099757/parquet-hadoop-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099758/jackson-core-asl-1.9.13.jar <- /home/hadoop/Workspaces/BigData/jackson-core-asl-1.9.13.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-core-asl-1.9.13.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099758/jackson-core-asl-1.9.13.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099759/kite-hadoop-compatibility-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-hadoop-compatibility-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-hadoop-compatibility-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099759/kite-hadoop-compatibility-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099760/kite-data-core-1.0.0.jar <- /home/hadoop/Workspaces/BigData/kite-data-core-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/kite-data-core-1.0.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099760/kite-data-core-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099761/jackson-annotations-2.3.0.jar <- /home/hadoop/Workspaces/BigData/jackson-annotations-2.3.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/jackson-annotations-2.3.0.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099761/jackson-annotations-2.3.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099762/ant-eclipse-1.0-jvm1.2.jar <- /home/hadoop/Workspaces/BigData/ant-eclipse-1.0-jvm1.2.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099762/ant-eclipse-1.0-jvm1.2.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099763/commons-compress-1.4.1.jar <- /home/hadoop/Workspaces/BigData/commons-compress-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/commons-compress-1.4.1.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099763/commons-compress-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-hadoop/mapred/local/1482911099764/hsqldb-1.8.0.10.jar <- /home/hadoop/Workspaces/BigData/hsqldb-1.8.0.10.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/local/sqoop/lib/hsqldb-1.8.0.10.jar as file:/tmp/hadoop-hadoop/mapred/local/1482911099764/hsqldb-1.8.0.10.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099728/avro-mapred-1.7.5-hadoop2.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099729/kite-data-hive-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099730/parquet-format-2.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099731/sqoop-1.4.6-hadoop200.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099732/parquet-common-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099733/sqoop-1.4.1-incubating.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099734/mysql-connector-java-5.1.40-bin.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099735/ojdbc6-11.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099736/commons-logging-1.1.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099737/parquet-encoding-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099738/mysql-connector-java-5.1.40.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099739/opencsv-2.3.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099740/snappy-java-1.0.5.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099741/commons-codec-1.4.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099742/parquet-column-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099743/jackson-mapper-asl-1.9.13.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099744/jackson-databind-2.3.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099745/parquet-generator-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099746/commons-io-2.4.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099747/commons-jexl-2.1.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099748/jackson-core-2.3.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099749/parquet-avro-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099750/kite-data-mapreduce-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099751/avro-1.7.5.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099752/ant-contrib-1.0b3.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099753/paranamer-2.3.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099754/slf4j-api-1.6.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099755/parquet-jackson-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099756/xz-1.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099757/parquet-hadoop-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099758/jackson-core-asl-1.9.13.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099759/kite-hadoop-compatibility-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099760/kite-data-core-1.0.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099761/jackson-annotations-2.3.0.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099762/ant-eclipse-1.0-jvm1.2.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099763/commons-compress-1.4.1.jar 
16/12/28 13:15:00 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-hadoop/mapred/local/1482911099764/hsqldb-1.8.0.10.jar 
16/12/28 13:15:00 INFO mapreduce.Job: The url to track the job: http://localhost:8080/ 
16/12/28 13:15:00 INFO mapreduce.Job: Running job: job_local1242973550_0001 
16/12/28 13:15:00 INFO mapred.LocalJobRunner: OutputCommitter set in config null 
16/12/28 13:15:00 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter 
16/12/28 13:15:00 ERROR output.FileOutputCommitter: Mkdirs failed to create file:/shipper/_temporary/0 
16/12/28 13:15:00 INFO mapred.LocalJobRunner: Waiting for map tasks 
16/12/28 13:15:00 INFO mapred.LocalJobRunner: Starting task: attempt_local1242973550_0001_m_000000_0 
16/12/28 13:15:00 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 
16/12/28 13:15:00 INFO mapred.MapTask: Processing split: com.cloudera.sq[email protected]3d93720d 
16/12/28 13:15:00 INFO mapred.LocalJobRunner: map task executor complete. 
16/12/28 13:15:00 WARN mapred.LocalJobRunner: job_local1242973550_0001 
java.lang.Exception: java.io.IOException: Mkdirs failed to create file:/shipper/_temporary/0/_temporary/attempt_local1242973550_0001_m_000000_0 (exists=false, cwd=file:/home/hadoop/Workspaces/BigData) 
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) 
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) 
Caused by: java.io.IOException: Mkdirs failed to create file:/shipper/_temporary/0/_temporary/attempt_local1242973550_0001_m_000000_0 (exists=false, cwd=file:/home/hadoop/Workspaces/BigData) 
    at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:442) 
    at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:428) 
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:908) 
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:889) 
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:786) 
    at org.apache.sqoop.mapreduce.RawKeyTextOutputFormat.getRecordWriter(RawKeyTextOutputFormat.java:98) 
    at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:644) 
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) 
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243) 
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
    at java.lang.Thread.run(Thread.java:744) 
16/12/28 13:15:01 INFO mapreduce.Job: Job job_local1242973550_0001 running in uber mode : false 
16/12/28 13:15:01 INFO mapreduce.Job: map 0% reduce 0% 
16/12/28 13:15:01 INFO mapreduce.Job: Job job_local1242973550_0001 failed with state FAILED due to: NA 
16/12/28 13:15:01 INFO mapreduce.Job: Counters: 0 
16/12/28 13:15:01 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 
16/12/28 13:15:01 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 3.3552 seconds (0 bytes/sec) 
16/12/28 13:15:01 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 
16/12/28 13:15:01 INFO mapreduce.ImportJobBase: Retrieved 0 records. 
16/12/28 13:15:01 ERROR tool.ImportTool: Error during import: Import job failed! 

コードです:

package org.sqoop.com; 

import java.io.FileInputStream; 
import java.util.Properties; 

import org.apache.hadoop.conf.Configuration; 
import org.apache.hadoop.fs.Path; 

import com.cloudera.sqoop.SqoopOptions; 
import com.cloudera.sqoop.SqoopOptions.FileLayout; 
import com.cloudera.sqoop.tool.ImportTool; 

public class MysqlExportHDFS { 

    public static void importMySQLToHDFS() throws Exception { 
     try { 
      String driver = "com.mysql.jdbc.Driver"; 
      Class.forName(driver).newInstance(); 

      Configuration config = new Configuration(); 
      config.addResource(new Path("/usr/local/hadoop/etc/hadoop/conf/core-site.xml")); 
      config.addResource(new Path("/usr/local/hadoop/etc/hadoop/conf/hdfs-site.xml")); 

      Properties properties = new Properties(); 
      properties.load(new FileInputStream("/home/hadoop/Workspaces/BigData/resources/sqoopImport.properties")); 

      SqoopOptions sqoopOptions = new SqoopOptions(); 
      sqoopOptions.setDriverClassName(driver); 
      sqoopOptions.setHadoopHome("/usr/local/hadoop"); 
      sqoopOptions.setConnectString(properties.getProperty("db_connection_string")); 
      sqoopOptions.setTableName(properties.getProperty("db_table_name")); 
      sqoopOptions.setUsername(properties.getProperty("db_username")); 
      sqoopOptions.setPassword(properties.getProperty("db_password")); 
      sqoopOptions.setNumMappers(1); 
      sqoopOptions.setTargetDir(properties.getProperty("db_output_directory")); 
      sqoopOptions.setFileLayout(FileLayout.TextFile); 
      new ImportTool().run(sqoopOptions); 
      } catch (Exception e) { 
      // TODO: handle exception 
      } 
     } 

    public static void main(String[] args) throws Exception{ 

     MysqlExportHDFS mySqlImport = new MysqlExportHDFS(); 
     mySqlImport.importMySQLToHDFS(); 
    } 
} 

答えて

0

このファイルの場所は、あなたの仕事のjarファイルを解凍するために、ローカルディスク上に作成されています。それはHDFS上ではありません。ローカルディスクディレクトリにmkdirへのアクセス権があるかどうか確認してください。 このジョブを実行するために使用しているユーザのローカルファイルシステムに対するアクセス権を、読み取り、書き込み、実行、削除に設定できます。詳細な解答は下のリンクをご覧ください。

Hadoop java.io.IOException: Mkdirs failed to create /some/path

MKDirs-failed-to-create-file

+0

問題が解決...非常にありがとう... – Yasodhara

関連する問題