原因を解決するには：java.lang.ClassNotFoundException：org.apache.hive.hcatalog.data.JsonSerDe？？

Question

selectクエリの結果を他のテーブルに挿入しようとしています。

説明： この問題を解決するためのさまざまな回避策を試しましたが、いずれも問題なく機能しています。

1. 私はシェルクラスパスをハイブにSERDESを追加するadd jarコマンドを使用しています。

2. /usr/local/hive/libから必要なすべてのjarファイルをhadoopのhdfsファイルシステム上にあるハイブクラスパスに移動しました。

3. 私も.bashrcファイルに環境変数を追加しました。これらの試みのすべての後

は、私はまだ同じエラーを取得しています。どんな助けもありがとう。

問合せ：

insert overwrite table outbound_log_event partition(batch) select c_ip, mdatetime, mdate, mtime, file_name, raw_line, fprint, mapping, time_taken, cs_username, cs_auth_group, x_exception_id, sc_filter_result, cs_categories, cs_referer, sc_status, s_action, cs_method, rs_content_type, cs_uri_scheme, cs_uri_port, cs_host, cs_uri_path, cs_uri_query, cs_uri_extension, cs_user_agent, s_ip, sc_bytes, cs_bytes, x_virus_id, cs_ip, x_bluecoat_application_name, x_bluecoat_application_operation, file_name as log_type, mdate as batch from outbound_log_event_staging; 

例外：

Diagnostic Messages for this Task: 
Error: java.lang.RuntimeException: Error in configuring object 
     at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112) 
     at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78) 
     at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) 
     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:449) 
     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) 
     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:422) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) 
     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
Caused by: java.lang.reflect.InvocationTargetException 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:498) 
     at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
     ... 9 more 
Caused by: java.lang.RuntimeException: Error in configuring object 
     at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112) 
     at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78) 
     at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) 
     at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) 
     ... 14 more 
Caused by: java.lang.reflect.InvocationTargetException 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:498) 
     at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
     ... 17 more 
Caused by: java.lang.RuntimeException: Map operator initialization failed 
     at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:140) 
     ... 22 more 
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hive.hcatalog.data.JsonSerDe 
     at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:446) 
     at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:362) 
     at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:481) 
     at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:438) 
     at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) 
     at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:481) 
     at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:438) 
     at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) 
     at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:442) 
     at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:119) 
     ... 22 more 
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hive.hcatalog.data.JsonSerDe 
     at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:74) 
     at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:355) 
     ... 31 more 
Caused by: java.lang.ClassNotFoundException: org.apache.hive.hcatalog.data.JsonSerDe 
     at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
     at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) 
     at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 
     at java.lang.Class.forName0(Native Method) 
     at java.lang.Class.forName(Class.java:348) 
     at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:71) 
     ... 32 more 

Answer 1

私は問題上記の決意を持っているハイブ-site.xmlでhive.aux.jars.path設定することもできます。

<property> 
    <name>hive.aux.jars.path</name> 
    <value>file:///usr/local/Cellar/hive–1.2.1/lib/hive-hcatalog-core-1.2.1.jar</value> 
    </property>