Pythonを使用したhive udfのランタイムエラー

Question

私は従業員の経験を計算するためにPythonでカスタムハイブudfを作成しようとしていますが、ランタイムエラーが発生しています。 これは、

select transform(eno,ename,hiredate) 
using 'python empexp.py' 
as (eno,ename,hiredate,cdate,e) 
from emp; 

しかし、私はランタイムエラーを取得していますが、ここでエラーログで、

import sys 
from datetime import datetime 
from dateutil.relativedelta import relativedelta 

cdate = datetime.now().date() 

for line in sys.stdin: 
    line = line.strip() 
    eno,ename,hiredate = line.split('\t') 

    hdate = datetime.strptime(hiredate,'%Y-%m-%d').date() 

    year = str(relativedelta(current_date,hdate).years) 
    month = str(relativedelta(current_date,hdate).months) 
    day = str(relativedelta(current_date,hdate).days) 

    e = year+'-'+month+'-'+day 
    list1 = [str(eno),str(ename),str(hiredate),str(cdate),str(e)] 

    print '\t'.join(list1) 

empexp.pyと私はハイブクエリを与えている以下でPythonスクリプトであります

Diagnostic Messages for this Task: 
Error: java.lang.RuntimeException: Hive Runtime Error while closing operators 
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207) 
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61) 
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) 
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) 
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) 
at java.security.AccessController.doPrivileged(Native Method) 
at javax.security.auth.Subject.doAs(Subject.java:422) 
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) 
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: [Error 20003]: An error occurred when trying to close the Operator running your custom script. 
at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:585) 
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) 
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) 
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) 
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189) 
... 8 more 


FAILED: Execution Error, return code 20003 from 
org.apache.hadoop.hive.ql.exec.mr.MapRedTask. 
An error occurred when trying to close the Operator running your custom  

script. 

私を助けてください。 ありがとうございます。

Answer 1

スクリプト内で関数を呼び出す前に、ファイルを分散キャッシュに追加する必要があります。あなたの助けのための

add file /path/empexp.py; 

その後

select transform(eno,ename,hiredate) 
using 'python empexp.py' 
as (eno,ename,hiredate,cdate,e) 
from emp;