ハイブスパークの無限の接続が

Question

イムはわずか10行と列に単純なクエリを実行しようとしている：

> 2017-05-10T16:23:40,397 DEBUG [IPC Parameter Sending Thread #0] 
> ipc.Client: IPC Client (1360312263) connection to /0.0.0.0:8032 from 
> ubuntu sending #1841 2017-05-10T16:23:40,397 DEBUG [IPC Client 
> (1360312263) connection to /0.0.0.0:8032 from ubuntu] ipc.Client: IPC 
> Client (1360312263) connection to /0.0.0.0:8032 from ubuntu got value 
> #1841 2017-05-10T16:23:40,397 DEBUG [main] ipc.ProtobufRpcEngine: Call: getApplicationReport took 0ms 2017-05-10T16:23:41,397 DEBUG 
> [main] security.UserGroupInformation: PrivilegedAction as:ubuntu 
> (auth:SIMPLE) 
> from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323) 
> 2017-05-10T16:23:41,398 DEBUG [IPC Parameter Sending Thread #0] 
> ipc.Client: IPC Client (1360312263) connection to /0.0.0.0:8032 from 
> ubuntu sending #1842 2017-05-10T16:23:41,398 DEBUG [IPC Client 
> (1360312263) connection to /0.0.0.0:8032 from ubuntu] ipc.Client: IPC 
> Client (1360312263) connection to /0.0.0.0:8032 from ubuntu got value 
> #1842 2017-05-10T16:23:41,398 DEBUG [main] ipc.ProtobufRpcEngine: Call: getApplicationReport took 1ms 2017-05-10T16:23:41,399 DEBUG 
> [main] security.UserGroupInformation: PrivilegedAction as:ubuntu 
> (auth:SIMPLE) 
> from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323) 
> 2017-05-10T16:23:41,399 DEBUG [IPC Parameter Sending Thread #0] 
> ipc.Client: IPC Client (1360312263) connection to /0.0.0.0:8032 from 
> ubuntu sending #1843 2017-05-10T16:23:41,399 DEBUG [IPC Client 
> (1360312263) connection to /0.0.0.0:8032 from ubuntu] ipc.Client: IPC 
> Client (1360312263) connection to /0.0.0.0:8032 from ubuntu got value 
> #1843 2017-05-10T16:23:41,399 DEBUG [main] ipc.ProtobufRpcEngine: Call: getApplicationReport took 0ms 

Answer 1

問題：スパークアプリケーションは、次のメッセージを無限に実行されますが

select MAX(Column3) from table; 

不健全なノードに関連していたため、タスクを割り当てることができませんでした。ディスクが97％使用されているため、解決策は、yarn-site.xmlの糸最大ディスク使用率を向上させることでした。

<property> 
     <name>yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage</name> 
     <value>99</value> 
</property>