ハイブのパフォーマンスを向上させる方法

Question

以下のクエリを実行しています。これは、ハイブで約2百万回あります。パフォーマンスを改善する方法はありますか？ソースハイブテーブルあなたは参加せずにそれを行うことができますcreated_date

select t.id, 
    case when t.amt_1_rank < 0.3*f.amt_1_count then t.amt_1 else null end as amt_1, 
    case when t.amt_2_rank < 0.3*f.amt_2_count then t.amt_2 else null end as amt_2, 
.. 
.. 
.. -- Like wise 30 columns e.g. amt_3,amt_3... 
    from (
    select a.id, 
    a.amt_1, 
    row_number() over (ORDER BY cast(a.amt_1 AS DECIMAL(8,7)) DESC) AS amt_1_rank, 
    a.amt_2, 
    row_number() over (ORDER BY cast(a.amt_2 AS DECIMAL(8,7)) DESC) AS amt_2_rank 
    from source_table a WHERE created_date='2017-10-15')t 
    join 
    ( 
    SELECT count(case when amt_1='.' then null else 1 end) AS amt_1_count, 
    count(case when amt_2='.' then null else 1 end) AS amt_2_count, 
.. 
.. 

    FROM source_table 
    WHERE created_date='2017-10-15' 
    ) f 

Answer 1

のパーティション列です：

select t.id, 
    case when t.amt_1_rank < 0.3*t.amt_1_count then t.amt_1 else null end as amt_1, 
    case when t.amt_2_rank < 0.3*t.amt_2_count then t.amt_2 else null end as amt_2, 
.. 
.. 
.. -- Like wise 30 columns e.g. amt_3,amt_3... 
    from (
    select a.id, 
      a.amt_1, 
      row_number() over (ORDER BY cast(a.amt_1 AS DECIMAL(8,7)) DESC) AS amt_1_rank, 
      a.amt_2, 
      row_number() over (ORDER BY cast(a.amt_2 AS DECIMAL(8,7)) DESC) AS amt_2_rank, 
      count(amt_1_flag) over()          AS amt_1_count, 
      count(amt_2_flag) over()          AS amt_2_count 
     from 
      (select a.*, 
        case when amt_1='.' then null else 1 end as amt_1_flag, 
        case when amt_2='.' then null else 1 end as amt_2_flag 
       from source_table a WHERE created_date='2017-10-15' 
      )a 

      )t