PySpark - 複数の辞書の値を分割する方法

Question

生のデータを処理した後、私の結果は次のようになります。複数の値を持つキー、値は辞書の値です - Key +各辞書の値

[('HOMICIDE', {'2017': 1}), 
('DECEPTIVE PRACTICE', {'2015': 2, '2017': 2, '2016': 8}), 
('ROBBERY', {'2016': 2}), 
('OTHER OFFENSE', {'2016': 3}), 
('MOTOR VEHICLE THEFT', {'2017': 2, '2016': 2})] 

どのように私は私がリストに辞書を移動する必要があり、そこから私が処理する必要があるかどうか

('HOMICIDE', '2017': 1), 
('DECEPTIVE PRACTICE', '2015': 2), 
('DECEPTIVE PRACTICE', '2017': 2,), 
('DECEPTIVE PRACTICE', '2016': 8), 
('ROBBERY', '2016': 2), 
('OTHER OFFENSE', '2016': 3), 
('MOTOR VEHICLE THEFT', '2017': 2) 
('MOTOR VEHICLE THEFT', '2016': 2)] 

に上記を行うことができますか？

Answer 1

ちょうどflatMapValues：

In [1]: rdd = sc.parallelize([('HOMICIDE', {'2017': 1}), 
    ...: ('DECEPTIVE PRACTICE', {'2015': 2, '2017': 2, '2016': 8}), 
    ...: ('ROBBERY', {'2016': 2}), 
    ...: ('OTHER OFFENSE', {'2016': 3}), 
    ...: ('MOTOR VEHICLE THEFT', {'2017': 2, '2016': 2})]) 

In [4]: rdd.flatMapValues(dict.items) 
Out[4]: PythonRDD[5] at RDD at PythonRDD.scala:48 

In [5]: rdd.flatMapValues(dict.items).collect() 
Out[5]: 
[('HOMICIDE', ('2017', 1)), 
('DECEPTIVE PRACTICE', ('2015', 2)), 
('DECEPTIVE PRACTICE', ('2017', 2)), 
('DECEPTIVE PRACTICE', ('2016', 8)), 
('ROBBERY', ('2016', 2)), 
('OTHER OFFENSE', ('2016', 3)), 
('MOTOR VEHICLE THEFT', ('2017', 2)), 
('MOTOR VEHICLE THEFT', ('2016', 2))] 

または長い道のりで

In [5]: rdd.flatMap(lambda x: [(x[0], k, v) for k, v in x[1].items()]).collect() 
Out[5]:                   
[('HOMICIDE', '2017', 1), 
('DECEPTIVE PRACTICE', '2015', 2), 
('DECEPTIVE PRACTICE', '2017', 2), 
('DECEPTIVE PRACTICE', '2016', 8), 
('ROBBERY', '2016', 2), 
('OTHER OFFENSE', '2016', 3), 
('MOTOR VEHICLE THEFT', '2017', 2), 
('MOTOR VEHICLE THEFT', '2016', 2)] 

Answer 2

怒鳴るが働いてしまった - rdd.flatMapValues。（ラムダデータ：data.items（）））（コレクト

@StackPointerと@ user8119199のおかげで