同じ値の複数列を集約するパンダ

Question

複数のデータソースを統合しようとしていますが、私は列を簡潔に組み合わせるのが難しいと感じています。

df = pd.DataFrame([['Address Data','City data','State Data', 'Zip Data', np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan], 
    [ np.nan,np.nan,np.nan,np.nan,'Address Data','City data','State Data', 'Zip Data',np.nan,np.nan,np.nan,np.nan], 
        [ np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,'Address Data','City data','State Data', 'Zip Data']], 
       columns = ['Address1','City1','State1','Zip1','Address2','City2','State2','Zip2','Address3','City3','State3','Zip3']) 

print df 

     Address1  City1  State1  Zip1  Address2  City2/ 
0 Address Data City Data State Data Zip Data   NaN  NaN 
1   NaN  NaN   NaN  NaN Address Data City Data 
2   NaN  NaN   NaN  NaN   NaN  NaN 

     State2  Zip2  Address3  City3  State3  Zip3 
0   NaN  NaN   NaN  NaN   NaN  NaN 
1 State Data Zip Data   NaN  NaN   NaN  NaN 
2   NaN  NaN Address Data City Data State Data Zip Data 

私はこのような何かに結合しようとしています：

#run code here to combine multiple versions of the columns 

print df 

    Address  City  State  Zip 
0 Address Data City data State Data Zip Data 
1 Address Data City data State Data Zip Data 
2 Address Data City data State Data Zip Data 

私は一度に複数の列を設定するには、このような行を使用することができます期待していた私は、このようになりますデータフレームを持っています：

df.loc[df['State1'].notnull(),['Address','city','state','State','Zip']] = df.loc[df['State1'].notnull(),['Address1','City1','State1','Zip1']].values 

ただし、.locでは複数の列を同時に作成することはできません。データをどのように組み合わせるかについてのご意見は大変ありがとうございます。

編集

以下のソリューションに基づいて、その解決策の問題は、私のデータは、実際にはもっとこのように見えることがあります：この場合

df = pd.DataFrame([['Address Data','City data','State Data', 'Zip Data', 'Address Data','City data','State Data', 'Zip Data',np.nan,np.nan,np.nan,np.nan], 
[ np.nan,np.nan,np.nan,np.nan,'Address Data','City data','State Data', 'Zip Data',np.nan,np.nan,np.nan,np.nan], 
       [ np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,'Address Data','City data','State Data', 'Zip Data']], 
      columns = ['Address1','City1','State1','Zip1','Address2','City2','State2','Zip2','Address3','City3','State3','Zip3']) 

を、以下soltuionはとき、私の4行を与えます私は欲しいのは、「セクション1がデータを持っていればセクション1を使用し、セクション2がデータ使用セクション2を持っているならば、そうでなければセクション3がデータを持っているならセクション3を使う」と言いたい。私は統合していない行の多くの他の属性があるので、各行はユニークなままでなければなりません。ありがとう！

Answer 1

私は解決策が全く異なっていると思いますので、私は新しい答えを作成決める：

import pandas as pd 
import numpy as np 

#random dataframe 
np.random.seed(1) 
df1 = pd.DataFrame(np.random.randint(10, size=(3,9))) 
df1.columns = ['a1','b1','c1','a2','b2','c2','a3','b3','c3'] 

df1.loc[[1,2],['a1','b1','c1']] = np.nan 
print (df1) 
    a1 b1 c1 a2 b2 c2 a3 b3 c3 
0 5.0 8.0 9.0 5 0 0 1 7 6 
1 NaN NaN NaN 5 2 4 2 4 7 
2 NaN NaN NaN 7 0 6 9 9 7 

#stack dataframe and extract string and numbers from column e 
df = df1.stack().reset_index() 
df.columns= ['d','e','f'] 
df[['g','h']] = df.e.str.extract(r'([a-zA-Z]+)([0-9]+)', expand=True) 

#append 1 to d, because index starts from 1 and compare with h 
df = df[df.d + 1 == df.h.astype(int)] 
#remove columns h, e 
df = df.drop(['h', 'e'], axis=1) 
#reshaping 
df = df.pivot(index='d', columns='g', values='f') 
#remove index and columns names (pandas 0.18.0+) 
df = df.rename_axis(None).rename_axis(None, axis=1) 

print (df) 
    a b c 
0 5.0 8.0 9.0 
1 5.0 2.0 4.0 
2 9.0 9.0 7.0 

編集：Iビットあなたのサンプルを変更してみてください。これについて

import pandas as pd 
import numpy as np 

df1 = pd.DataFrame([['Address Data1','City data1','State Data1', 'Zip Data1', 'Address Data2','City data2','State Data2', 'Zip Data2',np.nan,np.nan,np.nan,np.nan], 
[ np.nan,np.nan,np.nan,np.nan,'Address Data3','City data3','State Data3', 'Zip Data3',np.nan,np.nan,np.nan,np.nan], 
       [ np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,'Address Data4','City data4','State Data4', 'Zip Data4']], 
      columns = ['Address1','City1','State1','Zip1','Address2','City2','State2','Zip2','Address3','City3','State3','Zip3']) 

print (df1) 
     Address1  City1  State1  Zip1  Address2 \ 
0 Address Data1 City data1 State Data1 Zip Data1 Address Data2 
1   NaN   NaN   NaN  NaN Address Data3 
2   NaN   NaN   NaN  NaN   NaN 

     City2  State2  Zip2  Address3  City3  State3 \ 
0 City data2 State Data2 Zip Data2   NaN   NaN   NaN 
1 City data3 State Data3 Zip Data3   NaN   NaN   NaN 
2   NaN   NaN  NaN Address Data4 City data4 State Data4 

     Zip3 
0  NaN 
1  NaN 
2 Zip Data4 

#stack dataframe and extract string and numbers from column e 
df = df1.stack().reset_index() 
df.columns= ['d','e','f'] 
df[['g','h']] = df.e.str.extract(r'([a-zA-Z]+)([0-9]+)', expand=True) 

#append 1 to d, because index starts from 1 and compare with h 
df = df[df.d + 1 == df.h.astype(int)] 
#remove columns h, e 
df = df.drop(['h', 'e'], axis=1) 
#reshaping 
df = df.pivot(index='d', columns='g', values='f') 

df = df.rename_axis(None).rename_axis(None, axis=1) 
print (df) 
     Address  City  State  Zip 
0 Address Data1 City data1 State Data1 Zip Data1 
1 Address Data3 City data3 State Data3 Zip Data3 
2 Address Data4 City data4 State Data4 Zip Data4 

Answer 2

IIUC最初list理解することによって、列名を選択して使用することができますpd.lreshape：

a = ([col for col in df.columns if col.startswith('Address')]) 
c = ([col for col in df.columns if col.startswith('City')]) 
s = ([col for col in df.columns if col.startswith('State')]) 
z = ([col for col in df.columns if col.startswith('Zip')]) 

print (a) 
print (c) 
print (s) 
print (z) 
['Address1', 'Address2', 'Address3'] 
['City1', 'City2', 'City3'] 
['State1', 'State2', 'State3'] 
['Zip1', 'Zip2', 'Zip3'] 

df1 = pd.lreshape(df, {'Address':a, 'City' :c, 'State':s, 'Zip' :z}) 
print (df1) 

     Address  State  City  Zip 
0 Address Data State Data City data Zip Data 
1 Address Data State Data City data Zip Data 
2 Address Data State Data City data Zip Data 

EDIT：

はあなたがNaN値でませドロップ行が必要な場合は、パラメータdropna=Falseを追加します。

df1 = pd.lreshape(df, {'Address':a, 'City' :c, 'State':s, 'Zip' :z}, dropna=False) 
print (df1) 
     Address  State  City  Zip 
0 Address Data State Data City data Zip Data 
1   NaN   NaN  NaN  NaN 
2   NaN   NaN  NaN  NaN 
3   NaN   NaN  NaN  NaN 
4 Address Data State Data City data Zip Data 
5   NaN   NaN  NaN  NaN 
6   NaN   NaN  NaN  NaN 
7   NaN   NaN  NaN  NaN 
8 Address Data State Data City data Zip Data 

もう1つの小数点イオンが、データが（サンプル中はい、本当ではないかもしれない）注文素晴らしいことができます：

print (pd.DataFrame((df.values.reshape(9,4)), columns=['Address','City','State','Zip']) 
     .dropna()) 

     Address  City  State  Zip 
0 Address Data City data State Data Zip Data 
4 Address Data City data State Data Zip Data 
8 Address Data City data State Data Zip Data 

Answer 3

何？

uniq_cols = df.columns.str.replace(r'\d+$', '').unique() 

new = pd.DataFrame(columns=uniq_cols) 

for col in uniq_cols: 
    new[col] = df.filter(like=col).apply(lambda x: x.dropna()[0], axis=1) 

出力：

In [282]: new 
Out[282]: 
     Address  City  State  Zip 
0 Address Data City data State Data Zip Data 
1 Address Data City data State Data Zip Data 
2 Address Data City data State Data Zip Data 

説明：

In [283]: uniq_cols 
Out[283]: array(['Address', 'City', 'State', 'Zip'], dtype=object) 

In [284]: df.filter(like='Address').apply(lambda x: x.dropna()[0], axis=1) 
Out[284]: 
0 Address Data 
1 Address Data 
2 Address Data 
dtype: object