pandasデータフレームのdatetimeindexでの値のシフト

Question

長期間（> 1年）にわたってDateTimeIndexが30分のdfを持つため、> 17520行です。夏時間に関する理由から、2つのインデックス値がインデックスで繰り返され、2つの値が欠落しています。だから、重複値は次のとおりです。

In[1]: df[df.index.duplicated('first')] 
Out[2]: 
          a   b  c 
timestamp                 
2012-10-07 01:00:00   NaN  NaN  NaN  
2012-10-07 01:30:00   NaN  NaN  NaN  
2013-10-06 01:00:00   NaN  NaN  NaN  
2013-10-06 01:30:00   NaN  NaN  NaN  

は、私は1時間後に欠損値、これらを変更したい：

In[3]: df[df.index.duplicated('first')].shift(1,freq="H") 
Out[4]: 
          a   b  c 
timestamp                 
2012-10-07 02:00:00   NaN  NaN  NaN  
2012-10-07 02:30:00   NaN  NaN  NaN  
2013-10-06 02:00:00   NaN  NaN  NaN   
2013-10-06 02:30:00   NaN  NaN  NaN 

しかし、これは、インデックスを変更しない：

df[df.index.duplicated('first')] = df[df.index.duplicated('first')].shift(1,freq="H") 

何だろう？

Answer 1

私はあなたがdictによってrenameとduplicated indexをマップ必要だと思う：

print (df) 
        a b c 
timestamp      
2013-10-06 01:00:00 1 NaN NaN 
2013-10-06 01:30:00 2 NaN NaN 
2013-10-06 01:00:00 3 NaN NaN 
2013-10-06 01:30:00 4 NaN NaN 
2012-10-08 01:30:00 5 NaN NaN 
2013-10-10 01:00:00 6 NaN NaN 


df1 = df[df.index.duplicated('first')] 
d = dict(zip(df1.index, df1.shift(1,freq="H").index)) 
print (d) 
{Timestamp('2013-10-06 01:00:00'): Timestamp('2013-10-06 02:00:00'), 
Timestamp('2013-10-06 01:30:00'): Timestamp('2013-10-06 02:30:00')} 

df = df.rename(index=d) 
print (df) 
        a b c 
timestamp      
2013-10-06 02:00:00 1 NaN NaN 
2013-10-06 02:30:00 2 NaN NaN 
2013-10-06 02:00:00 3 NaN NaN 
2013-10-06 02:30:00 4 NaN NaN 
2012-10-08 01:30:00 5 NaN NaN 
2013-10-10 01:00:00 6 NaN NaN 

同様のソリューション：

idx = df.index[df.index.duplicated('first')] 
d = dict(zip(idx, idx.to_series().shift(freq="H").index)) 
print (d) 
{Timestamp('2013-10-06 01:00:00'): Timestamp('2013-10-06 02:00:00'), 
Timestamp('2013-10-06 01:30:00'): Timestamp('2013-10-06 02:30:00')} 

df = df.rename(index=d) 
print (df) 
        a b c 
timestamp      
2013-10-06 02:00:00 1 NaN NaN 
2013-10-06 02:30:00 2 NaN NaN 
2013-10-06 02:00:00 3 NaN NaN 
2013-10-06 02:30:00 4 NaN NaN 
2012-10-08 01:30:00 5 NaN NaN 
2013-10-10 01:00:00 6 NaN NaN 
2013-10-06 02:30:00 8 NaN NaN 
2012-10-08 01:30:00 9 NaN NaN 
2013-10-10 01:00:00 10 NaN NaN 

---

idx = df.index[df.index.duplicated('first')] 
s = idx.to_series().shift(freq="H") 
#swap index with values in Series 
d = pd.Series(s.index.values, index = s.values).to_dict() 
print (d) 
{Timestamp('2013-10-06 01:00:00'): Timestamp('2013-10-06 02:00:00'), 
Timestamp('2013-10-06 01:30:00'): Timestamp('2013-10-06 02:30:00')} 

df = df.rename(index=d) 
print (df) 
        a b c 
timestamp      
2013-10-06 02:00:00 1 NaN NaN 
2013-10-06 02:30:00 2 NaN NaN 
2013-10-06 02:00:00 3 NaN NaN 
2013-10-06 02:30:00 4 NaN NaN 
2012-10-08 01:30:00 5 NaN NaN 
2013-10-10 01:00:00 6 NaN NaN 

EDIT1：

あなたはcumcountによって作成されたtimedeltasとto_timedeltaを元のインデックスに追加する必要があります。

delta = pd.to_timedelta(df.groupby(level=0).cumcount(), unit='H') 
print (delta) 
timestamp 
2013-10-06 01:00:00 00:00:00 
2013-10-06 01:30:00 00:00:00 
2013-10-06 01:00:00 01:00:00 
2013-10-06 01:30:00 01:00:00 
2012-10-08 01:30:00 00:00:00 
2013-10-10 01:00:00 00:00:00 
dtype: timedelta64[ns] 

df.index = df.index + delta 
print (df) 
        a b c 
2013-10-06 01:00:00 1 NaN NaN 
2013-10-06 01:30:00 2 NaN NaN 
2013-10-06 02:00:00 3 NaN NaN 
2013-10-06 02:30:00 4 NaN NaN 
2012-10-08 01:30:00 5 NaN NaN 
2013-10-10 01:00:00 6 NaN NaN