Tensorflow MultiLSTMCellエラー

Question

私はtensorflow 1.1.0でリカレントニューラルネットワークを開発しようとしており、LSTMを返す関数を書いています。

def LSTM(x, num_units, num, num_layers=3): 
    cells = [] 
    for i in range(num_layers): 
    cell = LSTMCell(num_units=num_units, state_is_tuple=True) 
    cell = DropoutWrapper(cell=cell, output_keep_prob=0.5) 
    cells.append(cell) 

    lstm = MultiRNNCell(cells=cells, state_is_tuple=True) 
    val, state = tf.nn.dynamic_rnn(lstm, x, dtype=tf.float32) 

    val = tf.transpose(val, [1, 0, 2]) # rendo il risultato una sequenza 
    last = tf.gather(val, int(val.get_shape()[0]) - 1) # prendo l'output dell'ultimo elemento 

    return last 

この機能は、実際に動作しますが、私はそれを私は次のエラーを取得する複数回再利用しようとした場合：

C:\ProgramData\Anaconda3\envs\obra\python.exe C:/Users/Simone/Desktop/Cobra/LSTM_Function_Filtro.py 
Traceback (most recent call last): 
    File "C:/Users/Simone/Desktop/Cobra/LSTM_Function_Filtro.py", line 81, in <module> 
    Lstm2 = tf.nn.relu(tf.matmul(lyrs.LSTM(concat1, num_hidden, 1), W2) + B2) 
    File "C:\Users\Simone\Desktop\Cobra\Layers_OK.py", line 62, in LSTM 
    val, state = tf.nn.dynamic_rnn(lstm, x, dtype=tf.float32) 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\site-packages\tensorflow\python\ops\rnn.py", line 553, in dynamic_rnn 
    dtype=dtype) 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\site-packages\tensorflow\python\ops\rnn.py", line 720, in _dynamic_rnn_loop 
    swap_memory=swap_memory) 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2623, in while_loop 
    result = context.BuildLoop(cond, body, loop_vars, shape_invariants) 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2456, in BuildLoop 
    pred, body, original_loop_vars, loop_vars, shape_invariants) 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2406, in _BuildLoop 
    body_result = body(*packed_vars_for_body) 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\site-packages\tensorflow\python\ops\rnn.py", line 705, in _time_step 
    (output, new_state) = call_cell() 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\site-packages\tensorflow\python\ops\rnn.py", line 691, in <lambda> 
    call_cell = lambda: cell(input_t, state) 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\site-packages\tensorflow\contrib\rnn\python\ops\core_rnn_cell_impl.py", line 953, in __call__ 
    cur_inp, new_state = cell(cur_inp, cur_state) 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\site-packages\tensorflow\contrib\rnn\python\ops\core_rnn_cell_impl.py", line 713, in __call__ 
    output, new_state = self._cell(inputs, state, scope) 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\site-packages\tensorflow\contrib\rnn\python\ops\core_rnn_cell_impl.py", line 398, in __call__ 
    reuse=self._reuse) as unit_scope: 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\contextlib.py", line 59, in __enter__ 
    return next(self.gen) 
    File "C:\ProgramData\Anaconda3\envs\obra\lib\site-packages\tensorflow\contrib\rnn\python\ops\core_rnn_cell_impl.py", line 93, in _checked_scope 
    "the argument reuse=True." % (scope_name, type(cell).__name__)) 
ValueError: Attempt to have a second RNNCell use the weights of a variable scope that already has weights: 'rnn/multi_rnn_cell/cell_0/lstm_cell'; and the cell was not constructed as LSTMCell(..., reuse=True). To share the weights of an RNNCell, simply reuse it in your second calculation, or create a new one with the argument reuse=True. 

をまた私がのサイクルの終わりにtf.get_variable_scope().reuse_variables()を追加しようとしましたが、私は、エラー

Variable rnn/multi_rnn_cell/cell_0/lstm_cell/weights does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope? 

Answer 1

私は間違っていないよ場合は、パラメータを共有したり、同じコードを使用して別のRNNを作成したいを取得します。次のようにtf.variable_scopeを使用することができますもしそうなら：あなたの最初の利用時に

def LSTM(x, num_units, num_layers=3, reuse=False, scope="MultiRNNCell"): 
    with tf.variable_scope(name_or_scope=scope, reuse=reuse): 
     cells = [] 
     for i in range(num_layers): 
      cell = tf.nn.rnn_cell.LSTMCell(num_units=num_units, state_is_tuple=True) 
      cell = tf.nn.rnn_cell.DropoutWrapper(cell=cell, output_keep_prob=0.5) 
      cells.append(cell) 

     lstm = tf.nn.rnn_cell.MultiRNNCell(cells=cells, state_is_tuple=True) 
     val, state = tf.nn.dynamic_rnn(lstm, x, dtype=tf.float32) 

     val = tf.transpose(val, [1, 0, 2]) # rendo il risultato una sequenza 
     last = tf.gather(val, int(val.get_shape()[0]) - 1) # prendo l'output dell'ultimo elemento 

     return last 

をあなたはreuse引数Falseを渡す必要がありますので、tensorflowは、変数を作成します。パラメータを別のRNNと共有するには、Trueを渡すだけで十分です。新しいモデルを作成する場合は、新しいscopeという名前とreuse=Falseという名前を渡すことをお勧めします。以下の例では、実行を簡単にする必要があります。私はダミーのプレースホルダを作成しました。

MultiRNNCell/rnn/multi_rnn_cell/cell_0/lstm_cell/kernel:0 MultiRNNCell/rnn/multi_rnn_cell/cell_0/lstm_cell/bias:0 MultiRNNCell/rnn/multi_rnn_cell/cell_1/lstm_cell/kernel:0 MultiRNNCell/rnn/multi_rnn_cell/cell_1/lstm_cell/bias:0 MultiRNNCell/rnn/multi_rnn_cell/cell_2/lstm_cell/kernel:0 MultiRNNCell/rnn/multi_rnn_cell/cell_2/lstm_cell/bias:0

# of parameters: 108288

def list_parameters(): 
    num_param = 0 
    for v in tf.global_variables(): 
     print(v.name) 
     num_param += np.prod(v.get_shape().as_list()) 

    print("# of parameters: " + str(num_param)) 

x = tf.placeholder(dtype=tf.float32, 
       shape=[32, 50, 100], 
       name='input_data') 

lstm1 = LSTM(x, 64, 3, reuse=False, scope="MultiRNNCell") 
list_parameters() 

lstm2 = LSTM(x, 64, 3, reuse=True, scope="MultiRNNCell") 
list_parameters() 

MultiRNNCell/rnn/multi_rnn_cell/cell_0/lstm_cell/kernel:0 MultiRNNCell/rnn/multi_rnn_cell/cell_0/lstm_cell/bias:0 MultiRNNCell/rnn/multi_rnn_cell/cell_1/lstm_cell/kernel:0 MultiRNNCell/rnn/multi_rnn_cell/cell_1/lstm_cell/bias:0 MultiRNNCell/rnn/multi_rnn_cell/cell_2/lstm_cell/kernel:0 MultiRNNCell/rnn/multi_rnn_cell/cell_2/lstm_cell/bias:0

# of parameters: 108288

lstm1とlstm2パラメータを共有していることに留意されたいです。

lstm3 = LSTM(x, 64, 3, reuse=False, scope="NewMultiRNNCell") 
list_parameters() 

MultiRNNCell/rnn/multi_rnn_cell/cell_0/lstm_cell/kernel:0 MultiRNNCell/rnn/multi_rnn_cell/cell_0/lstm_cell/bias:0 MultiRNNCell/rnn/multi_rnn_cell/cell_1/lstm_cell/kernel:0 MultiRNNCell/rnn/multi_rnn_cell/cell_1/lstm_cell/bias:0 MultiRNNCell/rnn/multi_rnn_cell/cell_2/lstm_cell/kernel:0 MultiRNNCell/rnn/multi_rnn_cell/cell_2/lstm_cell/bias:0 NewMultiRNNCell/rnn/multi_rnn_cell/cell_0/lstm_cell/kernel:0 NewMultiRNNCell/rnn/multi_rnn_cell/cell_0/lstm_cell/bias:0 NewMultiRNNCell/rnn/multi_rnn_cell/cell_1/lstm_cell/kernel:0 NewMultiRNNCell/rnn/multi_rnn_cell/cell_1/lstm_cell/bias:0 NewMultiRNNCell/rnn/multi_rnn_cell/cell_2/lstm_cell/kernel:0 NewMultiRNNCell/rnn/multi_rnn_cell/cell_2/lstm_cell/bias:0

# of parameters: 216576

新しいパラメータのセットは、lstm3ためscopeのために作成され、従って、変数名は異なります。最後に、this postは変数の名前付けを明確に説明しています。