多層LSTMの前に埋め込み層を追加する方法は？

Question

私は、ストーリージェネレータを実行する際に、単語表現としてグローブベクトルを実装したいと考えています。私は2層のLSTMを使用しています。出力にsoftmaxのレイヤーが完全に接続されています。

archetectureは次のようになります。私の入力の場合

Input --> LSTM --> LSTM --> Fully connected --> Output 

、モデルは三つの言葉と出力これら3つの単語に基づいて単語を取る必要があります。各入力は次元25のベクトルです。私のテキストでは私は訓練のために使用していますが、100のラベルしかありません。各LSTMには512個の隠れユニットがあります。

参照してください以下の私のコード：

# Parameters 
learning_rate = 0.001 
training_iters = 50000 
display_step = 1000 
n_input = 3 
n_hidden = 512 

# tf Graph input 
x = tf.placeholder("float", [None, n_input, glove_dim]) 
y = tf.placeholder("float", [None, vocab_size]) 

# RNN output node weights and biases 
weights = {'out': tf.Variable(tf.random_normal([n_hidden, vocab_size]))} 
biases = {'out': tf.Variable(tf.random_normal([vocab_size]))} 

def RNN(x, weights, biases): 

    # reshape to [1, n_input] 
    x = tf.reshape(x, [-1, n_input]) 

    # Generate a n_input-element sequence of inputs 
    x = tf.split(x,n_input,1) 


    rnn_cell =rnn.MultiRNNCell([rnn.BasicLSTMCell(n_hidden),rnn.BasicLSTMCell(n_hidden)]) 

    # generate prediction 
    outputs, states = rnn.static_rnn(rnn_cell, x, dtype=tf.float32) 

    return tf.matmul(outputs[-1], weights['out']) + biases['out'] 

pred = RNN(x, weights, biases) 

# Loss and optimizer 
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y)) 
optimizer = tf.train.RMSPropOptimizer(learning_rate=learning_rate).minimize(cost) 

# Model evaluation 
correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1)) 
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32)) 

# Initializing the variables 
init = tf.global_variables_initializer() 

# Launch the graph 
with tf.Session() as session: 
    session.run(init) 
    step = 0 
    offset = random.randint(0,n_input+1) 
    end_offset = n_input + 1 
    acc_total = 0 
    loss_total = 0 

    writer.add_graph(session.graph) 

    while step < training_iters: 
     # Generate a minibatch. Add some randomness on selection process. 
     if offset > (len(training_data)-end_offset): 
      offset = random.randint(0, n_input+1) 

     symbols_in_keys = [ [glove_dictionary[ str(training_data[i])]] for i in range(offset, offset+n_input) ] 
     symbols_in_keys = np.reshape(np.array(symbols_in_keys), [-1, n_input, glove_dim]) 

     symbols_out_onehot = np.zeros([vocab_size], dtype=float) 
     symbols_out_onehot[dictionary[str(training_data[offset+n_input])]] = 1.0 
     symbols_out_onehot = np.reshape(symbols_out_onehot,[1,-1]) 

     _, acc, loss, onehot_pred = session.run([optimizer, accuracy, cost, pred], \ 
              feed_dict={x:symbols_in_keys, y: symbols_out_onehot}) 
     loss_total += loss 
     acc_total += acc 
     if (step+1) % display_step == 0: 
      print("Iter= " + str(step+1) + ", Average Loss= " + \ 
        "{:.6f}".format(loss_total/display_step) + ", Average Accuracy= " + \ 
        "{:.2f}%".format(100*acc_total/display_step)) 
      acc_total = 0 
      loss_total = 0 
      symbols_in = [training_data[i] for i in range(offset, offset + n_input)] 
      symbols_out = training_data[offset + n_input] 
      symbols_out_pred = reverse_dictionary[int(tf.argmax(onehot_pred, 1).eval())] 
      print("%s - [%s] vs [%s]" % (symbols_in,symbols_out,symbols_out_pred)) 
    step += 1 
    offset += (n_input+1) 
    print("Optimization Finished!") 
    print("Elapsed time: ", elapsed(time.time() - start_time)) 
    print("Run on command line.") 
    print("\ttensorboard --logdir=%s" % (logs_path)) 
    print("Point your web browser to: http://localhost:6006/") 
    while True: 
     prompt = "%s words: " % n_input 
     sentence = input(prompt) 
     sentence = sentence.strip() 
     words = sentence.split(' ') 
     if len(words) != n_input: 
      continue 
     try: 
      symbols_in_keys = [glove_dictionary[str(words[i])] for i in range(len(words))] 
      for i in range(32): 
       keys = np.reshape(np.array(symbols_in_keys), [-1, n_input, 1]) 
       onehot_pred = session.run(pred, feed_dict={x: keys}) 
       onehot_pred_index = int(tf.argmax(onehot_pred, 1).eval()) 
       sentence = "%s %s" % (sentence,reverse_dictionary[onehot_pred_index]) 
       symbols_in_keys = symbols_in_keys[1:] 
       symbols_in_keys.append(onehot_pred_index) 
      print(sentence) 
     except: 
      print("Word not in dictionary") 

私はこれを実行すると、私はエラーになっています：

InvalidArgumentError (see above for traceback): logits and labels must have the same first dimension, got logits shape [160,14313] and labels shape [10] 
    [[Node: SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits = SparseSoftmaxCrossEntropyWithLogits[T=DT_FLOAT, Tlabels=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"](add, Reshape_1)]] 

私は形状が決定されるlogitsする方法を知ることができると私は何を行うことができますが私のコードを修正するには？

Answer 1

私はあなたが行うとき

x

# reshape to [1, n_input] 
x = tf.reshape(x, [-1, n_input]) 

# Generate a n_input-element sequence of inputs 
x = tf.split(x,n_input,1) 

が最初 (batch_size * glove_dim, n_input)

に作り直さその後

は、このようにrnn.static_rnnがinput_sizeとして1を取ると、あなたはそれをプロジェクト(batch_size * glove_dim, 1)に分割されているから問題が来ると思いますweightの行列を掛けてvocab_sizeに設定します。出力は(batch_size * glove_dim, vocab_size)

たぶん、あなたはこれが実際に動作x = tf.split(x,n_input,1)