I am pretty new to ML and tensorflow and am getting stuck. I am trying to do text classification.
My dataset is in the form where each row has 2 columns: text and polarity
text = string/tweet
polarity = can be 0 or 1
I am generating BERT embeddings following this code https://github.com/strongio/keras-bert/blob/master/keras-bert.ipynb
I want to add a Bi-LSTM between Bert Layer and the Dense layer. I have done it like this:
bert_output = BertLayer(n_fine_tune_layers=3, pooling="mean")(bert_inputs) bert_output = tf.keras.layers.Reshape((max_seq_length, embedding_size))(bert_output) bilstm = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(128, dropout=0.2,recurrent_dropout=0.2,return_sequences=True))(bert_output) output = tf.keras.layers.Dense(1, activation="softmax")(bilstm) model = tf.keras.models.Model(inputs=bert_inputs, outputs=output) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) model.summary()
It gives an error:
ValueError: A target array with shape (1288, 1) was passed for an output of shape (None, 256, 1) while using as loss `binary_crossentropy`. This loss expects targets to have the same shape as the output.
This is the link of the notebook in colab:
What can I do to resolve this? Does it have something to do with what activation or loss is being used ? How can the shape be matched?
Any help will be appreciated.