7/28 Generative Pretrained Transformer
Finetuning BERT Models
We call training a pretrained model on a particular task finetuning. Here are a couple of common supervised NLP tasks:
- Sequence-level tasks:
- Sequence classification
- Question answering: does the answer actually answer the question?
- Sentence similarity
- Token-level tasks:
- Token classification
- Named Entity Recognition
CORRUPTION ALERT
Note that the original BERT model [1] was published in 2019. The paper introducing the IMDB dataset [2] was published in 2011. Therefore, it is possible that the pre-training corpus includes information on the validation set of the IMDB dataset. So, this is not a valid way of testing generalization performance. You need to make sure that your validation data didn't leak into your training data. The safest way is if it is possible to check that your dataset became public later than the time the pre-training corpus was crawled.
Domain Adaptation
We call Domain Adaptation the method of taking a model that has been pretrained on a general corpus such as Wikipeda or Common Crawl and running further pretraining on the corpus of the specific task that we ultimately want to finetune on. This often helps finetuning performance. Sometimes, one can even eschew general pretraining and get comparable result by pretraining only on the finetuning corpus [3].
Text Generation Models
Next Token Prediction
We generate text token by token. That is, given a partial sequence such as "The meaning of life is", we query the model what should the next token be.
Hiding ensemble dimensions, we use a trainable \(d\times v\) matrix, called the unembedding matrix, to transform a tensor \(Y=(b, s, d)\) of contextual token embeddings to a tensor \(Z=(b, s, v)\) of token logits. The vector \(Z[i, j]\) gives the prediction logits for the token that should follow the \(j\)-th token in the \(i\)-th sequence in the minibatch.
Causal Attention Mask
When the model sees "The meaning of life is", it does not know what tokens would follow the next token, so it should not take those positions into account.
Remember that (disregarding multiple heads), attention has the formula
that is the positional relations are encoded in the (batched) matrix \(\mathrm{softmax}(QK^T, \mathtt{axis}=-1)\), where its \(j, j'\) entry says how much the \(j'\)-th token embedding on the \(\ell\)-th level matters for the \(j\)-th token embedding on the \((\ell+1)\)-st level. We want the values for \(j'>j\) be zeros so that a token can't affect the tokens to its left. That is, we want the attention matrix be a lower triangular matrix. This operation is usually performed one step back, on the matrix \(QK^T\), using a so-called attention mask, by making the not masked entries \(-\infty\).
Next Token Prediction Pretraining
With this, we can perform pretraining in an unsupervised manner: for a minibatch \(S\) of sequences of token ids from the corpus, the loss is the cross-entropy between the next token logits we get out of this and the next token ids \(S[:, 1:]\); we end the sequences with the special token EOS.
It was discovered in [4] that after this pretraining one can finetune a model easier to different text generation tasks, for example question answering or text summarization. This is where GPT gets its name from: Generative Pretrained Transformer.
Variants on Generation
Based on one's preferences, for example how extraordinary of a text one wants to receive, there are variants on how to generate text based on the next token logits.
- Best First Search: At each step, we choose the token with the highest logit.
- Sampling: At each step, we sample between the logits with weights based on their prediction probabilities. In this context, the hyperparameter temperature is an exponent one can give the probabilities to control how unusual the received text is.
- Beam Seach: Another option is to keep multiple variations at hand. The hyperparameter beam width controls how many variants we keep.
This is just a very small sampling. For more options, you can see for example the transformers.GenerationConfig documentation. The transformers package was created by Hugging Face to help use transformer models.
References
[1] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. link
[3] Kundan Krishna, Saurabh Garg, Jeffrey Bigham, and Zachary Lipton: Downstream Datasets Make Surprisingly Good Pretraining Corpora. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12207--12222, 2023. link
[4] Alec Radford, Karthik Narasimhan, Tim Salimans and Ilya Sutskever. 2018. Improving Language Understanding by Generative Pre-Training. link
Dataset References
[2] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts: Learning Word Vectors for Sentiment Analysis, 2011. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL 2011), pages 142--150. link