Masked Language Modeling

← Back to Self-Supervised Learning

A pretraining task where random tokens in the input are masked and the model must predict them. Used by BERT and similar encoder models. Learns bidirectional contextual representations.

ml self-supervised mlm bert