【ML】Negative Sampling

Negative sampling is a technique used in machine learning, particularly in the context of neural networks and embeddings, to improve training efficiency by reducing the number of samples required for training.

In many machine learning problems, the number of possible outcomes is very large, making it computationally expensive to train models using all possible outcomes. Negative sampling aims to address this issue by only considering a small subset of negative samples during training, rather than all possible negative samples.

In the context of neural networks and embeddings, negative sampling involves randomly selecting a few negative examples (i.e., samples that should not be associated with a particular entity or feature) to train the model alongside a positive example. The goal is to train the model to distinguish between positive and negative examples, rather than trying to explicitly learn to classify all possible outcomes.

For example, in the context of word embeddings, negative sampling involves randomly selecting a few words that are not associated with a particular context word during training. By training the model to distinguish between positive context-target word pairs and negative context-non-target word pairs, the model can learn to better capture the semantic relationships between words while requiring fewer training examples.

Overall, negative sampling can improve the efficiency and scalability of machine learning models while still producing accurate results.