【ML】Basic Term

Objective Function: In machine learning, an objective function is a mathematical function that is used to evaluate how well a machine learning algorithm is performing on a given task.

For example:

1. Cross-entropy loss

2. Mean Squared Error

Prediction: In machine learning, a model's prediction refers to the output that the model generates for a given input. (It could be the prediction based on the training dataset, test dataset, or validation dataset.)

Optimization Algorithm:

In machine learning, an optimization algorithm is a method that is used to adjust the parameters of a model in order to minimize a given objective function. Optimization algorithms typically involve iteratively updating the model's parameters in the direction of the steepest descent of the objective function, using some form of gradient calculation.

For example:

1. Gradient Descent

2. Stochastic Gradient Descent

3. Adam

Overall, loss function = objective function that optimization algorithms seek to minimize.