A setting chosen before training a machine learning model, such as learning rate or number of layers, that shapes how the model learns.
Machine learning models learn parameters, such as the weights in a neural network, from data. Hyperparameters are different: they are decisions made by the practitioner that control the training process itself. Examples include the learning rate, batch size, number of training epochs, tree depth in a decision tree, and the number of layers in a network.
Picking good hyperparameters can make a large difference in accuracy and training time. Too high a learning rate may cause training to fail, while too low makes it painfully slow. Practitioners use techniques such as grid search, random search, and Bayesian optimization, evaluating each combination on a validation dataset. Careless tuning on test data can lead to overfitting and overly optimistic results.
When a machine learning model learns its training data too closely, including noise, and performs poorly on new, unseen data.
Temperature (AI)A setting that controls how random or predictable a language model's output is, with lower values giving more consistent answers.
What MLOps means, why most ML effort goes into everything around the model, and a simple maturity path for versioning, deploying and monitoring models in production.