Visualize gradient descent on 2D loss surfaces. Compare SGD, momentum, Adam optimizers. Watch convergence paths and learning rate effects.
Gradient Descent Visualizer turns successive optimization steps into paths across a loss surface with two adjustable dimensions. It compares how SGD, momentum, and Adam move from a starting point as the learning rate affects progress, oscillation, and stability. The main trap is treating the visually shortest path as proof that an optimizer is generally superior. A trajectory describes one surface, starting point, parameter configuration, and iteration budget; changing any of them can reverse the apparent result.
Each plotted path is an ordered sequence of parameter positions on the selected loss surface. In the SGD comparison, a step moves against the gradient used at that iteration, with its size influenced by the learning rate. Momentum also carries forward accumulated direction. Adam maintains gradient-based moving estimates that adjust the update separately along the two displayed parameter dimensions.
Closely spaced points can indicate that updates are becoming small, while repeated crossings of a valley suggest oscillation. Large jumps or movement away from a low region can indicate an excessive learning rate. For a meaningful visual comparison, keep the surface, starting point, and iteration count fixed. Equal learning rates still do not imply equal effective steps across SGD, momentum, and Adam.
Momentum retains part of the preceding update direction, so the optimizer can continue moving after the current gradient begins pointing back toward the valley floor. It may then cross the valley repeatedly while the motion decays. A smaller learning rate or different momentum setting can reduce this behavior, but overshooting is not automatically evidence that momentum has failed.
No. Adam adapts updates using running estimates of gradient behavior, which can produce a direct-looking path on some surfaces. The result still depends on the surface, starting point, learning rate, optimizer settings, and number of iterations. This visualization can compare those particular trajectories, but it cannot establish that Adam will train every real model faster or reach a better solution.
The behavioral notes were checked against the browser implementation. Standards and primary references below define the relevant format, formula, or platform behavior.
Calculate percentages, percentage change, and what percent one number is of another with step-by-step breakdowns.
Calculate tips and split bills between multiple people with customizable tip percentages and rounding options.
Calculate monthly payments, total interest, and generate amortization schedules for loans and mortgages.
Calculate Body Mass Index from height and weight with category classification and healthy range guidance.
Calculate sale prices, savings amounts, and final costs after single or stacked percentage discounts.
Compute mean, median, mode, standard deviation, variance, and other descriptive statistics from a data set.