1,045 technical terms and definitions
gradient compression communication-efficient prediction, gradient compression communication-efficient, compression communication-efficient prediction
Gradient Compression Communication-Efficient Prediction with Machine Learning Introduction & Motivation Gradient Compression Communication-Efficient Prediction with Machine…
privacy
Gradient Compression for Privacy is the use of gradient compression techniques (sparsification, quantization) to reduce privacy leakage in distributed training — by transmitting…
gradient compression and quantization for communication efficiency, gradient compression quantization, compression quantization communication, quantization communication efficiency
Gradient Compression and Quantization for Communication Efficiency Introduction & Motivation Gradient Compression and Quantization for Communication Efficiency addresses a central…
distributed training
Gradient compression techniques is the communication-reduction methods that lower distributed training bandwidth demand by encoding or sparsifying gradients - they reduce…
top k sparsification, gradient sparsity training, magnitude based pruning, sparse gradient communication
Gradient Compression Techniques are the family of methods that reduce gradient communication volume by transmitting only the most important gradient components — using…
learning rate, what is gradient descent
Gradient descent is the algorithm that actually does the "learning" in machine learning: it repeatedly nudges a model's parameters in the direction that reduces the loss function…
gradient descent, descent optimizers, cifar glue lamb lars nlp sgd
Gradient Descent Optimizers Introduction & Motivation Optimization: update parameters to minimize loss. SGD, Adam, AdamW. Applications: training convergence, hyperparameter…
gradient descent variants batch mini batch stochastic, gradient descent, variants batch mini, bert cifar gd gpu lr mnist, gradient descent variants batch mini-batch & stochastic
Gradient Descent Variants: Batch, Mini-Batch & Stochastic Introduction & Motivation Gradient descent update policy: determines optimization trajectory. Batch GD: full dataset per…
Gradient Descent Algorithm Variants Introduction & Motivation Gradient Descent Algorithm Variants addresses a central problem in optimization algorithm tuning and improvement: how…
gem, continual learning
Gradient episodic memory is a continual-learning algorithm that constrains new-task gradients so they do not increase loss on stored past-task examples - Projected gradients…
computer vision
Gradient flow in deep ViTs is the mechanism that determines whether supervision signals can propagate across many transformer layers without vanishing or exploding - controlling…
model training
Gradient Flow Preservation is a design principle for pruning and sparse training — ensuring that removing weights does not disrupt the backpropagation signal, keeping gradient…
gradient flow stability in deep networks prediction, gradient flow stability, flow stability deep, stability deep networks, deep networks prediction, gradient flow
Gradient Flow Stability in Deep Networks Prediction with Machine Learning Introduction & Motivation Gradient Flow Stability in Deep Networks Prediction with Machine Learning…
Evolution Strategies for Gradient-Free Meta-Learning Introduction & Motivation Evolution Strategies for Gradient-Free Meta-Learning addresses a central problem in fewshot…
Nelder-Mead Simplex Direct Search Algorithm Introduction & Motivation Nelder-Mead Simplex Direct Search Algorithm addresses a central problem in optimization algorithm tuning…
Score-CAM Gradient-Free Visual Activation Mapping Introduction & Motivation Score-CAM Gradient-Free Visual Activation Mapping addresses a central problem in explainability…
ai safety
Gradient Masking is a phenomenon where a defense accidentally or intentionally makes the model's gradients uninformative — causing gradient-based attacks to fail while the model…
Gradient Masking Pitfalls Introduction & Motivation Gradient Masking Pitfalls addresses a central problem in robustness of advrobust models under adversarial conditions: how…
Gradient Masking Pitfalls (Extended Study) Introduction & Motivation Gradient Masking Pitfalls (Extended Study) addresses a central problem in robustness of advrobust models under…
Gradient Masking Pitfalls (Extended Study) Introduction & Motivation Gradient Masking Pitfalls (Extended Study) addresses a central problem in robustness of advrobust models under…
optimization
Gradient Noise is the deliberate addition of noise to gradient updates during training — typically Gaussian noise with decaying variance, which helps escape local minima, improves…
Gradient Norm Expectation for Deep Model Active Selection Introduction & Motivation Gradient Norm Expectation for Deep Model Active Selection addresses a central problem…
optimization
Gradient Normalization is the practice of normalizing gradient magnitudes during training — either by clipping the gradient norm to a maximum value (gradient clipping) or…
generative models
Gradient Penalty is a regularization technique used primarily in GAN training (WGAN-GP) — penalizing the norm of the discriminator's gradient with respect to its input, enforcing…
distributed training
Gradient quantization for communication reduces the precision of gradient tensors before transmitting them between workers in distributed training, dramatically reducing network…
domain adaptation
The Gradient Reversal Layer (GRL) is the ingenious mathematical trick at the beating heart of Adversarial Domain Adaptation (specifically DANN), functioning as a simple, custom…
optimization
Mixed-precision training is the standard recipe that lets modern models train in half the memory and roughly twice the throughput without losing accuracy. The idea is simple…
Gradient Similarity and Dot-Product Task Affinity Introduction & Motivation Gradient Similarity and Dot-Product Task Affinity addresses a central problem in transfer learning…
optimization
Gradient Sparsification is a communication reduction technique for distributed training that transmits only a subset of gradient components — sending only the most important…
Gradient Surgery for Multi-Task Learning (PCGrad) Introduction & Motivation Gradient Surgery for Multi-Task Learning (PCGrad) addresses a central problem in multitask learning…
Gradient Surgery for Multi-Task Learning (PCGrad) (Extended Study) Introduction & Motivation Gradient Surgery for Multi-Task Learning (PCGrad) (Extended Study) addresses a central…
Gradient Surgery for Multi-Task Learning (PCGrad) (Extended Study) Introduction & Motivation Gradient Surgery for Multi-Task Learning (PCGrad) (Extended Study) addresses a central…
PCGrad Projected Conflicting Gradients Multi-Task Introduction & Motivation PCGrad Projected Conflicting Gradients Multi-Task addresses a central problem in optimization algorithm…
distributed training
Gradient synchronization is the distributed operation that aligns per-worker gradients into a shared update before parameter step - it ensures data-parallel replicas remain…
GradNorm Loss Balancing Introduction & Motivation GradNorm Loss Balancing addresses a central problem in multitask learning across several related prediction objectives: how…
GradNorm Loss Balancing (Extended Study) Introduction & Motivation GradNorm Loss Balancing (Extended Study) addresses a central problem in multitask learning across several…
GradNorm Loss Balancing (Extended Study) Introduction & Motivation GradNorm Loss Balancing (Extended Study) addresses a central problem in multitask learning across several…
Gradual Domain Adaptation along Intermediate Trajectories Introduction & Motivation Gradual Domain Adaptation along Intermediate Trajectories addresses a central problem…
percentage, traffic
Gradual Rollout Gradual rollout (also called canary deployment or progressive delivery) incrementally increases traffic to a new model or system version—1%, 5%, 10%, 25%, 50%…
deployment
Gradual rollout (also called canary deployment or progressive delivery) is a deployment strategy where a new version of a model, feature, or service is released to a small subset…
fine-tuning
Gradual Unfreezing is an alternative name for Progressive Unfreezing — the fine-tuning strategy where pre-trained layers are incrementally unfrozen from top to bottom over…
dashboard, visualize
Grafana is the open-source observability platform that connects to multiple data sources and renders unified dashboards for metrics, logs, and traces — serving as the "single pane…
mlops
Grafana is an open-source visualization and analytics platform that creates dashboards, graphs, and alerts from time-series data sources. It is the most widely used tool…
defects
Grain Boundaries are interfaces separating crystallites (grains) of the same material that have different crystallographic orientations — they are regions of atomic disorder where…
grain boundary analysis, grain boundary character distribution, gbcd, five parameter grain boundary, grain boundary misorientation, grain boundary plane characterization
A grain boundary is only a few atomic spacings wide, yet a connected network of those interfaces can govern the resistivity of an interconnect, the lifetime of a solder joint…
grain boundary density prediction, grain boundary density, boundary density prediction, grain boundary, boundary density, density prediction
Grain Boundary Density Prediction with Machine Learning Introduction & Motivation Grain Boundary Density Prediction with Machine Learning addresses a central problem in grain…
defects
Grain Boundary Energy is the excess free energy per unit area associated with the disordered atomic arrangement at a grain boundary compared to the perfect crystal interior — this…
high-angle grain boundary, defects, crystal defects
High-Angle Grain Boundary (HAGB) is a grain boundary with a misorientation angle exceeding approximately 15 degrees, where the atomic structure is fundamentally disordered…
grain boundary, boundary modeling, csl dft gb gbdb ml
Grain Boundary Modeling Introduction & Motivation Predicting grain boundary properties and segregation enables design of materials with optimized microstructure and mechanical…
defects
Grain Boundary Segregation is the thermodynamically driven accumulation of solute atoms (dopants, impurities, or alloying elements) at grain boundaries where the disordered atomic…