1,517 technical terms and definitions
Tensor Analysis: Multilinear Algebra, Index Notation, and Applications to Physics Tensors generalize scalars, vectors, and matrices to objects of arbitrary rank that transform…
Tensor Calculus, Metric Geometry, and Multilinear Quantum Spaces 1. Tensors beyond multidimensional arrays A multidimensional array is a coordinate representation. A tensor is…
Tensor Computation, Multi-Dimensional Analysis, and High-Order Tensor Networks 1. Executive Summary & Foundation of Tensor Mathematics Tensor computation extends linear algebra…
tensor computation precision drift prediction, tensor computation precision, computation precision drift, precision drift prediction, tensor computation, precision drift
Tensor Computation Precision Drift Prediction with Machine Learning Introduction & Motivation Tensor Computation Precision Drift Prediction with Machine Learning addresses…
optimization
Tensor contiguity is the property indicating whether tensor data occupies a compact, stride-consistent memory block - contiguous tensors typically enable faster kernels…
tensor cores, matrix unit, gpu tensor core
A tensor core is a small matrix-multiply-accumulate engine built into each GPU streaming multiprocessor: in one instruction it multiplies two low-precision matrix tiles and adds…
mixed precision math, matrix multiply accumulate mac, nvidia ai accelerator, sparsity tensor core
Tensor Core Architecture represents the revolutionary, highly specialized programmable matrix execution units integrated deep within modern NVIDIA and AMD GPUs, designed…
optimizing tensor operation datapath in ml accelerators, optimizing tensor operation, tensor operation datapath, operation datapath ml, datapath ml accelerators, optimizing tensor
Optimizing Tensor Operation Datapath in ML Accelerators Introduction & Motivation Optimizing Tensor Operation Datapath in ML Accelerators addresses a central problem in systolic…
tensor processing unit, matrix multiply accelerator, systolic array gemm, hardware matrix engine
Tensor/Matrix Multiply Accelerators are the dedicated hardware units (NVIDIA Tensor Cores, Google TPUs, Intel AMX, Apple ANE) that perform dense matrix multiplication operations…
wmma cuda, mma instruction ptx, tensor core utilization, mixed precision tensor cores
Tensor Core Programming is the specialized technique for utilizing dedicated matrix multiplication hardware units on NVIDIA GPUs (Volta, Turing, Ampere, Hopper) that perform 4×4×4…
cuda tensor cores, wmma api, mma instructions, tensor core optimization
Tensor Core Programming is the utilization of specialized matrix multiplication hardware on NVIDIA GPUs to achieve 10-20× higher throughput than CUDA cores — where Tensor Cores…
tensor core yield prediction, tensor core yield, core yield prediction, tensor core, core yield, yield prediction
Tensor Core Yield Prediction with Machine Learning Introduction & Motivation Tensor Core Yield Prediction with Machine Learning addresses a central problem in tensor processing…
ai accelerators predicting tensor core yield and performance uniformity on ai accelerators, ai accelerators predicting, accelerators predicting tensor, predicting tensor core, tensor core yield
AI Accelerators: Predicting tensor core yield and performance uniformity on AI accelerators Introduction & Motivation AI Accelerators: Predicting tensor core yield and performance…
predicting tensor core yield and performance uniformity on ai accelerators, predicting tensor core, tensor core yield, core yield performance, yield performance uniformity, predicting tensor
Predicting tensor core yield and performance uniformity on AI accelerators Introduction & Motivation Predicting tensor core yield and performance uniformity on AI accelerators…
hardware
A tensor core is a small matrix-multiply-accumulate engine built into each GPU streaming multiprocessor: in one instruction it multiplies two low-precision matrix tiles and adds…
matrix multiply accelerator, tensor core gpu
Tensor Cores — specialized hardware units in NVIDIA GPUs that perform matrix-multiply-accumulate (MMA) operations at enormous throughput, designed to accelerate deep learning…
model optimization
Tensor Decomposition is a family of methods that factor high-order tensors into compact components - It compresses multi-dimensional parameter blocks beyond simple matrix…
chemistry ai
Tensor Decomposition (specifically Tensor Network States) is an advanced applied mathematics technique used to compress the exponentially massive, fundamentally uncomputable…
Tensor Decomposition for Compression Introduction & Motivation Tensor Decomposition for Compression addresses a central problem in reducing the size and cost of compression…
Tensor Decomposition for Compression (Extended Study) Introduction & Motivation Tensor Decomposition for Compression (Extended Study) addresses a central problem in reducing…
Tensor Decomposition for Compression (Extended Study) Introduction & Motivation Tensor Decomposition for Compression (Extended Study) addresses a central problem in reducing…
recommendation systems
Tensor Factorization is multi-dimensional factorization that models user-item interactions with additional context dimensions - It extends matrix methods to capture richer…
graph neural networks
Tensor Field Networks (TFN) are the pioneering framework for 3D rotation-equivariant deep learning on point clouds and molecular structures that defines features not as scalars…
scientific ml
Tensor Field Networks (TFN) are the pioneering framework for 3D rotation-equivariant deep learning on point clouds and molecular structures that defines features not as scalars…
tensor format conversion cost and memory layout optimization, tensor format conversion, format conversion cost, conversion cost memory, cost memory layout, memory layout optimization
Tensor Format Conversion Cost and Memory Layout Optimization Introduction & Motivation Tensor Format Conversion Cost and Memory Layout Optimization addresses a central problem…
multimodal ai
Tensor Fusion is a multimodal fusion technique that captures all possible cross-modal interactions by computing the outer product of modality-specific feature vectors — creating…
multidimensional array, tensor shape, tensor rank, tensor dtype, tensor operation, autograd tensor
Tensor is a typed multidimensional array with shape, strides, storage, device, and often automatic-differentiation history. Tensors are the common representation for model…
tensor network methods for simulation, tensor network methods, network methods simulation, tensor network, network methods, methods simulation
Tensor Network Methods for Simulation Introduction & Motivation Tensor Network Methods for Simulation addresses a central problem in quantum-simulation process control…
megatron, column
Tensor parallelism splits individual model layers across multiple GPUs, distributing matrix operations within a single layer rather than assigning entire layers to different…
model training
Tensor parallelism is a model parallelism strategy that splits individual weight tensors (matrices) within a layer across multiple devices, enabling each device to compute…
megatron tensor parallel, layer parallel, intra layer parallelism, model sharding
Tensor parallelism is a model-parallel technique that splits the individual weight matrices inside a layer across multiple GPUs, so each device stores and multiplies only a slice…
megatron tensor parallel, column parallel, row parallel, sequence parallelism attention
Tensor Parallelism for Attention and MLPs is the technique of partitioning individual transformer layer computations (attention heads and MLP matrices) across multiple GPUs so…
megatron tensor parallel, model parallel column row, tensor parallel attention, intra layer parallelism
Tensor Parallelism is the distributed deep learning strategy that partitions individual weight matrices across multiple GPUs within a single layer — splitting the computation…
megatron tensor parallelism, column row parallelism, tensor model parallelism, attention parallelism
Tensor Parallelism is the model parallelism technique that splits individual weight matrices and tensors across multiple GPUs, with each GPU computing a portion of each layer's…
megatron tensor parallel, column row tensor split, tensor parallel attention, 1d 2d tensor parallel
Tensor Parallelism for LLM Training is a sophisticated model parallelism approach that partitions weight matrices across multiple GPUs/TPUs, enabling training…
model parallel sharding strategies, intra-layer tensor splitting, distributed matrix multiplication, megatron style tensor parallel
Tensor Parallelism for Large Models — Distributing individual tensor operations across multiple devices to train and serve models that exceed single-GPU memory capacity. Core…
model parallelism layer, intra layer parallelism, tensor model parallel, column row parallelism
Tensor Parallelism is the model parallelism technique that partitions individual layers across multiple devices by splitting weight matrices along specific dimensions — enabling…
model optimization
Tensor Train is a tensor factorization that decomposes large tensors into a sequence of low-rank core tensors - It controls parameter growth for very high-dimensional weight…
visualize, training
fastai: Making Neural Nets Uncool Again Overview fastai is a deep learning library layered on top of PyTorch. Its goal is to democratize deep learning by making it accessible…
mlops
TensorBoard is the visualization toolkit for inspecting training metrics, model graphs, embeddings, and profiling outputs - it remains a widely used baseline tool for local…
3d vision
TensoRF is the tensor-factorized radiance field representation that decomposes volumetric features for efficient neural rendering - it reduces memory and compute by replacing…
model optimization
TensorFlow Lite is a lightweight TensorFlow runtime for deploying optimized models on mobile and embedded systems - It supports quantization and delegated acceleration for edge…
infrastructure
TensorFlow Profiler is the integrated performance analysis toolkit for diagnosing TensorFlow training and inference bottlenecks - it combines step-time breakdowns, operator…
google, production
TensorFlow Serving is a high-performance, production-grade serving system developed by Google for deploying machine learning models — providing automatic model versioning…
inference optimization
TensorRT Optimization What is TensorRT? NVIDIA TensorRT is an SDK for high-performance deep learning inference. It optimizes models for NVIDIA GPUs, providing significant…
optimization, nvidia
TensorRT is NVIDIA's deep learning inference optimizer and runtime, providing kernel fusion, precision conversion, layer optimization, and hardware-specific tuning to deliver…
deployment
TensorRT is NVIDIA's inference optimizer and runtime: it takes a trained model and compiles it, ahead of time, into a serialized *engine* that runs as fast as possible on one…
trt, nvidia tensorrt, inference optimizer, tensorrt engine, model optimization
TensorRT is NVIDIA's inference optimizer and runtime: it takes a trained model and compiles it, ahead of time, into a serialized *engine* that runs as fast as possible on one…
tensorrt inference optimization and gpu acceleration, tensorrt inference optimization, inference optimization gpu, optimization gpu acceleration, tensorrt inference
TensorRT Inference Optimization and GPU Acceleration Introduction & Motivation TensorRT Inference Optimization and GPU Acceleration addresses a central problem in TensorRT engine…
tensorrt llm, trt-llm, trt llm, tensorrtllm, llm inference nvidia, deployment
TensorRT is NVIDIA's inference optimizer and runtime: it takes a trained model and compiles it, ahead of time, into a serialized *engine* that runs as fast as possible on one…