Transformer
A neural network architecture based on self-attention mechanisms that powers modern language models.
Introduced in the 2017 paper "Attention Is All You Need," transformers process input sequences in parallel rather than sequentially. The self-attention mechanism allows every token to attend to every other token, capturing long-range dependencies in text.
Transformers are the foundation of every major LLM. Their parallelizable architecture enables training on massive datasets and scaling to billions of parameters, which is what gives modern AI systems their remarkable language understanding capabilities.
Related Terms
More ai/ml Terms
Retrieval-Augmented Generation (RAG)
An AI architecture that combines information retrieval with text generation to produce answers grounded in source documents.
Vector Embedding
A numerical representation of text as a high-dimensional vector, enabling semantic similarity comparisons between passages.
BM25
A probabilistic keyword-ranking algorithm that scores documents by term frequency and inverse document frequency.
Chunking
The process of splitting large documents into smaller, overlapping segments optimized for retrieval and embedding.
Hallucination
When an AI model generates plausible-sounding but factually incorrect or fabricated information.
Large Language Model (LLM)
A neural network trained on massive text corpora that can understand and generate human language.
Analyze Documents Related to Transformer
Upload any document and get AI-powered analysis with verifiable citations.
Start Free