Transformer
Definition
The Transformer is a deep learning architecture based on self-attention, introduced by Google in 2017. It revolutionized natural language processing and serves as the foundation for large models like GPT and BERT. Its core advantage lies in its ability to process sequence data in parallel and capture long-range dependencies.
Usage Example
OpenAI's GPT series and Google's BERT are both built on the Transformer architecture.
Related Terms
Related terminology
Large Language Model
A deep learning language model trained on large-scale corpora, capable of text understanding, generation, and reasoning, with parameter scales typically ranging from billions to hundreds of billions.
Fine-tune
A technique that fine-tunes a pre-trained model using domain-specific data to adapt it for downstream tasks.
GPT
GPT (Generative Pre-trained Transformer) is a series of generative pre-trained language models developed by OpenAI. Trained via unsupervised learning on large-scale text corpora, GPT can generate coherent, high-quality text for applications such as conversation, writing, and coding.