Token Merging

Reduce Vision Transformer Compute with Token Merging

Reduce Vision Transformer Compute with Token Merging Vision transformers can spend substantial computation processing many patch tokens that carry similar information. A patch covering one part of a clear sky may produce a representation close to nearby sky patches, yet ordinary self-attention continues to process each token separately. Token merging reduces that redundancy by combining selected tokens as they move through the network. Unlike token pruning, which removes tokens, merging tries to preserve their information in a smaller set of representations. The practical goal is simple: reduce the token count in later transformer blocks while keeping task quality within an acceptable range.