New GAP architecture merges attention and MLP into one block
A researcher has introduced Gated Attention Projection (GAP), a Transformer alternative that reduces parameter counts by nearly 20% through architectural.
Try in Readertransformer architecturemachine learninggated attention projectionai research