自注意力和 CNN 的结合 ACmix : On the Integration of Self-Attention and Convolution
OntheIntegrationofSelf-AttentionandConvolutionFigure1.AsketchofACmix.Weexploreacloserrelationshipbetweenconvolutionandself-attentioninthesenseofsharingthesamecomputationoverhead(1×1convolutions),andco