Back to search
Top Contributors
75
commits
10
commits
[ICML'25] Our study systematically investigates massive values in LLMs' attention mechanisms. First, we observe massive values are concentrated in low-frequency dimensions across different attention heads exclusively in attention queries (Q) and keys (K) while absent in values (V).
Stars
86
Forks
2
Watchers
86
Open Issues
0
Overall repository health assessment
No package.json found
This might not be a Node.js project
75
commits
10
commits