Apple researchers introduce MemoryLLM for interpretable feed-forward memory in transformers.

MemoryLLM decouples feed-forward modules from self-attention, enabling study of FFN parameters as context-free token-wise neural retrieval memory and revealing how input tokens access memory locations across downstream tasks.