#효율·추론 비용

Designing efficient quantum circuits

FlashMask-3: Efficient and Expressive Mask-Aware Distributed Attention

Long-context training of large language and multimodal models couples structured attention masks (document, prefix-LM, share-question, sliding-window, block-sparse) with multi-node context parallelism, making the mask a dominant bottleneck.

Copy-Paste Language Models

Transformers are intuitively considered to be efficient at copying from their inputs. We actually show that the ability of LLMs to copy in context can be improved through explicit modeling.