8 дек. 2021 г. · We enhance auto-regressive language models by conditioning on document chunks retrieved from a large corpus, based on local similarity with preceding tokens.

4 мая 2021 г. · We enhance auto-regressive language models by conditioning on document chunks retrieved from a large corpus, based on local similarity with pre-.

A comprehensive study on a scalable pre-trained retrieval-augmented LM of RETRO, which outperforms GPT on text generation with much less degeneration.

8 дек. 2021 г. · We explore an alternate path for improving language models: we augment transformers with retrieval over a database of text passages ...

A retrieval-enhanced model (RETRO) combining a medium-sized transformer LM (25x less parameters than GPT-3), a 2-trillion token database, and a frozen BERT ...

Neural networks have proven to be powerful language models, first in the form of recurrent architectures. (Graves, 2013; Jozefowicz et al., 2016; Mikolov et al.

8 дек. 2021 г. · We enhance auto-regressive language models by conditioning on document chunks retrieved from a large corpus, based on local similarity with preceding tokens.

12 июл. 2022 г. · Bibliographic details on Improving Language Models by Retrieving from Trillions of Tokens.

4 янв. 2024 г. · Scaling the training data to trillions of tokens improves the performance of language models in machine translation and downstream tasks. Later ...

Продолжительность: 14:30
Опубликовано: 8 июл. 2023 г.