A Review on Language Models as Knowledge Bases

seen from China

seen from Malaysia
seen from United States
seen from United Kingdom

seen from Germany
seen from Türkiye

seen from Canada
seen from Türkiye
seen from United States
seen from United Kingdom

seen from Canada
seen from Malaysia

seen from Türkiye

seen from Türkiye

seen from Malaysia
seen from Malaysia

seen from Türkiye

seen from China
seen from South Africa

seen from Malaysia
A Review on Language Models as Knowledge Bases
Contents of GPT-3 & the Pile v1
The Paradigm Shift of Foundation Models
This is a paradigm shift because AI models in the past were often thought to perform one task at a time. While techniques in task-transfer and multi-task AI models were researched, one model was not expected to perform tasks that are very different from one another, even with transfer learning. This also meant that research in different tasks and modalities, like vision and language, were often done with different technical approaches — vision researchers studied topics like signal processing, while language researchers studied language grammar. Foundation models with deep learning leveraged incredible scaling to train one model that can perform many tasks.
Language model sizes to Apr. 2022 or While you were sleeping, AI sizes were exploding