A Review on Language Models as Knowledge Bases
seen from China
seen from United States

seen from France

seen from United States

seen from Malaysia

seen from United States

seen from Chile
seen from China

seen from Malaysia
seen from Chile
seen from Hong Kong SAR China
seen from Russia

seen from United States
seen from China
seen from France

seen from United States

seen from United States
seen from Türkiye
seen from United States
seen from China
A Review on Language Models as Knowledge Bases
Contents of GPT-3 & the Pile v1
The Paradigm Shift of Foundation Models
This is a paradigm shift because AI models in the past were often thought to perform one task at a time. While techniques in task-transfer and multi-task AI models were researched, one model was not expected to perform tasks that are very different from one another, even with transfer learning. This also meant that research in different tasks and modalities, like vision and language, were often done with different technical approaches — vision researchers studied topics like signal processing, while language researchers studied language grammar. Foundation models with deep learning leveraged incredible scaling to train one model that can perform many tasks.
Language model sizes to Apr. 2022 or While you were sleeping, AI sizes were exploding