DeepMind’s language model, which it calls Gopher, is significantly more accurate than these existing ultra-large language models on many tasks, particularly answering questions about specialized subjects like science and the humanities, and equal or nearly equal to them in others, such as logical reasoning and mathematics, according to the data DeepMind published.
This was the case even though Gopher is smaller than some ultra-large language software. Gopher has some 280 billion different parameters or variables that it can tune. That makes it larger than OpenAI’s GPT-3, which has 175 billion. But it is smaller than a system that Microsoft and Nivida collaborated on earlier this year, called Megatron, which has 535 billion, as well as ones constructed by Google, with 1.6 trillion parameters, and Alibaba, with 10 trillion.
DeepMind says its new language model can beat others 25 times its size. With a 7 billion parameter Retro model, it equals the performance of OpenAI’s GPT-3. Also, because researchers can see exactly which section of training text the Retro software is using to produce its output, it could be easier to detect bias or misinformation, the DeepMind researchers said.
Source: https://fortune.com/2021/12/08/deepmind-gopher-nlp-ultra-large-language-model-beats-gpt-3/
Gopher by DeepMind screenshots
