Gopher is a 280 billion parameter transformer language model developed by DeepMind for predicting and generating text.
Gopher is a large-scale transformer language model, part of a series of models developed by DeepMind ranging from 44 million to 280 billion parameters. This research investigates the strengths and weaknesses of these models, showing performance boosts with increased scale in areas like reading comprehension, fact-checking, and identifying toxic language. Gopher demonstrates significant advancement towards human expert performance on the Massive Multitask Language Understanding (MMLU) benchmark. DeepMind's research into Gopher also explored its capabilities through direct interaction, noting its surprising coherence in dialogue interactions, even providing correct citations without specific fine-tuning. However, the research also detailed persistent failure modes across model sizes, including repetition, reflection of stereotypical biases, and confident propagation of incorrect information. This analysis is crucial for understanding potential downstream harms and focusing mitigation efforts. Alongside Gopher, DeepMind released papers on ethical and social risks of large language models, presenting a taxonomy of 21 risks across six thematic areas. They also introduced the Retrieval-Enhanced Transformer (RETRO), an improved architecture that reduces training energy costs and enhances traceability of model outputs by using an Internet-scale retrieval mechanism, achieving comparable performance with fewer parameters.
Platforms: Web
View full Gopher profile on Tools-Radar | Browse Text & Writing tools | Alternatives to Gopher
Tools-Radar is a free directory of 10,000+ AI tools — discover, compare, and choose the right AI software for your needs. Visit tools-radar.com