GLM-130B

Categories: Coding & Developer Tools, Text & Writing, Research | Pricing: Free | Official Website ↗

GLM-130B is an open-source, bilingual (English & Chinese) bidirectional dense language model with 130 billion parameters.

GLM-130B is a large-scale, open-source language model developed by Tsinghua KEG. It features 130 billion parameters and is pre-trained on over 400 billion text tokens, split equally between English and Chinese. The model is based on the General Language Model (GLM) algorithm and is designed for efficient inference on a single A100 or V100 server. The model demonstrates strong performance in both English and Chinese language understanding and generation tasks, outperforming models like GPT-3 175B, OPT-175B, and BLOOM-176B on various benchmarks. It supports fast inference through SAT and FasterTransformer, and its training and inference capabilities are cross-platform, including NVIDIA, Hygon DCU, Ascend 910, and Sunway. The project aims to make highly accurate, bilingual language models accessible globally.

Key Features

130 billion parameters
Bilingual (English & Chinese) support
Bidirectional dense model
Pre-trained on 400B+ text tokens
Fast inference on single A100/V100 server
Reproducible results with open-sourced code
Cross-platform compatibility (NVIDIA, Hygon DCU, Ascend 910, Sunway)
Autoregressive blanking infilling objective

Pros

Open-source code and model checkpoints available
Supports both English and Chinese languages
High performance compared to other large models (GPT-3, OPT, BLOOM, ERNIE TITAN)
Fast inference with low resource requirements (single A100/V100 server)
Cross-platform compatibility for training and inference

Cons

Requires significant GPU memory (260G for FP16) for full model weights
Optimal training tokens (4.0T) are 10x larger than current training (400B)
API not explicitly mentioned as available
Still under active development with future work planned
Initial setup and deployment may require technical expertise

Use Cases

Natural language understanding
Text generation
Zero-shot learning
Cross-lingual transfer tasks
Research and development of large language models

Best For

AI researchers
Machine learning engineers
Developers building NLP applications
Academics studying large language models
Organizations requiring bilingual language models

Platforms: Web

Watch demo on YouTube ↗

View full GLM-130B profile on Tools-Radar | Browse Coding & Developer Tools tools | Alternatives to GLM-130B

Tools-Radar is a free directory of 10,000+ AI tools — discover, compare, and choose the right AI software for your needs. Visit tools-radar.com