Comprehensive List of Open-Source Language Models and Chinese LLMs

The Open-Source Language Model Pocket is a curated list of open-source language models, with a strong emphasis on models that are either Chinese-friendly or primarily developed by Chinese teams. This resource aims to provide a comprehensive overview of available models, covering a wide range of applications and domains. It serves as a valuable tool for researchers, developers, and enthusiasts looking to explore and utilize open-source language models for various projects. This pocket guide is continuously updated to reflect the rapidly evolving landscape of AI and language models.

This section highlights general-purpose language models that are either Chinese-friendly or developed by Chinese teams. These models are designed to handle a wide variety of tasks and are suitable for general applications. Examples include Baichuan, Chinese LLaMA & Alpaca, Tongyi Qianwen Qwen, and many others. These models often support both Chinese and English languages and are trained on large datasets to achieve broad capabilities. The list also includes models like ChatGLM, Skywork, and Yi-6B/34B, showcasing the diversity and innovation in the Chinese open-source community. Models such as Qwen1.5 and DeepSeek LLM represent the cutting edge, offering enhanced performance and capabilities for various natural language processing tasks.

This section focuses on language models specifically designed for healthcare and medical applications. These models are trained on medical knowledge and data to provide accurate and reliable information in the medical domain. Examples include BenCao, HuaTuo, BianQue, and Mingyi (MING). These models are capable of performing tasks such as medical question answering, diagnosis assistance, and medical text generation. The section also includes models like DoctorGLM and ChatMed, which are designed for specialized medical consultations. The inclusion of models like Llama-3-8B-UltraMedical and ProLLM highlights the ongoing advancements in this critical area.

This section lists language models tailored for finance and economic applications. These models are trained on financial data and are designed to understand and process financial language and concepts. Examples include PIXIU FinMA, XuanYuan, and FinGLM. These models can be used for tasks such as financial analysis, risk assessment, and economic forecasting. The development of models like Deepmoney and Cornucopia-LLaMA-Fin-Chinese demonstrates the growing interest in applying LLMs to the financial sector.

This section features language models designed for legal applications. These models are trained on legal texts and are capable of understanding and processing legal language. Examples include HanFei, Zhihai Luwen, and ChatLaw. These models can assist with tasks such as legal research, contract analysis, and legal document generation. The inclusion of models like LaWGPT and Lawyer LLaMA underscores the importance of specialized LLMs in the legal field.

This section highlights language models focused on education and mathematics. These models are trained on educational materials and mathematical data to assist with learning and problem-solving. Examples include TaoLi, EduChat, and InternLM-Math. These models can be used for tasks such as tutoring, homework assistance, and mathematical reasoning. The development of models like DeepSeekMath and Qwen2-Math reflects the increasing demand for AI-powered educational tools.

This section lists language models designed for code and programming-related tasks. These models are trained on code repositories and programming documentation to assist with code generation, debugging, and software development. Examples include CodeShell, DeepSeek Coder, and Magicoder. These models can be used for tasks such as code completion, bug detection, and code translation. Models like CodeQwen1.5 and CodeGemma showcase the advancements in AI-assisted coding.

This section includes a variety of other open-source models that are notable for their specific applications or unique features. These models cover a wide range of domains, including transportation (TransGPT), self-media (MediaGPT), and ancient Chinese language (Erya). This section also includes models developed outside of China, such as Cerebras, MPT-7B, and Dolly 1&2, providing a broader perspective on the open-source language model landscape. Models like Mistral 7B and Llama 3 represent significant contributions to the field.

This section provides resources and tools for training and inference of language models. It includes frameworks and techniques such as Alpaca-LoRA, ColossalAI, and DeepSpeed-Chat. These resources help developers fine-tune and deploy language models efficiently. The section also covers methods like DPO (Direct Preference Optimization) and QLoRA, which are used to improve model performance and reduce computational costs. Tools like llama.cpp and vLLM are also listed for optimized inference.

This section lists evaluation benchmarks used to assess the performance of language models. These benchmarks provide standardized metrics for evaluating models on various tasks. Examples include FlagEval, C-Eval, and HaluEval. These benchmarks help researchers and developers compare different models and track progress in the field. The section also includes benchmarks like CMB (Comprehensive Medical Benchmark in Chinese) and Fin-Eva, which are designed for specific domains.