Baichuan-13B-Chat

介绍

Baichuan-13B-Chat是Baichuan-13B系列模型中对齐后的版本，预训练模型可见 Baichuan-13B-Base 。

Baichuan-13B 是由百川智能继 Baichuan-7B 之后开发的包含130亿参数的开源可商用的大规模语言模型，在权威的中文和英文基准测试中取得了最佳效果。本次发布包含有预训练 ( Baichuan-13B-Base ) 和对齐 ( Baichuan-13B-Chat ) 两个版本。Baichuan-13B具有以下几个特点：

Baichuan-13B-Chat是Baichuan-13B系列模型中对齐后的版本，预训练模型可见 Baichuan-13B-Base 。

Baichuan-13B 是由百川智能继 Baichuan-7B 之后开发的包含130亿参数的开源可商用的大规模语言模型，在权威的中文和英文基准测试中取得了最佳效果。本次发布包含有预训练 ( Baichuan-13B-Base ) 和对齐 ( Baichuan-13B-Chat ) 两个版本。Baichuan-13B具有以下几个特点：

使用方式

以下是一个使用Baichuan-13B-Chat进行对话的示例，正确输出为"乔戈里峰。世界第二高峰———乔戈里峰西方登山者称其为k2峰，海拔高度是8611米，位于喀喇昆仑山脉的中巴边境上"

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation.utils import GenerationConfig
tokenizer = AutoTokenizer.from_pretrained("baichuan-inc/Baichuan-13B-Chat", use_fast=False, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("baichuan-inc/Baichuan-13B-Chat", device_map="auto", torch_dtype=torch.float16, trust_remote_code=True)
model.generation_config = GenerationConfig.from_pretrained("baichuan-inc/Baichuan-13B-Chat")
messages = []
messages.append({"role": "user", "content": "世界上第二高的山峰是哪座"})
response = model.chat(tokenizer, messages)
print(response)

Here is an example of a conversation using Baichuan-13B-Chat, the correct output is "K2. The world's second highest peak - K2, also known as Mount Godwin-Austen or Chhogori, with an altitude of 8611 meters, is located on the China-Pakistan border in the Karakoram Range."

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation.utils import GenerationConfig
tokenizer = AutoTokenizer.from_pretrained("baichuan-inc/Baichuan-13B-Chat", use_fast=False, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("baichuan-inc/Baichuan-13B-Chat", device_map="auto", torch_dtype=torch.float16, trust_remote_code=True)
model.generation_config = GenerationConfig.from_pretrained("baichuan-inc/Baichuan-13B-Chat")
messages = []
messages.append({"role": "user", "content": "Which moutain is the second highest one in the world?"})
response = model.chat(tokenizer, messages)
print(response)

量化部署

Baichuan-13B支持int8和int4量化，用户只需在推理代码中简单修改两行即可实现。请注意，如果是为了节省显存而进行量化，应加载原始精度模型到CPU后再开始量化；避免在from_pretrained时添加device_map='auto'或者其它会导致把原始精度模型直接加载到GPU的行为的参数。

使用int8量化 (To use int8 quantization):

model = AutoModelForCausalLM.from_pretrained("baichuan-inc/Baichuan-13B-Chat", torch_dtype=torch.float16, trust_remote_code=True)
model = model.quantize(8).cuda() 

同样的，如需使用int4量化 (Similarly, to use int4 quantization):

model = AutoModelForCausalLM.from_pretrained("baichuan-inc/Baichuan-13B-Chat", torch_dtype=torch.float16, trust_remote_code=True)
model = model.quantize(4).cuda()

模型详情

模型描述

• Developed by: 百川智能(Baichuan Intelligent Technology)

• Email : opensource@baichuan-inc.com

• Language(s) (NLP): Chinese/English

• License: 【Community License for Baichuan-13B Model】(ZH | EN)

  商业用途(For commercial use): 请通过Email联系申请书面授权。(Contact us via Email above to apply for written authorization.)

模型结构

整体模型基于Baichuan-7B，为了获得更好的推理性能，Baichuan-13B使用了ALiBi线性偏置技术，相对于Rotary Embedding计算量更小，对推理性能有显著提升；与标准的LLaMA-13B相比，生成2000个tokens的平均推理速度(tokens/s)，实测提升31.6%：

具体参数和见下表

The overall model is based on Baichuan-7B. In order to achieve better inference performance, Baichuan-13B uses ALiBi linear bias technology, which has a smaller computational load compared to Rotary Embedding, and significantly improves inference performance. Compared with the standard LLaMA-13B, the average inference speed (tokens/s) for generating 2000 tokens has been tested to increase by 31.6%:

The specific parameters are as follows:

使用须知

免责声明

我们在此声明，我们的开发团队并未基于Baichuan-13B模型开发任何应用，无论是在iOS、Android、网页或任何其他平台。我们强烈呼吁所有使用者，不要利用Baichuan-13B模型进行任何危害国家社会安全或违法的活动。另外，我们也要求使用者不要将Baichuan-13B模型用于未经适当安全审查和备案的互联网服务。我们希望所有的使用者都能遵守这个原则，确保科技的发展能在规范和合法的环境下进行。

我们已经尽我们所能，来确保模型训练过程中使用的数据的合规性。然而，尽管我们已经做出了巨大的努力，但由于模型和数据的复杂性，仍有可能存在一些无法预见的问题。因此，如果由于使用Baichuan-13B开源模型而导致的任何问题，包括但不限于数据安全问题、公共舆论风险，或模型被误导、滥用、传播或不当利用所带来的任何风险和问题，我们将不承担任何责任。

We hereby declare that our development team has not developed any applications based on the Baichuan-13B model, whether on iOS, Android, the web, or any other platform. We strongly urge all users not to use the Baichuan-13B model for any activities that harm national social security or are illegal. In addition, we also ask users not to use the Baichuan-13B model for internet services that have not undergone appropriate security review and filing. We hope that all users will adhere to this principle to ensure that technological development takes place in a regulated and legal environment.

We have done our utmost to ensure the compliance of the data used in the model training process. However, despite our great efforts, due to the complexity of the model and data, there may still be some unforeseen issues. Therefore, we will not take any responsibility for any issues arising from the use of the Baichuan-13B open-source model, including but not limited to data security issues, public opinion risks, or any risks and problems arising from the model being misled, misused, disseminated, or improperly exploited.

训练详情

训练具体设置参见 Baichuan-13B 。

For specific training settings, please refer to Baichuan-13B .

测评结果

C-Eval

MMLU

说明：我们采用了MMLU官方的 评测方案 。

CMMLU

说明：CMMLU是一个综合性的中文评估基准，专门用于评估语言模型在中文语境下的知识和推理能力。我们采用了其官方的 评测方案 。

微信群组