Ollama是什么？

Ollama 使你能够获取开源模型以便本地使用。它会自动从最佳来源获取模型，并且，如果你的电脑有专用的GPU，它会无缝地利用GPU加速而无需手动配置。通过修改提示（prompt）可以轻松地自定义模型，而且不需要Langchain作为前提条件。此外，你可以将Ollama作为一个docker镜像来访问，让你能够将你个性化的模型部署为一个docker容器。

Ollama是如何工作的？

• Ollama允许你在本地运行开源的大型语言模型，例如Llama 2，Mistral等。
• Ollama将模型权重、配置和数据打包成一个单一的包，由一个Modelfile定义。
• 它优化了安装和配置的细节，包括GPU的使用。

如何安装Ollama？

目前Ollama仅支持MacOS和Linux。对于Windows用户，我们可以通过WSL2安装Ollama。

在WSL2中安装Ollama：

curl https://ollama.ai/install.sh | sh

启动Ollama服务器

ollama serve

运行所需的模型

在这个示例中，我们选择了作为Mistral的LLM。打开另一个终端，并通过执行以下命令下载你选择的模型。

ollama run mistral

模型下载成功后，我们可以与模型进行交互进行推理。我们现在已经安装了Ollama，可以用它来实现RAG。

Chainlit是什么？

Chainlit是一个开源的Python包，能够极快地构建包含你自己的业务逻辑和数据的类似Chat GPT的应用。

主要特性：

• 快速构建：可以迅速地融入现有代码基础，或者从头开始在几分钟内开始开发。
• 数据连续性：利用用户生成的数据和反馈来增强性能。
• 可视化复杂推理：快速总览中间步骤，了解特定结果的产生过程。
• 提示优化：深入挖掘Prompt Playground内的提示，以发现问题并迭代改进。

可用的集成项：

• Langchain
• Haystack
• LLama-Index
• Langflow

在这里我们将使用Langchain框架实施RAG。

Langchain是什么？

Langchain是一个免费提供的框架，旨在简化大型语言模型（LLMs）应用程序的开发。它提供了一种针对链的标准化界面、与各种工具的广泛集成，以及为常见应用定制的完整链条。这帮助AI开发者创建应用程序，利用大型语言模型（如GPT-4）的集体力量，结合外部计算和数据源。该框架包含Python和JavaScript两种语言的软件包。

VectorStore是什么？

向量存储是特别设计用来高效存储和检索向量嵌入的数据库。它们之所以重要，是因为传统数据库，如SQL，并不是为存储和检索大量向量数据而精心调优的。

嵌入表示的是数据，通常是以数值向量形式呈现在高维空间中的非结构化数据，如文本。传统关系数据库不适合存储和检索这些向量表示。

向量存储有能力索引并快速搜索相似的向量，使用相似度算法。这项功能使应用能够基于提供的目标向量查询，识别相关的向量。

Chroma DB是什么？

Chroma DB是一个开源的向量存储，用于存储和检索向量嵌入。它的主要用途是保存嵌入及其元数据，以便稍后由大型语言模型使用。此外，它还可以用于文本数据上的语义搜索引擎。

主要特性：

• 支持不同的底层存储选项，如DuckDB用于独立或ClickHouse用于可扩展性。
• 提供适用于Python和JavaScript/TypeScript的SDKs。
• 专注于简单性、速度和支持分析。

嵌入

GPT4All 支持使用针对 CPU 优化的对比训练的句子转换器生成任意长度文档文本的高质量嵌入表示。这些嵌入表示在许多任务中的质量与OpenAI相当。

嵌入生成速度

下表列出了在稳定负载下使用 8 线程在配备 DDR5 5600 的 Intel i9-13900HX CPU 上捕获的文本文档的生成速度。

RAG是什么？

检索增强型生成（RAG）是一个用来提高大型语言模型（LLMs）生成答案准确性的人工智能框架。该框架通过整合外部知识源来补充LLM内部的信息表述。当将RAG应用于基于LLM的问答系统时，它提供了两个主要优势。首先，它确保模型可以访问最新和最可靠的事实。其次，它让用户可以看到模型的信息来源，使他们能够验证答案的准确性并建立对生成响应的信任。它包括两个功能：

1. 检索：从Vectorstore中检索与查询最匹配的上下文
2. 生成：基于检索到的上下文构建一个回答

RAG管道实施步骤：

1. 阅读PDF文件。
2. 考虑将一个大文档拆分成较小的部分或块，以确保模型在接收针对特定问题的相关信息时，不会因为资源过多而过载。这种方法类似于仅提供模型所需的片段，而不是一次性给予全部内容，考虑到每个模型都有令牌限制。
3. 将这些块转换为向量嵌入，以增强模型对数据的理解。然后，创建一个向量存储，以便高效存储和检索这些嵌入，使用FAISS向量存储来实现这一目的。
4. 有了向量存储，接下来使用Langchain的RetrievalQA来查询PDF文件。这涉及到使用向量存储作为检索器，并指定要使用的模型，以及根据具体需求调整其他参数。

代码实施

实施技术栈

• Chromadb — 向量存储
• gpt4all — 文本嵌入
• langchain — 促进使用LLM进行应用开发的框架
• chainlit — 构建类似ChatGPT的界面

文件夹结构

• 数据文件夹：以 .pdf 格式存储所需的文件
• Vectorstore/db：存储数据嵌入

安装所需的依赖项：

pip install chromadb
pip install langchain
pip install BeautifulSoup4
pip install gpt4all
pip install langchainhub
pip install pypdf
pip install chainlit

上传所需数据并加载进VectorStore中

• 执行以下脚本以将文档转换为嵌入式向量并储存到chromadb中
  
• python3 load_data_vdb.py

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.document_loaders import PyPDFLoader, DirectoryLoader
from langchain.document_loaders.pdf import PyPDFDirectoryLoader
from langchain.document_loaders import UnstructuredHTMLLoader, BSHTMLLoader
from langchain.vectorstores import Chroma
from langchain.embeddings import GPT4AllEmbeddings
from langchain.embeddings import OllamaEmbeddings  
import os
DATA_PATH="data/"
DB_PATH = "vectorstores/db/"
def create_vector_db():
    loader = PyPDFDirectoryLoader(DATA_PATH)
    documents = loader.load()
    print(f"Processed {len(documents)} pdf files")
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=50)
    texts=text_splitter.split_documents(documents)
    vectorstore = Chroma.from_documents(documents=texts, embedding=GPT4AllEmbeddings(),persist_directory=DB_PATH)      
    vectorstore.persist()
if __name__=="__main__":
    create_vector_db()

矢量存储被创建在vectorstores/db文件夹中。

运行聊天机器人应用程序以生成回应

• 使用chainlit创建用户界面。
• 接受用户查询
• 参考向量存储
• 检索匹配的上下文
• 将匹配的上下文传递给LLM
• 生成回应

执行以下命令：

chainlit run RAG.py

代码RAG.py

#import required dependencies
from langchain import hub
from langchain.embeddings import GPT4AllEmbeddings
from langchain.vectorstores import Chroma
from langchain.llms import Ollama
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
import chainlit as cl
from langchain.chains import RetrievalQA,RetrievalQAWithSourcesChain
# Set up RetrievelQA model
QA_CHAIN_PROMPT = hub.pull("rlm/rag-prompt-mistral")
#load the LLM
def load_llm():
 llm = Ollama(
 model="mistral",
 verbose=True,
 callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),
 )
 return llm

def retrieval_qa_chain(llm,vectorstore):
 qa_chain = RetrievalQA.from_chain_type(
    llm,
    retriever=vectorstore.as_retriever(),
    chain_type_kwargs={"prompt": QA_CHAIN_PROMPT},
    return_source_documents=True,
)
 return qa_chain

def qa_bot(): 
 llm=load_llm() 
 DB_PATH = "vectorstores/db/"
 vectorstore = Chroma(persist_directory=DB_PATH, embedding_function=GPT4AllEmbeddings())
 qa = retrieval_qa_chain(llm,vectorstore)
 return qa 
@cl.on_chat_start
async def start():
 chain=qa_bot()
 msg=cl.Message(content="Firing up the research info bot...")
 await msg.send()
 msg.content= "Hi, welcome to research info bot. What is your query?"
 await msg.update()
 cl.user_session.set("chain",chain)
@cl.on_message
async def main(message):
 chain=cl.user_session.get("chain")
 cb = cl.AsyncLangchainCallbackHandler(
 stream_final_answer=True,
 answer_prefix_tokens=["FINAL", "ANSWER"]
 )
 cb.answer_reached=True
 # res=await chain.acall(message, callbacks=[cb])
 res=await chain.acall(message.content, callbacks=[cb])
 print(f"response: {res}")
 answer=res["result"]
 answer=answer.replace(".",".\n")
 sources=res["source_documents"]
 if sources:
  answer+=f"\nSources: "+str(str(sources))
 else:
  answer+=f"\nNo Sources found"
 await cl.Message(content=answer).send() 

Chainlit界面快照

• 应用接口：http://localhost:8000

response: {'query': 'What is ESOPS?', 
'result': '\nESOPs are options given by a company to its employees to buy shares in their company at a predetermined price within a prescribed time period. The purchase price may or may not be at a discount to the ruling/ fair market value of the shares of the company. ESOPs are increasingly being accepted as a reward for employee productivity and have been recognised worldwide as an important tool for employee wealth sharing and motivation. They create a sense of ownership in employees and make them feel part of the organisation.', 'source_documents': [Document(page_content='16\nESOP S\n16.1 Meaning\nEmployee Stock Option Plans or ESOPs are increasingly being accepted as a\nreward for Employee Productivity. Earlier, the use of ESOPs was restricted to\nknowledge -based companies only but now they have spread across the entire\nspectrum o f industries. ESOPs have been recognised the world over as an important\ntool for Employee Wealth Sharing and Employee Motivation. ESOPs create a sense\nof ownership in the employees in relation to their employer company and they feel\nthat they are part of t he organisation.  Companies such as Infosys, Wipro, etc., have\nmade several millionaires out of their employees due to ESOPs.\nSimply put, an ESOP is an option (but not an obligation) given by a company to its\nemployees to buy shares in their company at a predetermined price within a\nprescribed time period. The purchase price may or may not be at a discount to the\nruling/ fair market value of the shares of the Company.', metadata={'page': 0, 'source': 'data/PDFFile5b28ce3c2eb412.05300945.pdf'}), Document(page_content='ESOPs 81\noptions to the trust and the trust then allocates the options as per the\nguidelines of the ESOP.\n16.2 Regulatory framework\n16.2.1 The primary legislation which concerns an ESOP is the Companies Act.\nEvery company which wants to issue an ESOP would need to comply with the\nprovisions of the Act. Depending upon whether the company issuing the ESOPs is a\nlisted company or an unlist ed company the applicable laws would vary. Listed\ncompanies are subjected to the SEBI Guidelines on ESOPs. In addition, in order that\nthese ESOPs remain tax neutral in the hands of the employees, the ESOPs should\nalso comply with the Guidelines issued by t he Central Government u/s. 17(2)(iii) of\nthe Income -tax Act, 1961. Hence, any ESOP by a listed company would be\nsubjected to both these legislations. In case of an unlisted public limited company,\nthe Unlisted Public Companies (Preferential Allotment) Rule s would apply along with', metadata={'page': 1, 'source': 'data/PDFFile5b28ce3c2eb412.05300945.pdf'}), Document(page_content='ESOPs 85\nindirectly, is more than 51% may pur chase Equity shares of foreign company.\nThe ESOP may be offered directly by the issuing company or through a trust/\na Special Purpose Vehicle/ a subsidiary.\n(d) JV/ WOS abroad\nAn Indian company’s domestic employees and directors may purchase the\nshares of the promoter company’s Joint Venture or Wholly Owned Subsidiary\nabroa d in the field of software, if:\n\uf0b7the maximum purchase consideration does not exceed the ceiling\nstipulated by the RBI from time to time (US$10,000 per employee in a\nblock of five cale ndar years);\n\uf0b7the Shares do not exceed 5% of the paid -up capital of the issuing\nCompany; and\n\uf0b7the post -allotment holding of the Indian  promoter company and the\nshares held by the employees must not be lower than the pre -allotment\nholding of the Indian promoter company .\n(e) ADR linked ESOPs\nResident employees and working directors of Indian Companies operating in\nknowledge based sectors can purchase foreign securities under the ADR/', 
metadata={'page': 5, 'source': 'data/PDFFile5b28ce3c2eb412.05300945.pdf'}), 
Document(page_content='disclosures.\n(e) The percentage of Equity Capital set aside for the ESOP depends entirely on\nthe company. There is no statutory requirement for this and it varies from\ncompany to company.', 
metadata={'page': 2, 'source': 'data/PDFFile5b28ce3c2eb412.05300945.pdf'})]}

查询2

response: {'query': 'What  has been mentioned about the companies act ?', 
'result': "The Companies Act is the primary legislation that concerns an Employee Stock Ownership Plan (ESOP). Listed companies are subjected to SEBI Guidelines on ESOPs and the Central Government's Income-tax Act, 1961. Unlisted public limited companies are governed by the Preferential Allotment Rules, 2003. Private limited companies are only subjected to the Income-tax Guidelines.", 
'source_documents': [Document(page_content='ESOPs 81\noptions to the trust and the trust then allocates the options as per the\nguidelines of the ESOP.\n16.2 Regulatory framework\n16.2.1 The primary legislation which concerns an ESOP is the Companies Act.\nEvery company which wants to issue an ESOP would need to comply with the\nprovisions of the Act. Depending upon whether the company issuing the ESOPs is a\nlisted company or an unlist ed company the applicable laws would vary. Listed\ncompanies are subjected to the SEBI Guidelines on ESOPs. In addition, in order that\nthese ESOPs remain tax neutral in the hands of the employees, the ESOPs should\nalso comply with the Guidelines issued by t he Central Government u/s. 17(2)(iii) of\nthe Income -tax Act, 1961. Hence, any ESOP by a listed company would be\nsubjected to both these legislations. In case of an unlisted public limited company,\nthe Unlisted Public Companies (Preferential Allotment) Rule s would apply along with', 
metadata={'page': 1, 'source': 'data/PDFFile5b28ce3c2eb412.05300945.pdf'}), 
Document(page_content='whether the company is a listed company or an unlisted company.  Further, s. 77 of\nthe Act prohibits a public company from directly or indirectly providing any financial\nassistance for the purpose of or in connection with a purchase of any shares in such\ncompany or in its holding company.  However, this prohibition does not apply if, in\naccordance with any scheme for the time being in force, the company has provided\na loan to a Trust for purchase of such shares to be held for the benefit of employees\nincluding working directors. Thus, if the company provides a loan to an ESOP Trust\nfor acqui ring shares under the ESOP and distributing them to the employees, then\ns.77 would not apply. S. 77 also permits the company to give a loan directly to the\nemployees for the purchase/ subscription of its shares. However, such loan is\nrestricted to a maximu m of 6 months’ salary of the employee.  It may be noted that', 
metadata={'page': 1, 'source': 'data/PDFFile5b28ce3c2eb412.05300945.pdf'}), 
Document(page_content='Hand Book o n Capital Market Regulations 84\n16.2.4 Unlisted Company Rules\nThe Preferential Allotment Ru les, 2003 are applicable to a preferential issue u/s.\n81(1A) of the Companies Act by unlisted public companies. The Rules do not\nprovide an exemption for an ESOP issue and hence, they would need to be\ncomplied with by an unlisted public company instituting  and ESOP. The main\nrequirements of these Rules are as follows:\n\uf0b7A Special Resolution of the shareholders is required\n\uf0b7If the issue consists of convertible warrants, then the price of the resultant\nshare s must be determined beforehand\n\uf0b7Several discl osures are required in the Explanatory Statement to the Notice,\nincluding the price or price band at which the allotment is proposed\n\uf0b7The relevant date on the basis of which the price has been arrived at must be\nstated\n\uf0b7The shareholding pattern pre and p ost issue must be stated\n\uf0b7The Special Resolution must be acted upon within 12 months', 
metadata={'page': 4, 'source': 'data/PDFFile5b28ce3c2eb412.05300945.pdf'}), 
Document(page_content='the Income -tax Guidelines. In case a private limited company issues an ESOP, the\nonly applicable legislation would be the Income -tax Guidelines. However, even in\ncase of unlisted private/ public companies, it is advisable that the ESOPs comply\nwith the SEBI Guidelines, because only then can the shares issued under the ESOP\nprior to the IPO be listed. Further, any outstanding options under the ESOP at the\ntime of the IPO would be allowed to remain outstanding only if the scheme is in\naccordance with the SEBI Guidelines.  Post -IPO, the company can issue fresh\noptions under its pre -IPO ESOP only if the ESOP is in accordance with the SEBI\nGuidelines.\n16.2.2 Companies Act\nEvery public company, whether listed or unlisted, would need to pass a  special\nresolution u/s. 81(1A) of the Act. The Explanatory Statement to the Notice calling the\nmeeting would require certain information which would vary depending upon', 
metadata={'page': 1, 'source': 'data/PDFFile5b28ce3c2eb412.05300945.pdf'})]}

查询3

回复：

response: {'query': 'What is the class aware auto encoder ?', 
'result': 'A Class Aware Autoencoder is a modified version of an autoencoder that incorporates class labels into the reference data during training. This ensures that the features learned are representations of individual data points as well as the corresponding class, leading to better feature extraction for training classifiers. The effectiveness of this method is measured by comparing the accuracy of classifiers trained on features extracted by the Class Aware Autoencoder with those trained on features extracted by traditional autoencoders.', 'source_documents': [Document(page_content='Fig. 7. First Row Shows original training data of CIFAR10, second row\nshows the same images encoded and then decoded using the Class Aware\nAutoencoders\nFig. 8. First Row Shows original training data of CIFAR10, second row shows\nthe same images encoded and then decoded using simple Autoencoders\nSimilarly, Figure 9 shows the output of the class aware\nautoencoder on the training images of the UTKFace dataset\nwhile Figure 10 shows the output of traditional autoencoders\non the same images.\nFig. 9. Original image padded with class indicator in top row with images\nafter encoding and decoding in the bottom row. Class Aware Autoencoder\nwas used\nFig. 10. Training images and images after applying a traditional auto encoder\nC. Training Classifier\nIn order to check the efficacy of the Class Aware Autoen-\ncoders, we use the features extracted to train a classifier. In thiscase, we train two similar classifiers – one with the features\nextracted by the Class Aware Autoencoders and the other', 
metadata={'page': 3, 'source': 'data/Class_Aware_Auto_Encoders_for_Better_Feature_Extraction.pdf'}), Document(page_content='Fig. 7. First Row Shows original training data of CIFAR10, second row\nshows the same images encoded and then decoded using the Class Aware\nAutoencoders\nFig. 8. First Row Shows original training data of CIFAR10, second row shows\nthe same images encoded and then decoded using simple Autoencoders\nSimilarly, Figure 9 shows the output of the class aware\nautoencoder on the training images of the UTKFace dataset\nwhile Figure 10 shows the output of traditional autoencoders\non the same images.\nFig. 9. Original image padded with class indicator in top row with images\nafter encoding and decoding in the bottom row. Class Aware Autoencoder\nwas used\nFig. 10. Training images and images after applying a traditional auto encoder\nC. Training Classifier\nIn order to check the efficacy of the Class Aware Autoen-\ncoders, we use the features extracted to train a classifier. In thiscase, we train two similar classifiers – one with the features\nextracted by the Class Aware Autoencoders and the other', metadata={'page': 3, 'source': 'data/Class_Aware_Auto_Encoders_for_Better_Feature_Extraction.pdf'}), 
Document(page_content='Class Aware Auto Encoders for Better Feature\nExtraction\nAshhadul Islam\nCollege Of Science & Engineering\nHamad Bin Khalifa University\nDoha, Qatar\naislam@hbku.edu.qaSamir Brahim Belhaouari\nCollege Of Science & Engineering\nHamad Bin Khalifa University\nDoha, Qatar\nsbelhaouari@hbku.edu.qa\nAbstract —In this work, a modified operation of Auto Encoder\nhas been proposed to generate better features from the input\ndata. General autoencoders work unsupervised and learn features\nusing the input data as a reference for output. In our method\nof training autoencoders, we include the class labels into the\nreference data so as to gear the learning of the autoencoder\ntowards the reference data as well as the specific class it belongs\nto. This ensures that the features learned are representations of\nindividual data points as well as the corresponding class. The\nefficacy of our method is measured by comparing the accuracy\nof classifiers trained on features extracted by our models from', 
metadata={'page': 0, 'source': 'data/Class_Aware_Auto_Encoders_for_Better_Feature_Extraction.pdf'}), Document(page_content='Class Aware Auto Encoders for Better Feature\nExtraction\nAshhadul Islam\nCollege Of Science & Engineering\nHamad Bin Khalifa University\nDoha, Qatar\naislam@hbku.edu.qaSamir Brahim Belhaouari\nCollege Of Science & Engineering\nHamad Bin Khalifa University\nDoha, Qatar\nsbelhaouari@hbku.edu.qa\nAbstract —In this work, a modified operation of Auto Encoder\nhas been proposed to generate better features from the input\ndata. General autoencoders work unsupervised and learn features\nusing the input data as a reference for output. In our method\nof training autoencoders, we include the class labels into the\nreference data so as to gear the learning of the autoencoder\ntowards the reference data as well as the specific class it belongs\nto. This ensures that the features learned are representations of\nindividual data points as well as the corresponding class. The\nefficacy of our method is measured by comparing the accuracy\nof classifiers trained on features extracted by our models from', metadata={'page': 0, 'source': 'data/Class_Aware_Auto_Encoders_for_Better_Feature_Extraction.pdf'})]}

结论

在这里，我们展示了如何使用 Ollama 和 Lanchain 在一个完全本地化的环境中执行 RAG 操作。推理速度取决于 CPU 的处理能力和数据负载，但是所有上述的推理都是在几秒钟内生成的，持续时间不超过 1 分钟。