Vald Demo: How to Use Vald with LangChain
Large Language Models (LLMs) are gaining considerable attention in today’s rapidly advancing AI-related technologies. They are beginning to be used in various applications, exemplified by services such as ChatGPT.
In addition, Retrieval Augmented Generation (RAG), which has shown promising results in tasks such as question answering and text generation, is attracting attention, too. Simply put, RAG is a system that combines an external knowledge database (such as a vector search engine built on pre-trained vectors) with an LLM to generate more refined and accurate responses.
In this context, many products that connect “Vector Search Engines” and “LLMs” are being developed and provided, such as Vald, which we are developing. Among them, we are paying attention to the LangChain library, and Vald has recently become available as part of its functionality. We expect various efforts to connect vector search and LLMs, such as LangChain and RAG, to continue to move positively.
As a first step, this post will demonstrate using Vald as a VectorStore
for LangChain.
Preparation
Please prepare the required dependencies to process with this tutorial as needed. Please refer to Get Started for preliminary preparations, such as building a Vald cluster.
- Helm: v3~
- Vald cluster: Running on the Kubernetes cluster (GKE, EKS, k3d, kind, etc.)
- LangChain
- sentence-transformers
- vald-client-python
Modify the values.yaml
Please modify the configuration file because this tutorial deals with 768-dimensional vectors.
...
## vald-agent-ngt settings
agent:
ngt:
dimension: 768
And then please deploy the Vald cluster with it.
Demo with Sample Code
We will use the original text file (scripts of TechVerse presentation) as a demo dataset. Please download it if necessary.
curl -LO https://raw.githubusercontent.com/vdaas/blog/main/assets/18-vald-with-langchain/oral-script.txt
First, import the necessary packages.
from langchain.document_loaders import TextLoader
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Vald
Second, load the text file for the dataset and insert it into the VectorStore (Vald cluster). Please modify the host and port used for connecting to the VectorStore in the below codes as needed.
raw_documents = TextLoader('oral-script.txt').load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
documents = text_splitter.split_documents(raw_documents)
embeddings = HuggingFaceEmbeddings()
db = Vald.from_documents(documents, embeddings, host="localhost", port=8080)
Finally, let’s perform a search process.
NOTE: It may take a few seconds to wait for the index to be ready before you can search with Vald, as you need to wait to complete the createIndex process.
query = "How to install Vald"
docs = db.similarity_search(query)
print(docs[0].page_content)
The example result is following.
From here, I will introduce how to install Vald. Vald can be deployed in two ways: "using the Helm command" and "using the vald-helm-operator."
Deployment using Helm can be done in a few steps.
All cluster operations are performed using Helm commands.
The vald-helm-operator, the other hand, utilizes Kubernetes' Custom Resource Definition feature (CRD).
Vald cluster is managed so that the vald-helm-operator is in the ideal state based on the predefined CRD.
All cluster settings can be changed using the kubectl command.
This is the recommended method for using Vald clusters in production environments.
Below, we will explain the deployment procedure using Helm.
The procedure has four significant steps.
Prepare a Kubernetes cluster for deployment.
Added Vald chart to Helm Repo
Create "values.yaml" describing the parameters you want to deploy for each user.
Deploy Vald cluster using Helm installation command.
Advanced similarity search techniques
In the previous demo, we performed a query-based similar search, but it is possible to achieve similar searches other than using queries. Let’s look at three examples here.
Similarity search by vector
When performing a search, you can also search using a vector, not just a query.
embedding_vector = embeddings.embed_query(query)
docs = db.similarity_search_by_vector(embedding_vector)
print(docs[0].page_content)
The example result is following.
From here, I will introduce how to install Vald. Vald can be deployed in two ways: "using the Helm command" and "using the vald-helm-operator."
Deployment using Helm can be done in a few steps.
All cluster operations are performed using Helm commands.
The vald-helm-operator, the other hand, utilizes Kubernetes' Custom Resource Definition feature (CRD).
Vald cluster is managed so that the vald-helm-operator is in the ideal state based on the predefined CRD.
All cluster settings can be changed using the kubectl command.
This is the recommended method for using Vald clusters in production environments.
Below, we will explain the deployment procedure using Helm.
The procedure has four significant steps.
Prepare a Kubernetes cluster for deployment.
Added Vald chart to Helm Repo
Create "values.yaml" describing the parameters you want to deploy for each user.
Deploy Vald cluster using Helm installation command.
Similarity search with the score
Second, it is possible to return the similarity search score and the search results.
docs_and_scores = db.similarity_search_with_score(query)
print(docs_and_scores[0])
The example result is following.
(Document(page_content='From here, I will introduce how to install Vald. Vald can be deployed in two ways: "using the Helm command" and "using the vald-helm-operator."\n\nDeployment using Helm can be done in a few steps.\n\nAll cluster operations are performed using Helm commands.\n\nThe vald-helm-operator, the other hand, utilizes Kubernetes\' Custom Resource Definition feature (CRD).\n\nVald cluster is managed so that the vald-helm-operator is in the ideal state based on the predefined CRD.\n\nAll cluster settings can be changed using the kubectl command.\n\nThis is the recommended method for using Vald clusters in production environments.\n\nBelow, we will explain the deployment procedure using Helm.\n\nThe procedure has four significant steps.\n\nPrepare a Kubernetes cluster for deployment.\n\nAdded Vald chart to Helm Repo\n\nCreate "values.yaml" describing the parameters you want to deploy for each user.\n\nDeploy Vald cluster using Helm installation command.'), 0.320773184299469)
Maximal Marginal Relevance Search (MMR)
Finally, it is possible to re-rank the results of the Vald search using an algorithm called Maximal Marginal Relevance Search(MMR).
retriever = db.as_retriever(search_type="mmr")
docs = retriever.get_relevant_documents(query)
print(docs[0].page_content)
The example result is following.
From here, I will introduce how to install Vald. Vald can be deployed in two ways: "using the Helm command" and "using the vald-helm-operator."
Deployment using Helm can be done in a few steps.
All cluster operations are performed using Helm commands.
The vald-helm-operator, the other hand, utilizes Kubernetes' Custom Resource Definition feature (CRD).
Vald cluster is managed so that the vald-helm-operator is in the ideal state based on the predefined CRD.
All cluster settings can be changed using the kubectl command.
This is the recommended method for using Vald clusters in production environments.
Below, we will explain the deployment procedure using Helm.
The procedure has four significant steps.
Prepare a Kubernetes cluster for deployment.
Added Vald chart to Helm Repo
Create "values.yaml" describing the parameters you want to deploy for each user.
Deploy Vald cluster using Helm installation command.
It is also possible to obtain the results directly using the corresponding functions in VectorStore.
db.max_marginal_relevance_search(query, k=2, fetch_k=10)
Conclusion
This article demonstrated a similar search using LangChain in a tutorial format.
For more detailed information about Vald, please refer to the following document.
If you have any questions or any requests, please feel free to contact us!
We’re waiting for your contributions!
See you next post :)