Vald Demo: How to Use Vald with LangChain

Large Language Models (LLMs) are gaining considerable attention in today’s rapidly advancing AI-related technologies. They are beginning to be used in various applications, exemplified by services such as ChatGPT.

In addition, Retrieval Augmented Generation (RAG), which has shown promising results in tasks such as question answering and text generation, is attracting attention, too. Simply put, RAG is a system that combines an external knowledge database (such as a vector search engine built on pre-trained vectors) with an LLM to generate more refined and accurate responses.

In this context, many products that connect “Vector Search Engines” and “LLMs” are being developed and provided, such as Vald, which we are developing. Among them, we are paying attention to the LangChain library, and Vald has recently become available as part of its functionality. We expect various efforts to connect vector search and LLMs, such as LangChain and RAG, to continue to move positively.

As a first step, this post will demonstrate using Vald as a VectorStore for LangChain.

Preparation

Please prepare the required dependencies to process with this tutorial as needed. Please refer to Get Started for preliminary preparations, such as building a Vald cluster.

• Helm: v3~
• Vald cluster: Running on the Kubernetes cluster (GKE, EKS, k3d, kind, etc.)
• LangChain
• sentence-transformers
• vald-client-python

Modify the values.yaml

Please modify the configuration file because this tutorial deals with 768-dimensional vectors.

...
## vald-agent-ngt settings
agent:
  ngt:
    dimension: 768

And then please deploy the Vald cluster with it.

Demo with Sample Code

We will use the original text file (scripts of TechVerse presentation) as a demo dataset. Please download it if necessary.

curl -LO https://raw.githubusercontent.com/vdaas/blog/main/assets/18-vald-with-langchain/oral-script.txt

First, import the necessary packages.

from langchain.document_loaders import TextLoader
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Vald

Second, load the text file for the dataset and insert it into the VectorStore (Vald cluster). Please modify the host and port used for connecting to the VectorStore in the below codes as needed.

raw_documents = TextLoader('oral-script.txt').load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
documents = text_splitter.split_documents(raw_documents)
embeddings = HuggingFaceEmbeddings()
db = Vald.from_documents(documents, embeddings, host="localhost", port=8080)

Finally, let’s perform a search process.

NOTE: It may take a few seconds to wait for the index to be ready before you can search with Vald, as you need to wait to complete the createIndex process.

Vald | Data Flow

query = "How to install Vald"
docs = db.similarity_search(query)
print(docs[0].page_content)

The example result is following.

From here, I will introduce how to install Vald. Vald can be deployed in two ways: "using the Helm command" and "using the vald-helm-operator."

Deployment using Helm can be done in a few steps.

All cluster operations are performed using Helm commands.

The vald-helm-operator, the other hand, utilizes Kubernetes' Custom Resource Definition feature (CRD).

Vald cluster is managed so that the vald-helm-operator is in the ideal state based on the predefined CRD.

All cluster settings can be changed using the kubectl command.

This is the recommended method for using Vald clusters in production environments.

Below, we will explain the deployment procedure using Helm.

The procedure has four significant steps.

Prepare a Kubernetes cluster for deployment.

Added Vald chart to Helm Repo

Create "values.yaml" describing the parameters you want to deploy for each user.

Deploy Vald cluster using Helm installation command.

Advanced similarity search techniques

In the previous demo, we performed a query-based similar search, but it is possible to achieve similar searches other than using queries. Let’s look at three examples here.

Similarity search by vector

When performing a search, you can also search using a vector, not just a query.

embedding_vector = embeddings.embed_query(query)
docs = db.similarity_search_by_vector(embedding_vector)
print(docs[0].page_content)

The example result is following.

From here, I will introduce how to install Vald. Vald can be deployed in two ways: "using the Helm command" and "using the vald-helm-operator."

Deployment using Helm can be done in a few steps.

All cluster operations are performed using Helm commands.

The vald-helm-operator, the other hand, utilizes Kubernetes' Custom Resource Definition feature (CRD).

Vald cluster is managed so that the vald-helm-operator is in the ideal state based on the predefined CRD.

All cluster settings can be changed using the kubectl command.

This is the recommended method for using Vald clusters in production environments.

Below, we will explain the deployment procedure using Helm.

The procedure has four significant steps.

Prepare a Kubernetes cluster for deployment.

Added Vald chart to Helm Repo

Create "values.yaml" describing the parameters you want to deploy for each user.

Deploy Vald cluster using Helm installation command.

Similarity search with the score

Second, it is possible to return the similarity search score and the search results.

docs_and_scores = db.similarity_search_with_score(query)
print(docs_and_scores[0])

The example result is following.

(Document(page_content='From here, I will introduce how to install Vald. Vald can be deployed in two ways: "using the Helm command" and "using the vald-helm-operator."\n\nDeployment using Helm can be done in a few steps.\n\nAll cluster operations are performed using Helm commands.\n\nThe vald-helm-operator, the other hand, utilizes Kubernetes\' Custom Resource Definition feature (CRD).\n\nVald cluster is managed so that the vald-helm-operator is in the ideal state based on the predefined CRD.\n\nAll cluster settings can be changed using the kubectl command.\n\nThis is the recommended method for using Vald clusters in production environments.\n\nBelow, we will explain the deployment procedure using Helm.\n\nThe procedure has four significant steps.\n\nPrepare a Kubernetes cluster for deployment.\n\nAdded Vald chart to Helm Repo\n\nCreate "values.yaml" describing the parameters you want to deploy for each user.\n\nDeploy Vald cluster using Helm installation command.'), 0.320773184299469)

Maximal Marginal Relevance Search (MMR)

Finally, it is possible to re-rank the results of the Vald search using an algorithm called Maximal Marginal Relevance Search(MMR).

retriever = db.as_retriever(search_type="mmr")
docs = retriever.get_relevant_documents(query)
print(docs[0].page_content)

The example result is following.

From here, I will introduce how to install Vald. Vald can be deployed in two ways: "using the Helm command" and "using the vald-helm-operator."

Deployment using Helm can be done in a few steps.

All cluster operations are performed using Helm commands.

The vald-helm-operator, the other hand, utilizes Kubernetes' Custom Resource Definition feature (CRD).

Vald cluster is managed so that the vald-helm-operator is in the ideal state based on the predefined CRD.

All cluster settings can be changed using the kubectl command.

This is the recommended method for using Vald clusters in production environments.

Below, we will explain the deployment procedure using Helm.

The procedure has four significant steps.

Prepare a Kubernetes cluster for deployment.

Added Vald chart to Helm Repo

Create "values.yaml" describing the parameters you want to deploy for each user.

Deploy Vald cluster using Helm installation command.

It is also possible to obtain the results directly using the corresponding functions in VectorStore.

db.max_marginal_relevance_search(query, k=2, fetch_k=10)

Conclusion

This article demonstrated a similar search using LangChain in a tutorial format.

For more detailed information about Vald, please refer to the following document.

Vald | Docs

If you have any questions or any requests, please feel free to contact us!

Slack

We’re waiting for your contributions!

Vald | Contributing Guide

See you next post :)

Other Post

Vald. A highly scalable distributed fast approximate nearest neighbour dense vector search engine.

A New World created by similar search: Cases where Vald can be used.

A Super Easy Way to Try Similarity Search using Vald