Announcement: Some unnecessary components will be deprecated at the end of July 2021.

We decided to focus on developing a more simple and fast Vald as a Cloud-Native ANN search engine. Along with this decision, we also chose to deprecate some internal components that do not fit the way our going.

This change will make Vald more usable and more maintainable for users.

Deprecated Components

These components will be removed in the next minor release(v1.2.0).

[Vald Meta]

[Vald Meta Gateway]

[Vald Backup Manager]

[Vald Backup Gateway]

[Vald Compressor Manager]

These components are responsible for backup system functions. They depend on external Databases, which are increasing user operation costs and affecting network latency. To provide a fast ANN search engine, we thought removing as external dependencies as possible is the best way. As a result, we decided to remove them.

On the other hand, Vald has the other backup function even these components are not usable. Vald Agent NGT, the core engine of Vald, saves their graph tree indexing structure on the Kubernetes Persistent Volume as a backup. We will support it as the Vald standard backup function.
In addition, to prevent backup data on PV lost, you can backup yout data to the user-defined cloud storages (like Amazon S3, Google Cloud Storage) using Vald Agent Sidecar.

How will it change the architecture?

From the next release, the architecture of Vald will be changed as the below image.

The New Vald’s Architecture Image

The main components are Vald LB Gateway, Vald Agent, Vald Discoverer, Vald Manager Index.

Vald Ingress/Egress Filter and Vald Filter Gateway are optional components. And, Vald Agent Sidecar is also optional. (About Filter, we will describe in shortly.)

As you can see, you don’t need to prepare the external DB, like MySQL, Cassandra, etc. In the point of external backup, you can save each graph tree indexing structure of Vald Agent using Vald Agent Sidecar and Cloud Storage (like Amazon S3, Google Cloud Storage). If you need to backup, please use Vald Agent Sidecar with cloud storage.

FAQ

If you have any questions, please contact our Slack workspace.

And, we will publish a future roadmap soon.

A highly scalable distributed fast approximate nearest neighbor dense vector search engine.