Announcement: Some unnecessary components will be deprecated at the end of July 2021.
We decided to focus on developing a more simple and fast Vald as a Cloud-Native ANN search engine. Along with this decision, we also chose to deprecate some internal components that do not fit the way our going.
This change will make Vald more usable and more maintainable for users.
Deprecated Components
These components will be removed in the next minor release(v1.2.0).
[Vald Meta]
- Saving metadata including UUID and vector into user-defined external Databases.
[Vald Meta Gateway]
- Forwarding metadata to Vald Meta.
[Vald Backup Manager]
- Saving the set of UUID, vector, Agent Pod IPs into the user-defined external Database.
[Vald Backup Gateway]
- Forwarding data which should be backed up to Vald Compressor Manager.
[Vald Compressor Manager]
- Compressing backup data to make them smaller and forwarding it to Vald Backup Manager.
These components are responsible for backup system functions. They depend on external Databases, which are increasing user operation costs and affecting network latency. To provide a fast ANN search engine, we thought removing as external dependencies as possible is the best way. As a result, we decided to remove them.
On the other hand, Vald has the other backup function even these components are not usable. Vald Agent NGT, the core engine of Vald, saves their graph tree indexing structure on the Kubernetes Persistent Volume as a backup. We will support it as the Vald standard backup function.
In addition, to prevent backup data on PV lost, you can backup yout data to the user-defined cloud storages (like Amazon S3, Google Cloud Storage) using Vald Agent Sidecar.
How will it change the architecture?
From the next release, the architecture of Vald will be changed as the below image.
The main components are Vald LB Gateway, Vald Agent, Vald Discoverer, Vald Manager Index.
Vald Ingress/Egress Filter and Vald Filter Gateway are optional components. And, Vald Agent Sidecar is also optional. (About Filter, we will describe in shortly.)
As you can see, you don’t need to prepare the external DB, like MySQL, Cassandra, etc. In the point of external backup, you can save each graph tree indexing structure of Vald Agent using Vald Agent Sidecar and Cloud Storage (like Amazon S3, Google Cloud Storage). If you need to backup, please use Vald Agent Sidecar with cloud storage.
FAQ
If you have any questions, please contact our Slack workspace.
And, we will publish a future roadmap soon.
