Dropbox has announced today that its new Nautilus ML-powered search engine is now in use and offering users more powerful search than ever before. Dropbox’s engineering team has also shared specifics on the architecture of the Nautilus search engine.
Dropbox detailed in a blog post today some of the challenges when it comes to search on its platform.
Search presents a unique challenge when it comes to Dropbox due to our massive scale—with hundreds of billions of pieces of content—and also due to the need for providing a personalized search experience to each of our 500M+ registered users.
Another difficulty when it comes to search is how often Dropbox content changes. The company’s engineering team had four main goals at it worked on developing the new ML-based Nautilus search engine.
- Deliver best-in-class performance, scalability, and reliability to deal with the scale of our data
- Provide a foundation for implementing intelligent document ranking and retrieval features
- Build a flexible system that would allow our engineers to easily customize the document indexing and query processing pipelines for running experiments
- And, as with any system that manages our users’ content, our search system needed to deliver on these objectives quickly, reliably, and with strong safeguards to preserve the privacy of our users’ data
Dropbox tested Nautilus in shadow mode before making it the primary search engine. For a full look at the infrastructure of Nautilus, check out the detailed blog post. Dropbox also mentions that it is currently looking for software engineers.