Dropbox unveils upgraded search engine Nautilus with machine learning capabilities

For a company which stores hundreds of billions of files, search is vital for Dropbox, both for its customers and internal-facing. As a result, the storage provider has overhauled its search with machine learning capabilities. The new platform, called Nautilus, had four goals on its launch; delivering top class performance, scalability and reliability, providing intellient document ranking and retrieval, flexibility for customising document-indexing and query-processing pipelines, and wrap it all up in a reliable, secure package. The architecture is based at a high level on indexing and serving. Indexing, naturally, is a key factor of any search, collecting, parsing, and storing data for retrieval. The serving function uses the index to return results from user queries. This is by no means uncommon, but with the sheer scale involved, more needs to be done. Dropbox generates 'offline' builds of the search index every few days on average, and puts together 'index mutations' that can be applied to both the live index and a persistent document store in almost real-time - to approximately a few seconds.