Comparative Analysis of Open Source and Commercial Embedding Models for Question Answering

In this industry track presentation, we will provide a comprehensive tour of the best performing embedding models for question answering, as determined by the Massive Text Embedding Benchmark.(1) We showcase these models while also considering solutions offered by OpenAI and Cohere, renowned for their state-of-the-art performance. Through rigorous evaluations on internal Salesforce datasets tailored for Question Answering on Knowledge articles, we compare the performance of these models using standardized metrics. Our analysis sheds light on the current state-of-the-art in question answering using embedding models across three diverse domains. We hope that this talk's outcomes will empower practitioners and researchers to make informed decisions when selecting the most suitable solution for their specific requirements.