This talk will walk you through the options of determining search quality - from purely offline metrics that work even before deploying version 1.0 to production to online A/B testing to check continuous improvement. I will highlight some Lucene and Elasticsearch features that can tremendously help you deploy your own search quality checks.