So you mentioned that the database is has 10 million pairs. Are you doing a brute force search? Additionally, how is the database updated? Surely the performance statistics will change as new data come in. Do you rerun all your analysis every some period of time?
Thanks,