Build domain-specific embedding models in under a day
nvidia and hugging face released a recipe to fine-tune embedding models on domain-specific documents in under a day using a single gpu. the pipeline uses llms to generate synthetic qa pairs and hard negative mining, eliminating manual labeling. internal tests on jira data improved recall@60 by 26% and nvidia docs data boosted recall@10 by over 10%.
NVIDIA and Hugging Face released a guide to train domain-specific embedding models on a single GPU within 24 hours. The pipeline utilizes NeMo Data Designer to generate synthetic question-answer pairs from domain documents using LLMs, removing the need for manual labeling. It also implements hard negative mining to improve model precision on difficult retrieval tasks. In practical applications, Atlassian reported a 26% increase in Recall@60 when applied to JIRA datasets, while NVIDIA documentation tests showed over 10% gains in Recall@10 and NDCG@10. The tutorial fine-tunes the open-source Llama-Nemotron-Embed-1B-v2 model.