Why You Need Domain Specific LLM and How To Design One
| nmb@konfitech.com
The impact that LLMs and GenAI has on knowledge work is hard to deny. In a study done by Harvard, you can see in the figure above that the quality of work has a very positive impact. This is especially true for a domain specific LLM, which can provide highly tailored and accurate information.
However, this is not for all tasks. In the study they talk about the changes of workflow as a knowledge worker is doing his work some tasks fall under “the frontier” and some go outside of it. You can think of everything under the frontier as mundane normal work, while everything outside of the frontier is expert level knowledge work, or outside of the average work. Such as domain specific knowledge.
In the figure below, you can see that this had a negative impact on the quality of the work. That for tasks “outside of the frontier” the control group did better, and when AI was involved, it had a negative impact.
This would have some interesting impacts on how we should work with AI and raise some concerns about using the “general” models out there like ChatGPT or Gemini. Which shows why it is so important that when adding a generative AI or LLM to your tech stack that you can design a specialized one that can give you the results you want and need.
This is why you would want to specialize your AI helper to a domain specific LLM.
- Precision and expertise: No longer will the AI provide you with generic responses, but more nuanced and appropriate responses to the field of work.
- Trustworthiness: You can have an increased trust in the responses of the AI that it is not irrelevant or non-applicable. It will filter those things out.
- AI Safety: In knowledge work it is critical to not get things wrong, using domain specific AI can make it include better safety mechanisms when giving information preventing misinformation.
- UX: The LLM understands language from the domain better and can keep up with domain specific conversations.
- Efficiency: This often requires a lesser compute than proprietary models and a lesser dataset. Meaning that the quality of the data is higher, and you will pay less due to lesser resource consumption.
Overall, the models can be more specific to tie together general knowledge and processes to domain specific knowledge.
The first step to designing your domain specific LLM is prompt engineering.
When doing prompt engineering you will add certain phrases to you your input or prompt to the LLM. Like when you ask a question of “summarize this paragraph” you can ask it to “summarize this paragraph like a lawyer”. This will guide the LLM to providing you with the domain specific information to lawyers and is a free and easy way to get domain specific information.
The second step is retrieval augmented generation or RAG for short. This is adding a data source to the LLMs original training data. For example, if you’re a doctor using Microsoft CoPilot which is built on OpenAI, if you then get access to medical records the model can access those records through your account. This way the LLM can get that information and consider it when crafting its responses. As a result, the answer will be grounded not only in its training data but also most up to date information that you chose to provide it with.
The third step is fine tuning, staying on the example of medical field, if you then choose to train the AI on mostly medical data it will inherently be much better in that specific field. Making it even more powerful when combined with a RAG.
Overall, specificity is key to succeeding in getting great LLMs. All these techniques are great when building LLMs for domain specific tasks and can reduce the flaws that AI presents when working in tasks “outside the frontier”.
Looking to build a domain specific LLM for your company, lets have a discussion: https://www.konfitech.com/contact-us
Or you don’t know where to start, read some of our case studies: https://www.konfitech.com/case-studies-and-tech-stack
Want to learn more about AI? Check out our page for Data & AI: https://www.konfitech.com/data-and-ai
Study referenced: https://www.hbs.edu/ris/Publication%20Files/24-013_d9b45b68-9e74-42d6-a1c6-c72fb70c7282.pdf