Commercializing LLMs: Lessons and Ideas for Agile Innovation
3see more slides, notes, and other material here: https://github.com/Aggregate-Intellect/practical-llms/
https://www.linkedin.com/in/josh-seltzer/
... (see more notes on the link above)
** IP strategy for large language models
6/21: When building large language models for commercialization, it's important to consider IP strategy and seek advice from experts.
7/21: Applying large language models to improve existing capabilities may not be patentable, but rethinking the entire approach to solve a problem in a new way is more likely to be patentable. #MLdevelopment #IPstrategy
** Large language models for R&D
8/21: During R&D, specify tasks, models to optimize, & create dataset to evaluate models against. Use large language models for weak label generation & data augmentation to make data set curation easier. #AI #ML #R&D
9/21: R&D conversations are often open & difficult to structure, but they can still be turned into components used by large language models. For example, to create a conversational bot for Slack, modularize into 3 LLM calls for message, trigger, & audience classification. #AI #ML #chatbot
10/21: With modularization & templated code repository, create conversational bot very quickly. Use LLM-generated info for message content classification, trigger classification, & audience classification. #AI #ML #chatbot #Slack
** Microservices and modularization of products based on large language models
11/21: Large language models (LLMs) can be used in various stages of a project, including engineering and production. They can be leveraged to build microservices or components that work together to produce the desired output.
12/21: Treating a LLM as one of the microservices of a product enables breaking down the problem into smaller and more manageable pieces. This allows for easier implementation and development, additional R&D, unit testing, and quality control.
13/21: This approach also allows for easier explainability and optimization of the process. The components in the architecture of a system that leverages LLMs should interface the language skills of GPT-like models and domain-specific expertise to get the best results.
14/21: In our slack app example, the LLM prompts used depend on the context and specifics of what is being done. In some cases, having a good prompt engineer is crucial, while in others, we can just pass exemplars that would have a bigger influence on the performance.
15/21: We can add a dialogue analysis component to control the prompts generated by LLMs in a particular domain. This component can infer information necessary to determine the most relevant examples when generating probing questions.
16/21: Another component can create dynamic prompts for in-context learning by choosing the best exemplars from a repository or by retrieving documents containing domain knowledge. This can significantly improve the performance of LLMs.
17/21: Finally, a quality control component can rank generated candidates in case the LLM generates a question or answer that is not suitable. Rule-based and human-crafted questions generated using question recipes can also be used to ensure that inappropriate responses are avoided.
** Production stage of large language models
18/21: In production, large language models can be optimized by combining them with other components to create an ecosystem of microservices that work together. This approach ensures better performance and improved efficiency of the overall system.
19/21: When optimizing costs during a project, evaluating both performance and cost is crucial. If a cheaper model performs almost as well as a more expensive one, it's advisable to choose the former. This way, you can save money without compromising on performance.
20/21: After some data is collected and the overall system performance is more well established, it's sometimes possible to replace large language models with cheaper and leaner models in production. This approach helps to reduce costs without sacrificing performance.
21/21: When building a startup with large language models, it's essential to consider the service level availability uptime of cloud infrastructure. Microsoft Azure Open AI service offers 99.9% uptime, ensuring that your system will be available to users when they need it.