Intel

Learn what a RAG system is and how to deploy it using OPEA’s open source tools and frameworksPhoto by Xavi Cabrera on UnsplashBy this point, most of us have used a large language model (LLM), like ChatGPT, to try to find quick answers to questions that rely on general knowledge and information. These questions range from the practical (What’s the best way to learn a new skill?) to the philosophical (What is the meaning of life?).Image 1: ChatGPT gives what are some of the most common questions it get askedBut how do you get answers to questions that are personal? How much does your LLM know about you? Or your family?Let’s test ChatGPT and see how much it knows about my parents.Image 2: ChatGPT answer to “Do you know who is my mum?”It’s understandable to feel frustrated when a model doesn’t recognize you, but it’s important to remember that these models don’t have much information about our personal lives. Unless you’re a celebrity or have your own Wikipedia […]

Boost your RAG system’s accuracy by adding a reranker to select the most relevant context chunks.Photo by shawnanggg on UnsplashBy Eduardo Rojas Oviedo with Ezequiel LanzaIn our previous post, Tabular Data, RAG, & LLMs, we explored how to improve a large language model’s (LLM’s) ability to generate responses by feeding it small tabular data in various formats to provide necessary context. However, when the provided context is only partially correct, mismatches can lead to less accurate responses from our LLM.In a typical RAG architecture, the retriever part gets chunks of data based on a similarity search from documents stored in a knowledge base. But are those chunks always the most relevant for our case?Using the example from our previous article, what if we ask, “Who are the top billionaires in the tech industry in 2024?” and we get documents related to the overall list of world billionaires, including those from various industries that are not specific to tech? […]

Learn how to create a container to expose your local large language model (LLM) for consumption by an API.Photo by Jukebox Print on UnsplashPresented by Ezequiel Lanza — Open Source AI Evangelist (Intel)Let’s say you’re building a chatbot for your company and you need to find the best way to deploy it. You might start by building your logic in a Jupyter Notebook, but this won’t work when you want to deploy it in a live environment where users can interact with it. You might then start thinking about a suitable strategy for making your application scalable, portable, and efficient. Cloud native development emerges as the best alternative, allowing you to focus on topics like scaling and dimensioning while treating each piece as an isolated module.So, what happens next? You need to create the core components of your application, starting with the large language model (LLM). You might want an LLM container that receives a string input and returns the model’s response […]

How to ingest small tabular data when working with LLMs.Photo by Mika Baumeister on UnsplashBy Eduardo Rojas Oviedo with Ezequiel LanzaSay you’re a financial analyst working for an investment firm. Your job involves staying ahead of market trends and identifying potential investment opportunities for your clients, who are often curious about the world’s richest people and their sources of wealth. You might consider using a retrieval augmented generation (RAG) system to easily and quickly identify market trends, investment opportunities, and economic risks as well as answer questions like, “Which industry has the highest number of billionaires?” or “How does the gender distribution of billionaires compare across different regions?”Your first step is to go to the source to get that information. However, as you do an initial inspection, you discover a possible obstacle. The document contains not only text but also TABLES!Image 1 : A table which includes a ranking of the […]

Take the complexity out of deploying cloud native LLMs with LangChain and Intel Developer CloudBy Arun Gupta and Ezequiel LanzaImagine you could give a personal assistant to each employee. Productivity across your organization would spike as employees can see how AI can help them and feel empowered to focus on strategic thinking. This dream scenario is possible with powerful AI technology, such as department-specific chatbots that deliver fast, high-quality results.However, businesses often need to weave together multiple large language models (LLMs) to support diverse use cases. Because each model may have different compute and storage needs or specific knowledge to be used by internal departments in unique ways, the complexity can quickly skyrocket.The right set of tools can take the complexity out of the deployment process. Here we’ll explore a reference architecture for building and deploying multiple LLMs in a single user interface with Kubernetes and LangChain. You can also […]

Two methods to optimize your vector database when using RAGPhoto by Ilya Pavlov on UnsplashBy Cathy Zhang and Dr. Malini BhandaruContributors: Lin Yang and Changyan LiuGenerative AI (GenAI) models, which are seeing exponential adoption in our daily lives, are being improved by retrieval-augmented generation (RAG), a technique used to enhance response accuracy and reliability by fetching facts from external sources. RAG helps a regular large language model (LLM) understand context and reduce hallucinations by leveraging a giant database of unstructured data stored as vectors — a mathematical presentation that helps capture context and relationships between data.RAG helps to retrieve more contextual information and thus generate better responses, but the vector databases they rely on are getting ever larger to provide rich content to draw upon. Just as trillion-parameter LLMs are on the horizon, vector databases of billions of vectors are not far behind. As optimization […]

Unlock more accurate and meaningful AI outcomes with RAG (retrieval-augmented generation).Photo by No Revisions on UnsplashBy Eduardo Rojas Oviedo and Ezequiel LanzaThe retrieval-augmented generation (RAG) process has gained popularity due to its potential to enhance the understanding of large language models (LLMs), providing them with context and helping to prevent hallucinations. The RAG process involves several steps, from ingesting documents in chunks to extracting context to prompting the LLM model with that context. While known to significantly improve predictions, RAG can occasionally lead to incorrect results. The way documents are ingested plays a crucial role in this process. For instance, if our “context documents” contain typos or unusual characters for an LLM, such as emojis, it could potentially confuse the LLM’s understanding of the provided context.In this post, we’ll demonstrate the use of four common natural language processing (NLP) techniques to clean […]

Seamlessly inference from edge to cloud and across devicesPhoto by Sharad Bhat on UnsplashPresented by Ria Cheruvu & Dr. Paula RamosGenerative AI is revolutionizing how organizations create high-quality text and images. However, challenges such as high inference time, unsatisfactory customer experience, and intricated developer experience can pose obstacles to the seamless adoption of generative AI. Developers are often torn between using expensive machines, such as servers or cloud providers, and porting workloads to an edge device with optimized models. While the cloud offers limitless compute power on demand, the edge better supports real-time processing, data sovereignty, and cost efficiency.With a hybrid AI approach, developers don’t have to choose. By allowing developers to automatically process AI workloads using available or targeted system resources on the edge or in the cloud, hybrid AI combines the unique strengths of the edge and the cloud. In this demo, we’ll […]

Help meet scalability, security, and sustainability demands.Photo by SpaceX on UnsplashPresented by Dr. Malini BhandaruBuilding an effective AI pipeline presents developers with unique scalability, security, and sustainability challenges. Developers must ensure that AI pipelines can scale to meet dynamic demands while balancing costs. Developers must also protect the data — which is often sensitive, proprietary, or regulated — used in AI models, as well as the valuable AI models themselves. Lastly, developers must build pipelines that use resources responsibly and that help advance corporate sustainability goals.In this demo, we’ll show you how our reference solution addresses these challenges to help developers deliver improved scalability, security, and sustainability to AI pipelines for a variety of use cases. Watch the full demo here.A complete set of resourcesThe Intel® reference solution combines a set of cloud-native tools to help overcome the challenges of […]

Hey Mr, DJ! Which large language model should I use?Photo by Marcela Laskoski on UnsplashPresented by Ezequiel Lanza — AI Open Source Evangelist (Intel)Developers working in artificial intelligence must make a pivotal decision at the start of any language project. Do you use a foundation model or a fine-tuned model? How do you decide? If you’ve ever planned a party, it’s a lot like choosing a DJ.Imagine you’re planning an event, and you need to pick a DJ for the evening entertainment. Do you go with someone who sticks to the tried-and-true hits or the one who dives deep into the knobs for a mind-blowing, unique experience? Will you go with the DJ who plays a broad mix of the classics, spanning multiple crowd-pleasing genres? Or will you choose the one with the hip-hop playlist that would make DJ Kool Herc jealous?The answer depends on the type of party you want to throw — just as selecting the right language model depends on the type of results you want to […]