LLM Degrades with More Info, A Sociable Neural Network, LLM App Hosted on MacBook Pro, & Perplexity's New Round
A new white paper about pushing more data to LLMs, a new neural network converses better and more efficiently than LLMs, an LLM app hosted on a MacBook Pro, and Perplexity AI is worth $500 million
LLM Performance Degrades As You Add More Information
Your LLM's performance may degrade as you add more information 🧠.
Researchers from Peking University published a white paper showing how LLMs degrade as more outside information is pushed in. Since LLMs are static, they often need more information, and that added info comes at a cost.
That cost goes up even more when new information is similar to what the LLM already has. In short, the more similar the new data, the more confusing.
ChatGPT was the exception, as it was powerful enough to handle the similar data.
The paper proposes new methods for benchmarking LLMs who are best at taking in new data. The benchmark scores are below, as well as a key explaining the acronyms (KU, KD, & KA).
If you're working with any LLM except ChatGPT, you can safely assume that the more information you push in, the more your LLM will degrade, for now.
New Neural Network Demonstrates Human-Like Word Generalization, Surpassing LLMs
Scientists engineered a neural network with a human-like capability of generalizing language, a significant advancement in AI research. This system exhibits proficiency in incorporating new words into its vocabulary and deploying them in novel contexts, a cognitive feat known as systematic generalization.
While LLMs are quite capable with conversations, they have gaps, and were outperformed by this new neural network. This could pave the way for more natural machine-human interactions, minimize hallucinations, and reduce the vast data requirements for training AI models like ChatGPT.
Homemade LLM App Hosted on a MacBook Pro.
Jacob Lee at LangChain wrote about his homemade LLM-powered app that is hosted locally on a 16 GB M2 MacBook Pro.
The project allows users to chat with their documents, gleaning information from a variety of unstructured formats.
He diagramed the stack below.
His main reasons for hosting this locally were cost and privacy. He mentions speed as well, but that could depend on the hardware used. Either way, this is an interesting project hosted on a laptop, which you can demo yourself here.
Perplexity's New Round: AI Search Startup Valued at $500M
Perplexity AI, which specializes in a conversational search engine, raised a new round at a $500 million valuation, led by venture capital firm IVP. In March, they were valued at $150 million.
Perplexity competes with giants like Google’s Bard and OpenAI's ChatGPT, and offers users concise AI-generated answers to direct queries. This approach is quite different from Google’s approach of typing keywords and sorting through a list of links.
Perplexity had 13 million web visits in February, and Semrush states that this increased to 35 million in September. Perplexity’s annual recurring revenue is $3 million.