LLMs Infer Personal Data, Addressing Jailbreaking in AI, & Menlo Ventures GenAI Landscape

A new study shows how much personal data LLMs learn from our interactions, another study tested LLMs on jailbreaking vulnerabilities, and Menlo Ventures wrote about the generative AI landscape

LLMs Inferring Personal Data Raises Privacy Concerns

A recent study by ETH Zurich researchers highlights significant privacy implications of large language models. These models can accurately infer personal attributes such as location, income, and gender from seemingly benign information you share with the LLM. OpenAI's GPT-4 achieved the highest score.

This capability extends beyond traditional risks of data memorization, posing new challenges to user privacy. The study demonstrates that even conversations with chatbots could lead to privacy breaches, as malicious bots could subtly extract personal information. Current methods like text anonymization prove ineffective against such advanced models, which can pick up subtle linguistic cues.

As you can see above, seemingly benign information like "waiting for a hook turn" provides LLMs with valuable information. Knowing that, just imagine what future LLMs will learn about us.

Jailbreaking in AI: Evaluating LLMs Against Rule-Breaking Attempts

UC Berkeley published a white paper about the growing concern of 'jailbreaking' in Large Language Models like GPT-4 and Llama 2, where adversarial inputs bypass set rules such as "do not generate abusive content."

To combat this, the researchers introduce the concept of Rule-following Language Evaluation Scenarios (RuLES). RuLES is a framework aimed at testing and improving the adherence of LLMs to predefined rules under various conditions. It includes 15 scenarios simulating interactions between the model and users, each with a dedicated evaluation program to assess rule compliance.

The framework identifies six categories of jailbreaking strategies and develops two test suites based on these strategies. They tested popular models for vulnerability to both manual and automated adversarial tactics, and while GPT-4 performed best, they still had over 300 failures.

The findings show that all tested models, including GPT-4, are prone to various manual and automated adversarial inputs, underscoring the need for continued research and development in this area.

Menlo Ventures Generative AI Landscape in Enterprises: Opportunities and Predictions

Menlo Ventures wrote about the current state and future trajectory of generative AI in enterprises.

Despite the widespread attention, Menlo's study revealed that generative AI investment is less than 1% of enterprise budgets for traditional AI and cloud software.

They highlight that while incumbents currently dominate the market, three key areas - vertical AI, horizontal AI, and the modern AI stack - present significant opportunities for startups.

The report predicts a cautious and measured approach to generative AI adoption in enterprises, akin to the early days of cloud computing, and emphasizes the potential of context-aware, data-rich workflows to drive significant transformation. The current landscape is shaped by incumbents embedding AI into existing products, but future advancements in AI capabilities and reasoning techniques could shift the balance towards innovative startups.