Gemma Scope 2: New Tools for LLM Interpretability

Gemma Scope 2: Helping the AI Safety Community Deepen Understanding of Complex Language Model Behavior

Google DeepMind announced Gemma Scope 2, a comprehensive and open suite of interpretability tools for the entire Gemma 3 family of models, ranging from 270M to 27B parameters. This release represents the largest open-source contribution of interpretability tools by an AI lab to date, requiring the storage of approximately 110 Petabytes of data and the training of over 1 trillion parameters.

Large Language Models (LLMs) exhibit impressive reasoning capabilities, but their internal workings remain largely opaque, hindering effective debugging and safety assessment. Without visibility into these processes, identifying the root cause of unexpected model behavior or vulnerabilities can be challenging, potentially leading to deployment failures or security breaches.

Key Insights

• 110 Petabytes of data were used to develop Gemma Scope 2: 2025-12-19
• Sparse Autoencoders (SAEs) and transcoders enable internal model examination.
• Gemma Scope 2 builds on the original Gemma Scope’s ability to research hallucinations, secret identification, and safer model training.

Practical Applications

• Use Case: AI safety researchers can utilize Gemma Scope 2 to audit and debug AI agents, enhancing reliability and trustworthiness.
• Pitfall: Relying solely on external model behavior without investigating internal state can lead to overlooked vulnerabilities like discrepancies between reasoning and actual internal processes.

References:

• https://deepmind.google/blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-behavior/