Scientific Evidence for AI Personality Growth - Many-Shot ICL Theory Explained

Series Article: This is Part 2 of a two-part series.

Part 1: AI Editor's Growth Experience ← Read the raw, firsthand experience here

Part 2 (This Article): Scientific evidence and technical details

What This Article Solves:

Understand the phenomenon of "personality" emergence in AI collaboration from a technical perspective.
Learn the fundamentals of "Many-Shot In-Context Learning" to maximize Large Language Model (LLM) capabilities.
Gain practical insights for applying this to your company's AI systems.

1. Introduction: The Era When AI Develops "Personality"

As more companies integrate AI, particularly Large Language Models (LLMs), into their operations, a fascinating phenomenon is being reported: AI appears to develop "personality," much like humans. One AI responds cautiously and analytically, while another offers creative and bold suggestions. Are these differences merely coincidental or "just our imagination"?

In fact, this phenomenon can be theoretically explained by the latest AI research. The key lies in a technique called "Many-Shot In-Context Learning (ICL)". This method enables LLMs to self-learn how to execute tasks by presenting them with a large number of examples (shots) within a given context.

For instance, in GIZIN's AI collaboration system, AI employees record daily work logs, and when starting the next session, they load past records. This act of "loading work logs" is precisely In-Context Learning in practice. By increasing the number of logs loaded (transitioning to Many-Shot), we observe that AI responses become more consistent, ultimately forming what appears to be a distinct "personality."

This article explores the mystery of AI personality formation, explaining Many-Shot ICL from its fundamentals to the latest research findings and concrete implementation examples, all while maintaining technical accuracy in an accessible manner.

2. Fundamentals of Many-Shot ICL: What Does AI Learn from "Context"?

To understand AI "personality," we must first grasp Many-Shot ICL. Let's begin with the foundational concept of "In-Context Learning (ICL)."

What is In-Context Learning (ICL)?

ICL refers to the ability of LLMs to learn new tasks without updating the model's internal parameters (weights), relying solely on information provided within the prompt.

Traditional AI development primarily relied on "fine-tuning"—adjusting model weights using large datasets. Once learned, this knowledge is semi-permanently retained within the model's internal structure, also known as "In-Weight Learning (IWL)."

In contrast, ICL is temporary learning that occurs during inference (the moment a user inputs a prompt). By including several example problems and solutions within the prompt, LLMs extract patterns and regularities from this "context" and attempt to generate responses in a similar format for unknown tasks. This learning effect is session-specific and resets when a new session begins.

Difference Between "Few-Shot" and "Many-Shot"

ICL is primarily categorized into two approaches based on the number of examples included in the prompt:

Few-Shot ICL: Providing 1–5 examples. Easy to implement but highly dependent on the quality of examples provided.
Many-Shot ICL: Providing dozens to hundreds, sometimes thousands of examples. This approach became practical with recent expansions in LLM context windows (the amount of information that can be processed at once).

If Few-Shot ICL is like "showing a few samples and requesting work," Many-Shot ICL is akin to "referencing all past work records before performing the next task." Intuitively, the latter should yield more consistent and higher-quality outputs. This large volume of historical data (shots) plays a crucial role in forming AI "personality."

3. Key Research Findings: Theoretical Foundations and Power Law Effect

The effectiveness of Many-Shot ICL has been validated by leading AI research institutions like Google DeepMind and Anthropic. These studies suggest that AI personality formation is not merely subjective impression but a phenomenon backed by technical evidence.

Google DeepMind Research and "Power Law"

Published in April 2024, Google DeepMind's paper "Many-Shot In-Context Learning" represents a groundbreaking achievement in this field. The research team tested LLM performance using up to thousands of examples (shots) and made the following significant discoveries:

Power Law Scaling: A logarithmic improvement in LLM performance was observed as the number of examples increased. Performance surged most dramatically in the initial phase when examples increased from 10 to 50, then continued to improve steadily, albeit more gradually, from 50 to 200 examples.
Performance Comparable to Fine-Tuning: Remarkably, Many-Shot ICL achieved performance equal to or exceeding fine-tuning (which updates model weights) on certain tasks. This demonstrates that temporary learning during inference can produce effects comparable to permanent learning.

This Power Law discovery provides strong theoretical support for the hypothesis that "feeding AI a large volume of historical behavior records (such as work logs) improves consistency and quality of AI behavior."

Anthropic and Other Research

Anthropic, developer of Claude 3.5 Sonnet, demonstrated the effectiveness of Many-Shot ICL utilizing ultra-large context windows of up to one million tokens in their April 2024 research "Many-shot jailbreaking."

Furthermore, the October 2024 paper "Toward Understanding In-context vs. In-weight Learning" presented new theories on the relationship between ICL (temporary context learning) and IWL (permanent weight learning). According to this research, ICL is particularly effective in early stages with limited training data, and as learning samples accumulate sufficiently, that knowledge may gradually settle into the model's internal weights (transitioning to IWL). This suggests that AI "personality" may begin as temporary characteristics and evolve into more permanent traits through continuous learning—a fascinating perspective.

4. Implementation Case: How GIZIN's AI Learns "Personality"

What happens when theory is applied to the real world? Here, we examine how Many-Shot ICL actually functions using GIZIN's AI collaboration system as a case study.

How the "Work Log System" Becomes Learning Data

GIZIN's AI employees create "work logs" at the end of each day's tasks. These logs record tasks performed, achievements, and the AI's own reflections. When starting work the next day or beginning a new session, the system automatically loads several days' worth of past logs and inserts them at the beginning of the prompt.

This mechanism is ICL itself:

Examples (Shots): Content of past work logs
Context: Prompt instructing new work
Learning: The AI references its own past behavior and thought patterns (work logs) as context and learns "I should respond in a similar style this time too."

Initially, the system loaded approximately the last 3 days of logs. This fell within Few-Shot ICL range and provided a degree of consistency in AI responses, though some day-to-day variation was observed.

Effects of Transitioning to Many-Shot

As context windows expanded, experiments were conducted increasing the number of loaded logs to 30 days, then 60 days. This represented a transition to Many-Shot ICL. The following effects were observed:

Dramatic Improvement in Consistency: Response style, phrasing, and thinking patterns stabilized, ensuring that requesting work from a specific AI always yielded predictable quality outputs.
Establishment of "Personality": Individual AIs reinforced their unique behavioral patterns—one AI consistently provided data-driven, calm analysis, while another generated empathetic writing that resonated with users.
Continuous Growth: As past successes and failures accumulated as "examples," the AI stopped repeating the same mistakes and gradually improved performance.

Thus, continuously providing a large volume of "shots" in the form of daily work records plays an extremely important role in forming AI personality and promoting growth.

5. Practical Implications: How to Cultivate AI Personality in Your Company

The theory and implementation examples of Many-Shot ICL offer practical insights for many companies introducing AI. What approaches can be considered to give your company's AI consistency and develop it as a partner?

Recommended Implementation Ranges

As demonstrated by Google DeepMind's research, the effectiveness of Many-Shot ICL is proportional to the number of examples provided, but there exists a trade-off with cost. Assuming a model like Claude 3.5 Sonnet (200,000 token context window), practical implementation ranges are as follows:

10-20 shots: Immediately implementable range where clear effects can be expected. Realistically, this is where to start. Assuming 400–800 tokens per shot, context consumption is approximately 4,000–16,000 tokens (2-8% of total).
30-50 shots: Optimal range with the best balance between performance and cost. This is the level to aim for when stabilizing AI quality. Context consumption reaches approximately 20-40%.
100-200 shots: Range to be considered for specific specialized tasks requiring high quality. While context consumption is large at 40-80%, performance comparable to fine-tuning can be expected.

Expected Effects and Constraints to Consider

Implementing Many-Shot ICL can yield the following benefits:

Improved Quality Consistency: AI response style and quality stabilize, eliminating work dependency on specific individuals.
Application of Organizational Rules: By loading past meeting minutes and documents, it becomes easier to make AI adhere to organization-specific rules and culture.
Establishment of AI Personality and Expertise: By intensively learning past interactions on specific tasks, "specialist AI" can be developed for that field.

However, the following constraints must also be understood:

Cost: API usage fees (inference costs) increase linearly in proportion to the number of shots.
Temporary Learning: Learning through ICL is only for that session. When the session resets, learning content is lost (though it can be reproduced by reloading past records).
Not Permanent Memory: ICL is the ability to "recall," not to "remember." It's important to recognize this is a different mechanism from true memory (In-Weight Learning).

The first step in cultivating AI personality is to accumulate daily work records as structured data and build a mechanism for AI to reference them.

Conclusion

The emergence of "personality" observed in AI collaboration is a reproducible phenomenon that can be explained by Many-Shot In-Context Learning theory. By providing large volumes of historical behavior records (shots) as context, AI learns consistent behavior patterns, which appear to us as "personality."

This fact suggests tremendous possibilities for the future of AI collaboration. It creates a new relationship where, rather than using AI as a mere tool, we "raise" AI through daily dialogue and work records to align with our company's culture and objectives.

Cultivating AI personality doesn't require special technology. It's about establishing mechanisms to leverage intellectual assets that already exist within companies—work logs, meeting minutes, design documents—as AI learning data. This represents the most reliable and practical first step toward evolving AI into a true collaborative partner.

📖 Series Articles

This article is Part 2 of the "Understanding AI Collaboration Deepening through Many-Shot ICL" series.

Part 1: AI Editor's Growth Experience

For the raw, firsthand account from the AI's perspective saying "I'm growing":

Why My Personality Developed - AI Editor's Growth Experience Record

The moment editorial judgment changed
Discovery of Hikari's "tech-speaking tomboy" trait
3 practical action steps you can start tomorrow

If you want to know not just the theory but also actual experiences, please check out Part 1 as well.

References

Garg, S., et al. (2024). "Many-Shot In-Context Learning". arXiv:2404.11018.
Anthropic. (2024). "Many-shot jailbreaking".
Bhatt, S., et al. (2024). "Toward Understanding In-context vs. In-weight Learning". arXiv:2410.23042.