research1d
Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories
Researchers found that popular LLMs like Llama-3 and Mistral-7B produce stories with low diversity, often repeating common tropes like 'Elias in the lighthouse'. This issue persists even with fine-tuning and larger model sizes. The study suggests that improving LLM diversity may require new techniques beyond scaling up model parameters.
Key takeaways
- LLMs like Llama-3 and Mistral-7B produce stories with low diversity.
- Low diversity issue persists with fine-tuning and larger model sizes.
- New techniques may be needed to improve LLM diversity.