The Core Question

The fundamental question is: How well do LLM-simulated policy attitudes align with actual survey data? And critically, which types of questions show better alignment, and how much does this vary across different models?

Approach

Our technical pipeline involves:

1. Retrieval — matching survey questions to relevant demographic-conditioned prompts
2. LLM Simulation — generating persona-conditional responses at scale
3. Statistical Validation — comparing simulated distributions against ground-truth survey results

Early Observations

While we’re still in the early stages, a few patterns are emerging:

• LLMs tend to capture broad demographic trends (e.g., age-based opinion gaps) reasonably well
• But they often underestimate variance — the simulated responses are more “centrist” than real survey data
• Different models (and model sizes) show meaningfully different levels of alignment

Why This Matters

If LLMs can reliably simulate population attitudes, this could complement traditional survey methods — especially for rapid pre-testing of questionnaires or exploring hypothetical scenarios. But if the alignment is poor or systematically biased, using LLMs as “synthetic respondents” could lead to misleading conclusions.

I’ll share more detailed results as the project progresses. Stay tuned!

This project is part of my coursework in the Big Data Communication program at Nanjing University.

The Framework

The knowledge base is organized around three layers:

1. Chapter Notes (14 topics) — from search marketing and e-commerce to AI marketing and data privacy
2. Concept Notes (19 entries) — deep dives into specific mechanisms like RFM models, recommendation systems, and private domain traffic
3. Mechanism Notes — cross-cutting themes that span multiple chapters, like “Cookie deprecation → first-party data rise”

Human-AI Collaboration

One design principle I stuck to: the human perspective leads, AI supports. Each note follows a format where:

• 🙋 My questions drive the exploration
• 🤖 AI responses provide technical depth
• 📎 Cross-references link related concepts

This isn’t about letting AI write the knowledge base — it’s about using AI as a thought partner to probe deeper into concepts I’m genuinely curious about.

The Graph View

The most satisfying part is watching the Obsidian graph view grow organically over time. What began as disconnected notes gradually formed clusters, then bridges between clusters. The visual representation makes it obvious which concepts are central hubs and which are peripheral — a kind of intellectual cartography.

Tools Used

• Obsidian for note-taking and graph visualization
• Mermaid.js for inline diagrams
• Claude as an AI discussion partner
• Python for data processing and analysis

If you’re interested in knowledge management or computational approaches to learning, I’d love to connect and chat!