Mean pairwise overlap of long answer words. Median 35%.
Repeated observation analysis
Three runs, same prompts: what stayed stable?
A measured comparison of 120 published answers from 3 AI products. Published panel coverage is 120/171; incomplete platform cells are not treated as observed results.
Mean Jaccard overlap of cited domains. Median 100%.
Series with at least one cited domain visible in all three runs.
270 unique domains across 120 pairwise run comparisons.
Platform completeness stays explicit
The public analysis counts published observations only. Captured drafts remain excluded until the complete repeat series has been reviewed, imported and published.
0 captured drafts · 0 pending
18 captured drafts · 33 pending
0 captured drafts · 0 pending
Repeated-run results by product
Metrics include only series with all three published runs. Citation mean is the average number of visible domains per run, not a ranking score.
| Product | Complete series | Answer overlap | Domain overlap | Citation mean | Recurring domain |
|---|---|---|---|---|---|
| Perplexity60 published runs | 20 | 39% | 88% | 9.8 | 19/20 |
| Microsoft Copilot9 published runs | 3 | 20% | 2% | 5.6 | 0/3 |
| ChatGPT51 published runs | 17 | 34% | 49% | 1.5 | 4/17 |
Same question, different product
A comparison appears only when both products have three published runs for the exact same fixed question. Similarity is averaged across all cross-platform run pairs.
| Question and platforms | Answer overlap | Domain overlap | Shared domains | Citation mean |
|---|---|---|---|---|
| Do AI search engines prefer official documentation?ChatGPT vs Perplexityofficial-documentation-preference | 8% | 0% | None | 0.0 / 10.0 |
| How can a new website get cited by ChatGPT?ChatGPT vs Perplexityget-cited-chatgpt | 23% | 0% | None | 1.3 / 10.0 |
| How can a new website get cited by Perplexity?ChatGPT vs Perplexityget-cited-perplexity | 19% | 0% | None | 1.7 / 10.0 |
| How can a publisher verify that an AI answer cited its source correctly?ChatGPT vs Perplexityverify-ai-citation | 9% | 0% | None | 1.3 / 10.0 |
| How do AI search engines choose comparison pages as sources?ChatGPT vs Perplexitycomparison-page-citations | 9% | 0% | None | 0.3 / 10.0 |
| How do AI search engines choose which sources to cite?Microsoft Copilot vs Perplexityhow-ai-search-selects-sources | 19% | 0% | None | 5.3 / 5.3 |
| How do AI search engines decide whether information is fresh enough?ChatGPT vs Perplexitysource-freshness | 9% | 0% | None | 2.3 / 10.0 |
| How do source changes affect future AI citations?ChatGPT vs Perplexitysource-change-answer-drift | 9% | 0% | None | 0.7 / 10.0 |
| How does ChatGPT Search choose the sources it cites?ChatGPT vs Perplexitychatgpt-search-source-selection | 8% | 0% | None | 2.0 / 10.0 |
| How does Google AI Mode choose the sources it cites?ChatGPT vs Perplexitygoogle-ai-mode-source-selection | 10% | 0% | None | 4.7 / 10.0 |
| How should a B2B SaaS company measure AI search visibility?ChatGPT vs Perplexityb2b-saas-visibility | 11% | 0% | None | 5.0 / 10.0 |
| How should a new website measure citations in ChatGPT and Perplexity?Microsoft Copilot vs Perplexitymeasure-chatgpt-citations | 16% | 1% | blog.hubspot.com | 6.3 / 10.0 |
| How should a publisher audit an AI-generated answer?ChatGPT vs Perplexityaudit-ai-answer | 9% | 0% | None | 2.7 / 10.0 |
| How should a team track AI citations over time?ChatGPT vs Perplexitytrack-citations-over-time | 8% | 0% | None | 0.0 / 10.0 |
| What causes inaccurate AI citations?ChatGPT vs Perplexityinaccurate-ai-citations | 10% | 0% | None | 0.0 / 10.0 |
| What evidence makes a product page citable by AI search?ChatGPT vs Perplexityproduct-page-evidence | 8% | 0% | None | 1.0 / 10.0 |
| What is the difference between an AI mention and an AI citation?ChatGPT vs Perplexitymention-vs-citation | 9% | 0% | None | 0.0 / 10.0 |
| What makes a web page more likely to be cited by AI search?Microsoft Copilot vs Perplexitycitation-worthy-pages | 10% | 6% | blog.hubspot.com, developers.google.com, searchable.com | 5.0 / 11.0 |
| Which pages should a local business optimize for AI search?ChatGPT vs Perplexitylocal-business-ai-search | 8% | 0% | None | 0.3 / 10.0 |
| Which source types do AI search engines cite most often?ChatGPT vs Perplexitysource-types-cited-most | 24% | 6% | bavior | 3.0 / 10.0 |
What changed across three runs
Answer overlap is a lexical similarity check, not a semantic truth score. Domain overlap compares captured citation hosts. Low overlap means the visible evidence set changed, not necessarily that one answer was wrong.
| Product and question | Captured sources | Answer overlap | Domain overlap | Recurring / single-run domains |
|---|---|---|---|---|
| ChatGPTDo AI search engines prefer official documentation?R1R2R3 | 0 / 0 / 0 | 33% | 100% | 0 recurring0 seen in one run |
| ChatGPTHow can a new website get cited by ChatGPT?R1R2R3 | 1 / 1 / 2 | 38% | 67% | 1 recurring1 seen in one run |
| ChatGPTHow can a new website get cited by Perplexity?R1R2R3 | 1 / 2 / 2 | 38% | 11% | 0 recurring3 seen in one run |
| ChatGPTHow can a publisher verify that an AI answer cited its source correctly?R1R2R3 | 0 / 4 / 0 | 31% | 33% | 0 recurring3 seen in one run |
| ChatGPTHow do AI search engines choose comparison pages as sources?R1R2R3 | 1 / 0 / 0 | 32% | 33% | 0 recurring1 seen in one run |
| ChatGPTHow do AI search engines decide whether information is fresh enough?R1R2R3 | 2 / 3 / 2 | 31% | 42% | 1 recurring2 seen in one run |
| ChatGPTHow do source changes affect future AI citations?R1R2R3 | 0 / 1 / 1 | 27% | 0% | 0 recurring2 seen in one run |
| ChatGPTHow does ChatGPT Search choose the sources it cites?R1R2R3 | 2 / 1 / 3 | 39% | 67% | 1 recurring1 seen in one run |
| ChatGPTHow does Google AI Mode choose the sources it cites?R1R2R3 | 6 / 3 / 5 | 29% | 72% | 3 recurring1 seen in one run |
| ChatGPTHow should a B2B SaaS company measure AI search visibility?R1R2R3 | 2 / 6 / 7 | 39% | 11% | 0 recurring7 seen in one run |
| ChatGPTHow should a publisher audit an AI-generated answer?R1R2R3 | 4 / 4 / 0 | 36% | 11% | 0 recurring4 seen in one run |
| ChatGPTHow should a team track AI citations over time?R1R2R3 | 0 / 0 / 0 | 38% | 100% | 0 recurring0 seen in one run |
| ChatGPTWhat causes inaccurate AI citations?R1R2R3 | 0 / 0 / 0 | 32% | 100% | 0 recurring0 seen in one run |
| ChatGPTWhat evidence makes a product page citable by AI search?R1R2R3 | 3 / 0 / 0 | 32% | 33% | 0 recurring1 seen in one run |
| ChatGPTWhat is the difference between an AI mention and an AI citation?R1R2R3 | 0 / 0 / 0 | 37% | 100% | 0 recurring0 seen in one run |
| ChatGPTWhich pages should a local business optimize for AI search?R1R2R3 | 0 / 0 / 1 | 36% | 33% | 0 recurring1 seen in one run |
| ChatGPTWhich source types do AI search engines cite most often?R1R2R3 | 3 / 3 / 3 | 33% | 20% | 0 recurring3 seen in one run |
| Microsoft CopilotHow do AI search engines choose which sources to cite?R1R2R3 | 4 / 6 / 6 | 25% | 4% | 0 recurring13 seen in one run |
| Microsoft CopilotHow should a new website measure citations in ChatGPT and Perplexity?R1R2R3 | 4 / 9 / 6 | 21% | 0% | 0 recurring18 seen in one run |
| Microsoft CopilotWhat makes a web page more likely to be cited by AI search?R1R2R3 | 3 / 6 / 6 | 14% | 3% | 0 recurring12 seen in one run |
| PerplexityDo AI search engines prefer official documentation?R1R2R3 | 10 / 10 / 10 | 45% | 100% | 10 recurring0 seen in one run |
| PerplexityHow can a new website get cited by ChatGPT?R1R2R3 | 10 / 10 / 10 | 39% | 100% | 10 recurring0 seen in one run |
| PerplexityHow can a new website get cited by Perplexity?R1R2R3 | 10 / 10 / 10 | 34% | 88% | 9 recurring1 seen in one run |
| PerplexityHow can a publisher verify that an AI answer cited its source correctly?R1R2R3 | 10 / 10 / 10 | 32% | 100% | 10 recurring0 seen in one run |
| PerplexityHow do AI search engines choose comparison pages as sources?R1R2R3 | 10 / 10 / 10 | 31% | 100% | 10 recurring0 seen in one run |
| PerplexityHow do AI search engines choose which sources to cite?R1R2R3 Partial source capture | 10 / 3 / 3 | 23% | 14% | 0 recurring10 seen in one run |
| PerplexityHow do AI search engines decide whether information is fresh enough?R1R2R3 | 10 / 10 / 10 | 40% | 100% | 10 recurring0 seen in one run |
| PerplexityHow do source changes affect future AI citations?R1R2R3 | 10 / 10 / 10 | 31% | 100% | 10 recurring0 seen in one run |
| PerplexityHow does ChatGPT Search choose the sources it cites?R1R2R3 | 10 / 10 / 10 | 44% | 100% | 6 recurring0 seen in one run |
| PerplexityHow does Google AI Mode choose the sources it cites?R1R2R3 | 10 / 10 / 10 | 64% | 100% | 10 recurring0 seen in one run |
| PerplexityHow should a B2B SaaS company measure AI search visibility?R1R2R3 | 10 / 10 / 10 | 36% | 100% | 10 recurring0 seen in one run |
| PerplexityHow should a new website measure citations in ChatGPT and Perplexity?R1R2R3 | 10 / 10 / 10 | 30% | 50% | 4 recurring6 seen in one run |
| PerplexityHow should a publisher audit an AI-generated answer?R1R2R3 | 10 / 10 / 10 | 43% | 100% | 10 recurring0 seen in one run |
| PerplexityHow should a team track AI citations over time?R1R2R3 | 10 / 10 / 10 | 47% | 100% | 9 recurring0 seen in one run |
| PerplexityWhat causes inaccurate AI citations?R1R2R3 | 10 / 10 / 10 | 32% | 100% | 9 recurring0 seen in one run |
| PerplexityWhat evidence makes a product page citable by AI search?R1R2R3 | 10 / 10 / 10 | 46% | 100% | 10 recurring0 seen in one run |
| PerplexityWhat is the difference between an AI mention and an AI citation?R1R2R3 | 10 / 10 / 10 | 61% | 100% | 10 recurring0 seen in one run |
| PerplexityWhat makes a web page more likely to be cited by AI search?R1R2R3 Partial source capture | 10 / 20 / 3 | 17% | 13% | 1 recurring16 seen in one run |
| PerplexityWhich pages should a local business optimize for AI search?R1R2R3 | 10 / 10 / 10 | 45% | 100% | 10 recurring0 seen in one run |
| PerplexityWhich source types do AI search engines cite most often?R1R2R3 | 10 / 10 / 10 | 43% | 100% | 9 recurring0 seen in one run |
Where the captured citations came from
Category assignment is rule-based by hostname. It is useful for audit and triage, not a claim about source quality.
Domains seen across the sample
- searchablePractitioner and vendor14 runs
- searchcounselcoPractitioner and vendor9 runs
- developers.google.comOfficial documentation8 runs
- google for developersPractitioner and vendor7 runs
- aofirsPractitioner and vendor6 runs
- blog.hubspotMedia and editorial6 runs
- createandgrowPractitioner and vendor6 runs
- linkedinPractitioner and vendor6 runs
- omniusPractitioner and vendor6 runs
- straightnorthPractitioner and vendor6 runs
- baviorPractitioner and vendor5 runs
- shilaPractitioner and vendor5 runs
- blog.hubspot.comMedia and editorial4 runs
- openai help centerPractitioner and vendor4 runs
Three useful findings
The answers repeatedly made similar arguments, but exact wording varied enough that a single answer should not be treated as a fixed script.
Domain overlap was lower than answer-theme overlap. A brand or page can therefore be absent from one run without proving persistent exclusion.
The panel records Perplexity, Copilot and ChatGPT separately. Missing platform cells remain visible and are never filled with another product's answer.
What this analysis cannot establish
This is a fixed-panel convenience sample from authenticated browser sessions. It cannot estimate all-user behavior, prove a ranking factor or establish causation. Published coverage is currently 120/171; 2 published series contain a run where the source panel was incomplete. Missing or captured-only platform cells are excluded rather than imputed.