Repeated observation analysis

Three runs, same prompts: what stayed stable?

A measured comparison of 120 published answers from 3 AI products. Published panel coverage is 120/171; incomplete platform cells are not treated as observed results.

120 observations40 question series3 AI products120/171 published slots
Lexical answer overlap36%

Mean pairwise overlap of long answer words. Median 35%.

Captured-domain overlap65%

Mean Jaccard overlap of cited domains. Median 100%.

Recurring-domain series23/40

Series with at least one cited domain visible in all three runs.

Captured citation rows718

270 unique domains across 120 pairwise run comparisons.

Published coverage

Platform completeness stays explicit

The public analysis counts published observations only. Captured drafts remain excluded until the complete repeat series has been reviewed, imported and published.

Perplexity60/60

0 captured drafts · 0 pending

Microsoft Copilot9/60

18 captured drafts · 33 pending

ChatGPT51/51

0 captured drafts · 0 pending

Platform-level stability

Repeated-run results by product

Metrics include only series with all three published runs. Citation mean is the average number of visible domains per run, not a ranking score.

ProductComplete seriesAnswer overlapDomain overlapCitation meanRecurring domain
Perplexity60 published runs2039%88%9.819/20
Microsoft Copilot9 published runs320%2%5.60/3
ChatGPT51 published runs1734%49%1.54/17
Cross-platform comparison

Same question, different product

A comparison appears only when both products have three published runs for the exact same fixed question. Similarity is averaged across all cross-platform run pairs.

Question and platformsAnswer overlapDomain overlapShared domainsCitation mean
Do AI search engines prefer official documentation?ChatGPT vs Perplexityofficial-documentation-preference8%0%None0.0 / 10.0
How can a new website get cited by ChatGPT?ChatGPT vs Perplexityget-cited-chatgpt23%0%None1.3 / 10.0
How can a new website get cited by Perplexity?ChatGPT vs Perplexityget-cited-perplexity19%0%None1.7 / 10.0
How can a publisher verify that an AI answer cited its source correctly?ChatGPT vs Perplexityverify-ai-citation9%0%None1.3 / 10.0
How do AI search engines choose comparison pages as sources?ChatGPT vs Perplexitycomparison-page-citations9%0%None0.3 / 10.0
How do AI search engines choose which sources to cite?Microsoft Copilot vs Perplexityhow-ai-search-selects-sources19%0%None5.3 / 5.3
How do AI search engines decide whether information is fresh enough?ChatGPT vs Perplexitysource-freshness9%0%None2.3 / 10.0
How do source changes affect future AI citations?ChatGPT vs Perplexitysource-change-answer-drift9%0%None0.7 / 10.0
How does ChatGPT Search choose the sources it cites?ChatGPT vs Perplexitychatgpt-search-source-selection8%0%None2.0 / 10.0
How does Google AI Mode choose the sources it cites?ChatGPT vs Perplexitygoogle-ai-mode-source-selection10%0%None4.7 / 10.0
How should a B2B SaaS company measure AI search visibility?ChatGPT vs Perplexityb2b-saas-visibility11%0%None5.0 / 10.0
How should a new website measure citations in ChatGPT and Perplexity?Microsoft Copilot vs Perplexitymeasure-chatgpt-citations16%1%blog.hubspot.com6.3 / 10.0
How should a publisher audit an AI-generated answer?ChatGPT vs Perplexityaudit-ai-answer9%0%None2.7 / 10.0
How should a team track AI citations over time?ChatGPT vs Perplexitytrack-citations-over-time8%0%None0.0 / 10.0
What causes inaccurate AI citations?ChatGPT vs Perplexityinaccurate-ai-citations10%0%None0.0 / 10.0
What evidence makes a product page citable by AI search?ChatGPT vs Perplexityproduct-page-evidence8%0%None1.0 / 10.0
What is the difference between an AI mention and an AI citation?ChatGPT vs Perplexitymention-vs-citation9%0%None0.0 / 10.0
What makes a web page more likely to be cited by AI search?Microsoft Copilot vs Perplexitycitation-worthy-pages10%6%blog.hubspot.com, developers.google.com, searchable.com5.0 / 11.0
Which pages should a local business optimize for AI search?ChatGPT vs Perplexitylocal-business-ai-search8%0%None0.3 / 10.0
Which source types do AI search engines cite most often?ChatGPT vs Perplexitysource-types-cited-most24%6%bavior3.0 / 10.0
Question-level comparison

What changed across three runs

Answer overlap is a lexical similarity check, not a semantic truth score. Domain overlap compares captured citation hosts. Low overlap means the visible evidence set changed, not necessarily that one answer was wrong.

Product and questionCaptured sourcesAnswer overlapDomain overlapRecurring / single-run domains
ChatGPTDo AI search engines prefer official documentation?R1R2R30 / 0 / 033%100%0 recurring0 seen in one run
ChatGPTHow can a new website get cited by ChatGPT?R1R2R31 / 1 / 238%67%1 recurring1 seen in one run
ChatGPTHow can a new website get cited by Perplexity?R1R2R31 / 2 / 238%11%0 recurring3 seen in one run
ChatGPTHow can a publisher verify that an AI answer cited its source correctly?R1R2R30 / 4 / 031%33%0 recurring3 seen in one run
ChatGPTHow do AI search engines choose comparison pages as sources?R1R2R31 / 0 / 032%33%0 recurring1 seen in one run
ChatGPTHow do AI search engines decide whether information is fresh enough?R1R2R32 / 3 / 231%42%1 recurring2 seen in one run
ChatGPTHow do source changes affect future AI citations?R1R2R30 / 1 / 127%0%0 recurring2 seen in one run
ChatGPTHow does ChatGPT Search choose the sources it cites?R1R2R32 / 1 / 339%67%1 recurring1 seen in one run
ChatGPTHow does Google AI Mode choose the sources it cites?R1R2R36 / 3 / 529%72%3 recurring1 seen in one run
ChatGPTHow should a B2B SaaS company measure AI search visibility?R1R2R32 / 6 / 739%11%0 recurring7 seen in one run
ChatGPTHow should a publisher audit an AI-generated answer?R1R2R34 / 4 / 036%11%0 recurring4 seen in one run
ChatGPTHow should a team track AI citations over time?R1R2R30 / 0 / 038%100%0 recurring0 seen in one run
ChatGPTWhat causes inaccurate AI citations?R1R2R30 / 0 / 032%100%0 recurring0 seen in one run
ChatGPTWhat evidence makes a product page citable by AI search?R1R2R33 / 0 / 032%33%0 recurring1 seen in one run
ChatGPTWhat is the difference between an AI mention and an AI citation?R1R2R30 / 0 / 037%100%0 recurring0 seen in one run
ChatGPTWhich pages should a local business optimize for AI search?R1R2R30 / 0 / 136%33%0 recurring1 seen in one run
ChatGPTWhich source types do AI search engines cite most often?R1R2R33 / 3 / 333%20%0 recurring3 seen in one run
Microsoft CopilotHow do AI search engines choose which sources to cite?R1R2R34 / 6 / 625%4%0 recurring13 seen in one run
Microsoft CopilotHow should a new website measure citations in ChatGPT and Perplexity?R1R2R34 / 9 / 621%0%0 recurring18 seen in one run
Microsoft CopilotWhat makes a web page more likely to be cited by AI search?R1R2R33 / 6 / 614%3%0 recurring12 seen in one run
PerplexityDo AI search engines prefer official documentation?R1R2R310 / 10 / 1045%100%10 recurring0 seen in one run
PerplexityHow can a new website get cited by ChatGPT?R1R2R310 / 10 / 1039%100%10 recurring0 seen in one run
PerplexityHow can a new website get cited by Perplexity?R1R2R310 / 10 / 1034%88%9 recurring1 seen in one run
PerplexityHow can a publisher verify that an AI answer cited its source correctly?R1R2R310 / 10 / 1032%100%10 recurring0 seen in one run
PerplexityHow do AI search engines choose comparison pages as sources?R1R2R310 / 10 / 1031%100%10 recurring0 seen in one run
PerplexityHow do AI search engines choose which sources to cite?R1R2R3 Partial source capture10 / 3 / 323%14%0 recurring10 seen in one run
PerplexityHow do AI search engines decide whether information is fresh enough?R1R2R310 / 10 / 1040%100%10 recurring0 seen in one run
PerplexityHow do source changes affect future AI citations?R1R2R310 / 10 / 1031%100%10 recurring0 seen in one run
PerplexityHow does ChatGPT Search choose the sources it cites?R1R2R310 / 10 / 1044%100%6 recurring0 seen in one run
PerplexityHow does Google AI Mode choose the sources it cites?R1R2R310 / 10 / 1064%100%10 recurring0 seen in one run
PerplexityHow should a B2B SaaS company measure AI search visibility?R1R2R310 / 10 / 1036%100%10 recurring0 seen in one run
PerplexityHow should a new website measure citations in ChatGPT and Perplexity?R1R2R310 / 10 / 1030%50%4 recurring6 seen in one run
PerplexityHow should a publisher audit an AI-generated answer?R1R2R310 / 10 / 1043%100%10 recurring0 seen in one run
PerplexityHow should a team track AI citations over time?R1R2R310 / 10 / 1047%100%9 recurring0 seen in one run
PerplexityWhat causes inaccurate AI citations?R1R2R310 / 10 / 1032%100%9 recurring0 seen in one run
PerplexityWhat evidence makes a product page citable by AI search?R1R2R310 / 10 / 1046%100%10 recurring0 seen in one run
PerplexityWhat is the difference between an AI mention and an AI citation?R1R2R310 / 10 / 1061%100%10 recurring0 seen in one run
PerplexityWhat makes a web page more likely to be cited by AI search?R1R2R3 Partial source capture10 / 20 / 317%13%1 recurring16 seen in one run
PerplexityWhich pages should a local business optimize for AI search?R1R2R310 / 10 / 1045%100%10 recurring0 seen in one run
PerplexityWhich source types do AI search engines cite most often?R1R2R310 / 10 / 1043%100%9 recurring0 seen in one run
Source mix

Where the captured citations came from

Practitioner and vendor629 source appearances · 93%
Media and editorial23 source appearances · 3%
Official documentation9 source appearances · 1%
Community and discussion8 source appearances · 1%
Research and standards4 source appearances · 1%

Category assignment is rule-based by hostname. It is useful for audit and triage, not a claim about source quality.

Most recurrent hosts

Domains seen across the sample

  1. searchablePractitioner and vendor14 runs
  2. searchcounselcoPractitioner and vendor9 runs
  3. developers.google.comOfficial documentation8 runs
  4. google for developersPractitioner and vendor7 runs
  5. aofirsPractitioner and vendor6 runs
  6. blog.hubspotMedia and editorial6 runs
  7. createandgrowPractitioner and vendor6 runs
  8. linkedinPractitioner and vendor6 runs
  9. omniusPractitioner and vendor6 runs
  10. straightnorthPractitioner and vendor6 runs
  11. baviorPractitioner and vendor5 runs
  12. shilaPractitioner and vendor5 runs
  13. blog.hubspot.comMedia and editorial4 runs
  14. openai help centerPractitioner and vendor4 runs
What the evidence supports

Three useful findings

01 · Wording is not stable

The answers repeatedly made similar arguments, but exact wording varied enough that a single answer should not be treated as a fixed script.

02 · Source sets move

Domain overlap was lower than answer-theme overlap. A brand or page can therefore be absent from one run without proving persistent exclusion.

03 · Platform coverage is not interchangeable

The panel records Perplexity, Copilot and ChatGPT separately. Missing platform cells remain visible and are never filled with another product's answer.

Evidence boundary

What this analysis cannot establish

This is a fixed-panel convenience sample from authenticated browser sessions. It cannot estimate all-user behavior, prove a ranking factor or establish causation. Published coverage is currently 120/171; 2 published series contain a run where the source panel was incomplete. Missing or captured-only platform cells are excluded rather than imputed.