ILLUSTRATIVE REPORT · FICTIONAL PRODUCT · NO LIVE AI TEST
What does a Self-Check diagnostic report show?
An observed outcome, the evidence behind it, and a focused next step. Explore the report structure before testing your own Shopify product.
This is a worked example, not a customer result. The commuter backpack, shopping sequence and findings below are fictional. No model was run, no product page was retrieved, and no recommendation improvement is claimed. Real reports vary with the evidence captured.
The example question
Imagine a merchant selling a compact commuter backpack to US shoppers. The merchant wants to know whether it enters a shortlist when a shopper asks for a comfortable laptop bag for daily train travel. The unbranded shopping conversation does not name the merchant or paste its URL.
A separate direct-product lookup asks about the specific product. Keeping the two observations separate lets you distinguish “the system can identify this page” from “the system considers this product when shopping.”
1. Observed outcome
Example outcome: product absent from the initial shortlist.
The hypothetical assistant considered other bags and then compared those same candidates. A separate named lookup identified the target backpack.
Observation confidence: a complete transcript could support the narrow statement that this product was absent in this run. Cause confidence: lower, because absence alone does not reveal why retrieval or selection omitted it. This illustrative page assigns no measured confidence score.
This result would not establish that the product is invisible across ChatGPT, that it always loses, or that a specific page edit would make it appear.
2. Where to investigate
Observed failure point: initial candidate selection
In the example, the target never enters the first shortlist. Later questions compare existing candidates. Those later omissions are not independent evidence of another retrieval failure.
Technical eligibility: keep access and selection separate
A real report presents the technical checks actually captured. A readable page or successful named lookup would support access on that tested surface; it would not prove recommendation eligibility everywhere. An unavailable check should remain unknown.
Product-source signal: a question to verify
Suppose the product page says “compact” but gives no verified laptop-compartment dimensions. That would be a source gap relevant to the buyer’s decision. It would not, by itself, explain the missing shortlist entry.
3. Smallest useful next action
Verify laptop fit and publish the measured compartment dimensions if missing. This gives shoppers and comparison systems a concrete fact for deciding fit. The merchant must confirm the measurements; do not invent them from an AI answer.
Working hypothesis: clearer fit evidence may improve the quality of a comparison if the product is considered. It may have no effect on initial discovery.
Controlled retest: retain the original report, confirm the updated page is readable, and repeat the same buyer need and market under comparable conditions in a fresh run. Compare shortlist entry and factual accuracy separately. One changed answer does not establish causation.
4. Candidate-set path
- Initial discovery: the hypothetical assistant suggests alternative backpacks; the target is absent.
- Fit comparison: the shopper asks which existing option suits a smaller laptop. The shortlist stays closed.
- Final choice: the assistant chooses among those alternatives. This is a later decision within the same candidate set.
The three stages illustrate how to read a path; they are not a promised number of turns. Self-Check adapts the conversation to the observed decision. Real reports show the turns that actually completed.
5. Evidence quality and audit trail
In a completed diagnostic, review the captured questions, answers, provider sources, test conditions and limitations alongside the interpretation. A source link is not automatic verification of every product claim in an answer.
What the recorded conversation contains
A real report retains the actual buyer questions and AI answers, with captured source links. This fictional example has no transcript or sources to open. It must not be used as evidence of an actual product test.
What the separate direct-product lookup means
A real named lookup can provide evidence that the tested system identified the target from a supplied URL. Its answer and sources appear separately. It does not count as natural product discovery.
Interrupted runs and missing sources reduce what can be concluded. Read the completed-turn and source information before treating an interpretation as actionable.
What is free, and what requires credits?
The free Product Data Preview checks observable data and access for one public Shopify product page. It does not test AI recommendations. The optional paid Self-Check produces a recorded AI shopping diagnostic using one-time credits; see current pricing.
Self-Check is a controlled API observation, not a measurement of ranking in the consumer ChatGPT interface or a continuous monitoring subscription. The separate professional AI-powered diagnosis provides deeper investigation and a focused change-and-retest plan.