How we measure what LeLoop earns you

The method behind "LeLoop earned you $X"

1. A comparison group that never sees a popup

• A small share of your new visitors (10% by default) never sees a LeLoop popup. Everyone else sees your popups as usual.
• Who is in which group is decided at random, the first time a visitor arrives, and stays the same for that browser. It's decided with a secret key, so nobody can choose their own group.
• Visitors who had already seen one of your popups in the 13 months before their first visit in the measurement aren't counted on either side, because they can't stand for "never saw a popup".
• Each new measurement counts only new visitors. Everyone from an earlier measurement that began in the past 13 months and 13 weeks sits it out, so both groups start on the same footing.
• Where your store asks shoppers for cookie consent, only shoppers who allow it are counted: LeLoop does nothing for the others, and a shopper who allows analytics but not marketing sees your popups but isn't counted, because their orders can't be matched.

2. Real orders, matched the same way for both groups

• We use your real paid orders, sent to us and signed by Shopify. Numbers sent from a web page can't get in.
• An order counts for a visitor when their own shopping cart carries their (anonymous) visitor tag, for both groups alike. A popup discount code on its own doesn't count, because only shoppers who saw a popup can have one, and that would tilt the comparison in our favour.
• Only orders in your store's main currency, placed after the visitor's first visit, are included.

3. From the difference to a dollar amount

We compare what each group spends per visitor. The extra spend per visitor, multiplied by the visitors who could see your popups, is what LeLoop earned you.

Because shoppers vary, one number isn't enough. We work out a range by simulating 4,000 plausible versions of your data, allowing for how likely people are to buy and for lopsided order sizes (a few big orders can swing an average).

4. When we make a claim, and when we don't

• We only say "LeLoop earned you about $X" when we're 99% sure the true amount is above zero. We also show the range: at least this much, up to that much.
• If the range still includes zero, we say "Too early to call" instead of guessing.
• If your popups seem to be costing sales, we tell you that too, with suggestions.
• There's no estimate at all until each group has at least 200 visitors and 5 buyers.

5. Weekly updates, then a final answer

Look at a noisy number often enough and sooner or later chance makes it look like a result. So the verdict is worked out once a week and kept until the next update; only the visitor counts update live.

Each measurement runs 13 weeks. The week-13 result is final, and from then on nobody is held back from your popups. You can start a new measurement whenever you like. Changing the size of the comparison group also starts a fresh one, because mixing periods with different sizes would distort the result.

6. Tested before we trusted it

We ran the method on thousands of simulated stores where we knew the true answer:

• Where popups made no difference and the result was checked every week for three months, LeLoop claimed earnings in only 3–6% of stores.
• The true amount fell inside our range 97–99% of the time.
• The honest limit: proving a small improvement takes many visitors. With the 10% comparison group, a 30% increase in sales was confirmed in 30% of simulated stores after 40,000 visitors, and in 94% after 200,000. Until then, the dashboard says "too early" rather than guess.

7. What it doesn't capture

• Shoppers who switch devices: a new browser looks like a new visitor.
• Refunds: we count paid orders.

How this differs from "revenue from popups"

Your dashboard also shows revenue from popups, which credits an order to the popup a shopper saw last (in the past 13 months). That's useful for comparing your popups with each other, but it can't tell you whether those shoppers would have bought anyway. The comparison group is how we answer that.