From September 21 to 23 of this year, a lot of Shopify store owners opened their dashboards and found a different conversion rate waiting for them. For some it jumped. For others it sagged. Not one extra order had come in, and none had gone missing.
What changed was the counting. Shopify rolled out a new way of measuring sessions: it now filters identified bots out of its session reports by default, it no longer cuts a shopper's visit in two at midnight UTC, and it counts some sessions that never loaded a page, like a customer who jumps straight to checkout from a cart link. Conversion rate is orders divided by sessions, so when the bottom of that fraction moved, the number moved with it. Shopify's own help page says a higher or lower rate after the update "isn't automatically good or bad" and tells merchants to treat it as a new baseline.
That is worth sitting with for a second, because conversion rate is the number most owners check first. It is on the home screen. It is what agencies quote, what app vendors promise to raise, and what gets screenshotted into the group chat when it goes up. And in one week it moved for stores across Shopify without the business underneath it changing at all.
It turns out that is not a one-time glitch. It is how the number behaves all the time.
Conversion Rate Counts Buyers. It Has No Idea What They Spent.
Shopify defines conversion rate as the percentage of sessions that ended in a purchase. A hundred visits, two orders, two percent. Google Analytics calls the same idea ecommerce conversion rate, and it works the same way.
Read that definition again and notice what is missing. It does not know whether the order was $18 or $180. It does not know whether you sold it at full price or at 30 percent off. It does not know what the product cost you, what the shipping cost you, or whether the customer sent half of it back. One order is one order. A store that sells a single sticker to every visitor would have a magnificent conversion rate and no business.
That would be a harmless quirk if owners only looked at the number. The trouble is that they act on it. Here are three of the most common changes a store makes to "improve conversion," and each one can push the rate up while the money goes down:
- A sitewide discount. More people buy. Every one of them pays less, and your product cost, packing and shipping did not get a discount.
- A lower free-shipping threshold. Drop it from $75 to $50 and more carts tip over the line. You are now paying to ship smaller orders, which is exactly the math we walked through in Free Shipping Is Not Free.
- Leading with your cheapest product. Put the $19 item in the ads and on the homepage, and it will convert better than the $89 one. Of course it will. It also fills the store with $19 orders.
None of these are bad moves by default. Some of them are exactly right for a given store. The point is that conversion rate cannot tell you which ones worked, because it was never measuring money.
Turn Up Your Ads and Your Conversion Rate Drops. Nothing on Your Site Broke.
The second problem is that conversion rate is mostly a fact about who showed up, not about your store.
Anthony Morgan, who runs the Shopify conversion agency Enavi, put it bluntly on the eCommerce Fastlane podcast: increase your paid social spend on colder audiences and watch your conversion rate over the following week. It falls, sometimes hard, because the people arriving are earlier in their thinking. The site is identical. The visitors are not.
Every store has the same pattern underneath. People who open your email already know you. Returning visitors came back on purpose. Someone searching Google for the exact product you sell is close to buying. Someone who tapped a Meta ad while scrolling past a friend's vacation photos is not. Those groups convert at wildly different rates, and your store-wide conversion rate is just a blend of them, weighted by how many of each you happened to get that week.
Here is what that looks like for an example store doing about 10,000 sessions a week. Most of its traffic comes from Google Search and returning visitors, with a modest Meta budget aimed at new customers:
- Email: 1,000 sessions at 4.5 percent, 45 orders
- Returning and direct: 2,400 sessions at 3.5 percent, 84 orders
- Google Search: 4,600 sessions at 2.5 percent, 115 orders
- Meta, new audiences: 2,000 sessions at 0.8 percent, 16 orders
That is 260 orders from 10,000 sessions, a 2.6 percent conversion rate.
Now the owner decides to grow and adds enough Meta budget to bring in 8,000 more new-audience sessions. Every other channel stays the same, and the new traffic converts at the same 0.8 percent. The store now takes 324 orders from 18,000 sessions. Sixty-four more orders, and a conversion rate of 1.8 percent.
If that owner judges the week by conversion rate, the growth looks like a disaster and the obvious fix is to turn the new ads off, which would push the rate back up and throw away the extra 64 orders. Whether those orders were worth what the ads cost is a real question, and it gets answered with your margin, which we covered in our piece on break-even ROAS. But conversion rate cannot answer it either way. It only tells you the mix changed.
So the first rule is simple: never compare a store-wide conversion rate across two periods with different traffic. If you want to know whether the store itself got better, compare like with like. Email against email, new visitors against new visitors, the same ad channel against itself.
Revenue per Visitor Is Two Numbers You Already Have.
The fix is not a fancy new metric. It is two numbers you already look at, multiplied together.
Revenue per visitor = conversion rate x average order value. Or, if you prefer, total sales divided by sessions. Same answer either way. (Technically it is revenue per session, since that is what your tools count. Adam Pearce, who runs the Shopify agency Blend Commerce, says revenue per session is the number his team judges its own work by, precisely because it does not care which channel gets the credit.)
It works because it catches both halves of what a store does: how many people buy, and how much each of them spends. A change that wins buyers by shrinking orders shows up immediately. So does a change that loses a few buyers but makes every order bigger, which is why the order-value levers in You Don't Need More Traffic. You Need a Bigger Cart. only show their worth on this number.
Take a price increase, the change owners are most afraid to make. Say a store sells its best product at $40 and converts 2.5 percent of 10,000 sessions. That is 250 orders and $10,000 in sales, or $1.00 per visitor. The owner raises the price to $46. Fewer people buy, and conversion drops to 2.2 percent, or 220 orders. On the conversion rate dashboard that is a 12 percent decline and a reason to panic.
Revenue per visitor says otherwise: 220 orders at $46 is $10,120, or about $1.01 per visitor. The store took in slightly more money from the same traffic, while packing and shipping 30 fewer boxes.
Where to find it:
- Shopify: your Analytics overview shows total sales and sessions for any date range. Divide one by the other. For a channel-by-channel view, the Total sales by referrer report and the Sessions by referrer report give you the two halves for each source.
- GA4: build a free-form exploration with Session default channel group as the row and Sessions and Total revenue as the values, then divide. It takes about ten minutes the first time.
Pick one and stick with it. The two tools will not agree to the penny, and switching between them mid-comparison will invent changes that did not happen.
Profit per Visitor Is the Number Your Bank Account Reads.
Revenue per visitor is a big step up. It still counts the whole price of every order, though, and you do not keep the whole price.
Davie Fogarty, who founded the wearable-blanket brand The Oodie, makes this point in his walkthrough of Shopify conversion work: once you have revenue per visitor, subtract your costs and you get profit per visitor, and that is the one that matters. An improvement that does not move it, he says, is just noise.
You do not need an accountant for this. For each order, take the sale price and subtract what it costs to fill: the product itself, packing and shipping, card processing fees, and any discount you gave. What is left, divided by sessions, is profit per visitor. If you have already worked out your contribution margin for setting ad targets, you have done the hard part. (If you have not, our break-even ROAS piece walks through it.)
Go back to that price increase. Say the product costs $16, packing and shipping cost $7 an order, and card fees run about 3 percent.
- At $40: each order leaves about $15.80. Times 250 orders, that is $3,950, or about 40 cents per visitor.
- At $46: each order leaves about $21.62. Times 220 orders, that is about $4,756, or about 48 cents per visitor.
Conversion rate fell 12 percent. Revenue per visitor barely moved. Profit per visitor rose about 20 percent. That is the same change, read three ways, and only one of the three readings tells the owner what happened to the business.
A Discount Popup Wins on Conversion Rate and Loses on Profit.
Now run the opposite experiment, the one most stores actually run.
A store averages $62 an order and converts 2 percent of 10,000 weekly sessions: 200 orders, $12,400. Its owner launches 20 percent off sitewide. Conversion climbs to 2.6 percent, which is 260 orders, and because everything is cheaper, the average order drops to $50. Sales come in at $13,000.
Every number on the dashboard went the right way. Conversion rate is up 30 percent. Revenue is up $600. Revenue per visitor rose from $1.24 to $1.30.
Now subtract the costs. The product in an average order costs the store $20 whatever price it sells at, packing and shipping run $8, and card fees are about 3 percent.
- Full price: about $32.14 kept per order, times 200 orders, is about $6,430. That is 64 cents per visitor.
- On sale: about $20.50 kept per order, times 260 orders, is $5,330. That is 53 cents per visitor.
The busiest week of the month made about $1,100 less than an ordinary one. Sixty more boxes went out the door to earn it.
This is why it matters which number you use to judge which change. A rough guide:
- Changes that remove friction (a faster page, fewer checkout steps, Apple Pay, clearer delivery dates): conversion rate is fine. These rarely change what people buy or what you charge, so more buyers really is more money.
- Changes to price, offers or thresholds (discounts, popups, free-shipping lines, bundles, tiered pricing): judge on profit per visitor. These deliberately trade order value or margin for buyers, and only a profit number can tell you if the trade paid.
- Changes to what you promote (which product leads the ads, the homepage or the email): revenue per visitor at minimum, profit per visitor if your margins differ much from product to product.
None of this is an argument against discounts. It is an argument against judging them with a number that cannot see what they cost. If a sale is building something, like new customers who come back at full price, then the case for it lives in your repeat-order numbers. We wrote about that, and about building an offer that does not depend on a price cut, in You Discount Because You Never Built an Offer.
Your Store Is Too Small for an A/B Test. It Is Big Enough to Compare Two Months.
The textbook way to settle any of this is a split test: show half your visitors the old version and half the new, and let the data decide. For most small stores, that is not realistic. Pearce draws the line at about 10,000 visits a month, saying below that you cannot learn anything from a test beyond anecdotes, and he puts proper testing closer to 50,000. Other practitioners set the bar differently, but nobody sets it low. We dug into why small-sample winners so often turn out to be luck in You Won the A/B Test. You Probably Just Got Lucky.
A small store can still make a fair comparison. It just has to be disciplined about it.
- Compare matching periods. Four weeks before the change against four weeks after, not a quiet week against a payday week. Avoid stretches that cross a holiday, a big email send or a sale.
- Compare by channel, not store-wide. If you added ad spend during the same month, the store-wide number is contaminated. Look at the channels you did not touch, like email and returning visitors, and see whether their revenue and profit per visitor moved.
- Change one thing at a time. If the price went up the same week the new product photos went live, you will never know which one did it.
- Mind the September line. If your before period falls ahead of Shopify's September 21 session change, your session counts were measured differently. Compare orders and sales directly, or pick periods that both sit after it.
This will not give you laboratory certainty. It will stop you from killing a good change because a dashboard tile turned red.
Profit per Visitor, Checked Every Monday, Catches a Bad Change Before Black Friday.
Black Friday is about five weeks away, and it is the season when stores make the most changes at once: new offers, lower thresholds, bigger ad budgets, popups everywhere. It is also when conversion rate is easiest to misread, because a sale will lift it no matter what else is going on.
So set up the simple version now, while you have a normal month to measure against. One sheet, one row per week, one column group per channel:
- Sessions
- Orders
- Sales
- Your cost to fill those orders (product, shipping, fees, discounts)
- Revenue per visitor and profit per visitor, calculated from the four above
Conversion rate and average order value still belong on the sheet. They explain why profit per visitor moved, but neither one is the verdict.
Fill it in every Monday. It takes fifteen minutes once you know where the numbers live. When you try something, you will see within a few weeks whether it paid, in dollars, by channel, without waiting for a test you do not have the traffic to run.
And when someone, whether it is an app vendor, an agency or a podcast guest, promises to double your conversion rate, ask them the only follow-up that matters: what will it do to profit per visitor?
This is the way we look at store performance when we work with online stores. We start from what each visitor is worth to the business, then point the ad budget at the channels where that number is highest. If you would like a second set of eyes on your numbers, our ecommerce marketing team does this every week. And if you would rather build the Monday sheet yourself, everything you need is above.
