You installed a points app about a year ago. The dashboard is very pleased with itself. It says members spend three times what everyone else does, and it credits the program with a tidy share of your revenue.
Here is the question the dashboard is not built to answer: would those people have bought anyway?
For most small stores, a lot of them would have. The customers who join a loyalty program first are the ones who were already coming back. They were buying from you before the app existed. Now they buy on the same schedule and collect a few dollars off each time. That is not loyalty the program created. It is a discount you started paying on orders you were already getting.
So here is the direct answer to "is my loyalty program working?" An ecommerce loyalty program earns its keep only if it changes what customers do. That means more first-time buyers placing a second order, shorter gaps between orders, or bigger carts than those same people would have produced without it. Everything else is a rebate with a nice interface. The rest of this piece is how to tell which one you have, what it is really costing you, and how to point it at the behavior you actually need.
Your Loyalty Dashboard Counts Every Order a Member Places
Open the reporting in almost any loyalty app and you will find some version of "revenue from members" or "loyalty-driven sales." Read the definition carefully. In most cases it is exactly what it sounds like: the total value of every order placed by someone enrolled in the program.
That number includes the customer who has bought the same coffee from you every five weeks for three years. It includes the gift buyer who happened to create an account at checkout, which enrolled them automatically. It includes everyone the program was simply standing next to when they bought.
This is not a knock on the software. A reporting screen can only show what it can see, and what it can see is who is a member and what they spent. What it cannot see is the version of your store where the program never existed. That missing comparison is the entire question.
Peter Fader, a marketing professor at Wharton who has spent his career on customer value, makes the point bluntly: most companies launch loyalty programs because competitors have them, then measure them by how much was sold "through" the program rather than by whether customers stayed longer or bought more because of it. Big companies do this. A two-person store with a $79-a-month app and no analyst is even more likely to.
Your Best Customers Joined First. That Is Why the Numbers Look So Good.
Think about who signs up when you launch a program. Not a random slice of your customers. The people who shop with you most often see the sign-up prompt most often, care most about the reward, and have the most points to earn. Your regulars enroll in the first month. Your one-time buyers mostly never notice it exists.
So when the app compares members to non-members, it is comparing your best customers to everyone else. Of course members spend more. They spent more before they joined. The comparison measures who joined, not what the program did.
The same flaw hides inside advice from the loyalty industry itself. One app founder, asked on a Shopify podcast how merchants can tell if a program is working, suggested comparing customers who redeem points against customers who do not. On the same episode he noted that paid members spend about three times as much as other returning customers. Both comparisons have the same problem: the people who redeem and the people who pay for a membership are the people who already liked you most. You would expect them to spend more if the program did nothing at all.
The concept worth borrowing comes from Len Llaguno at Kyros, a firm that does the accounting behind large loyalty programs. He calls it incremental value: the change in a customer's lifetime value that happens because of the program, measured after you subtract what the points cost you. Not "what do members spend." "What do members spend that they would not have, minus what we gave back to get it." That is the number a program has to beat, and it is the only one worth tracking.
Three Honest Ways to Find Out Whether Your Points Program Works
You do not need a data team for any of these. You need your order export, a spreadsheet, and the discipline to compare like with like.
Hold a slice of new customers out of the program
This is the cleanest test, and on Smile, one of the most widely used loyalty apps on Shopify, it is a built-in setting: you can exclude customers by their Shopify customer tag, and excluded customers do not see the program at all. Other apps offer similar exclusion rules, so check yours.
The setup: tag a random fifth of new customers as excluded. You do not need anything clever to make it random. Tagging every new customer whose first order number ends in 0 or 5 works fine, and Shopify Flow can apply the tag automatically. Then wait. After 90 days, and again after 180, compare the share of each group that placed a second order and how much each group spent in total.
If the program is changing behavior, the members should pull clearly ahead. If the two groups look the same, the program is rebating orders that were coming anyway.
A fair worry for a small store is volume. If you get 200 new customers a month, your excluded group grows by about 40 a month, which is not much. That is fine, for a reason that becomes obvious in the cost section below: a program has to produce a fairly large lift just to pay for itself. A small store's test only needs to be sensitive enough to spot a big effect, because a small effect is not worth paying for.
Compare the same months, before and after launch
If you cannot run a holdout, or you launched long ago and want to look back, compare cohorts. Take the first-time buyers you acquired in, say, February through April of the year before the program launched. Take the first-time buyers from the same three months of the year after. For each group, measure what share came back for a second order within 90 days and what they spent in their first year.
Use the same months on purpose. Seasonality will swamp a loyalty effect if you compare spring buyers to holiday buyers. And write down anything else that changed between the two periods: a price increase, a new best seller, a big shift in your ad mix. Any of those can move repeat behavior on its own. This test is weaker than a holdout, but it asks the right question, which the dashboard does not.
Check whether redeemers changed their pace
Pull the list of customers who have redeemed points. For each one, count their orders in the 12 months before they joined and the 12 months after. If a customer ordered four times a year before joining and four times a year after, the program changed nothing about their behavior. It just started charging you a discount on their fifth, ninth and thirteenth orders.
This matters because redemption rate is the metric loyalty apps push hardest, and it is easy to misread. One app vendor's rule of thumb is that 20 to 30 percent of issued points being redeemed is healthy and that a sustained rate above 40 percent, with profit not growing, signals a program giving away too much. Kyros finds nearly the opposite at scale: when more points get redeemed, customer value tends to rise, which suggests unredeemed points mostly belong to customers who drifted away. Both can be true, and the redeemer check is how you settle it for your store. High redemption from customers whose pace increased is a program working. High redemption from customers whose pace did not budge is a leak.
The App Fee Is the Cheapest Part of a Points Program
When owners price a loyalty program, they price the app. Smile's monthly plans are currently $15, $79 and $199, with the tiered VIP features on the $199 plan and larger plans above that. Other apps are in the same range. That number is real, but it is the smallest line in the budget.
The big cost is the points themselves. Most programs give back something in the range of 5 to 10 percent of order value, and the loyalty vendors themselves suggest starting there. Every point you issue is a promise to discount a future order, and it gets paid mostly by the customers who buy most often, who are the ones you were least at risk of losing.
Work it through with round numbers. Say your store does $600,000 a year at a 40 percent gross margin, and program members account for $162,000 of that. You give back 5 percent in points, and about 70 percent of those points eventually get redeemed. That is roughly $5,700 a year in discounts. Add a $79-a-month app and you are near $6,600.
At a 40 percent margin, you need about $16,500 in extra sales just to earn that $6,600 back. That is about 10 percent more than your members were already spending. It has to come from orders they would not have placed without the program. Orders they were going to place anyway do not count, because you are paying the discount on those too.
Two things fall out of that math. First, the lower your margin, the harder a cash-back program is to justify. A store at 30 percent margin needs members to spend nearly 12 percent more just to cover the points, before the app fee. Second, unredeemed points are not a profit center. It is tempting to see breakage, the points that expire unused, as money you kept. Mostly it is customers who stopped paying attention to you, which is the opposite of what you bought the program for. We made the same argument about unused balances in the piece on gift cards, and it holds here.
Reward the Order That Was Not Going to Happen
If a flat 5 percent back on everything mostly rebates your regulars, the fix is not to cancel the program on reflex. It is to aim the reward at the behavior you are actually missing. For most small stores, that is the second order. It is where the biggest drop happens, and it is where the money is, as we laid out in Your First Sale Barely Breaks Even.
Here is what that looks like in practice.
Make the first reward pay off on order two, with a deadline. Give a first-time buyer enough points on their first order to be worth something real on their second, and have those points expire on a timeline that matches how often people actually reorder your product. If your customers typically come back in 45 days, a points balance that is only valid for 60 days pulls the second order forward. A balance valid forever pulls nothing.
Time your expirations to your slow months. Every store has quiet stretches. Instead of letting points expire at random, set expirations so a reminder lands just before a slow period, when an extra reason to order is worth the most. Several loyalty apps support this, and it turns a cost you were paying anyway into demand exactly when you need it.
Put double points where people do not come back. Your order data will show which products get reordered and which get bought once. Bonus points on the categories with weak repeat rates, or on items you are carrying too much of, direct the incentive to where it changes a decision.
Pay for behavior, not just purchases. Points for a product review, for a photo, for a referral that becomes a first order. These are actions customers were not going to take unprompted, and each one is worth something to you beyond the order. That is a much better use of a point than a discount on a reorder that was already in the mail.
Stop paying the same rate on everything. If your top customers earn full points on every routine reorder, you are running a discount on your most dependable revenue. Cap it, lower the base rate and raise the targeted ones, or move those customers to perks instead of cash back. A standing percentage off on routine orders teaches the same lesson any habitual discount does, which we took apart in You Discount Because You Never Built an Offer.
Perks That Cost Less Than Points and Mean More
Not every reward has to be a discount, and the best ones usually are not. For the customers who already buy often, the useful question is not "what discount would they like" but "what can we give them that costs us little and is genuinely hard to get elsewhere."
Some options that cost a small store very little:
- Early access to new products or limited runs, a day or a week before everyone else.
- First call on restocks when a popular item sells out.
- Members-only products, a color, size or bundle that is not sold to the public.
- Easier returns or faster shipping for top-tier members, which costs you far less than a standing percentage off.
Tiers make these feel earned rather than handed out, and several apps build them in. Keep the tier names meaningful to your customers, and give people ways to climb besides spending, such as reviews and referrals, so the top tier does not read as a price list.
A paid membership is a bigger step. Wharton's Fader favors a paid layer on top of a free program, reserved for the small group who value it most. Best Buy's My Best Buy Total, at $199.99 a year, is his example. The logic is that a few customers are worth far more than the rest, so build something specifically for them instead of spreading one program thinly over everyone. It only works if you have enough committed repeat buyers to fill it. One vendor's rule of thumb is to wait until about 30 percent of your customers buy again. If you are weighing a paid program, much of what we wrote about subscriptions applies: it is a different business model, not a bigger discount.
One thing this section deliberately does not cover is simple recognition, the unexpected extra in a regular's order or the note that says you noticed this is their tenth order. That is often more powerful than any program and costs almost nothing. We covered it at length in Nobody Is Loyal to a Catalog, and it works with or without an app.
Nobody Uses a Loyalty Program They Have Never Heard Of
Before you judge whether a program changes behavior, check how many customers even know it exists. A restaurant marketing agency described asking the owner of a 50-plus-location fast-food franchise group what share of diners used the brand's loyalty app. When he checked, it was under 3 percent. A program cannot change the behavior of people who never joined it.
Online stores have the same gap, just less visibly. Pull your enrollment rate: of the customers who ordered in the last 90 days, what share are members? If it is in the single digits, the program is a quiet perk for a few regulars, and the question of whether it works is premature.
The fixes are unglamorous and effective:
- Tell people on the order confirmation page, where they have just bought and are paying attention. Show the points they just earned, not a generic "join our rewards" banner.
- Put the program in the first post-purchase email, with the balance and what it is worth on the next order. Your post-purchase email flows are the natural home for this.
- Send a monthly points statement, a short email showing each member's balance, what it is worth in dollars, and when it expires. It reminds people that the program exists, and it creates a reason to come back.
- Put a small card in the box with the balance written on it. Physical mail gets read.
Fix enrollment first, then measure. Otherwise you are grading a program most of your customers have never seen.
When to Switch Your Loyalty Program Off
Give a program six to twelve months, and give it a fair shot by aiming it at the second order and fixing enrollment first. Then look at the tests above. If held-out customers come back just as often as members, if first-time buyers after launch return at the same rate as before, and if your redeemers are ordering on exactly their old schedule, the program is not working. Switch it off.
That is a harder call than it sounds, because a program feels like an asset and ending one feels like taking something away. It helps to remember what you are actually ending: a standing discount on orders you were going to get anyway.
Shut it down cleanly:
- Stop new points first, and announce it plainly.
- Give members a clear window to use what they have, 60 to 90 days is common, and remind them twice before it closes. Check what your program's own terms promised about balances and follow them.
- Put the money somewhere that changes behavior: a better post-purchase sequence, a real second-order offer with a deadline, or recognition for your best customers. Your customers will not miss the points as much as you fear, because most of them were never counting.
And if the tests say it is working, you now have proof instead of a dashboard, which puts you in a very different position the next time the app raises its price or pitches you on the next tier.
This is the kind of work we do every day for online stores: finding out which of your programs and promotions actually change what customers do, and moving the money to the ones that do. If you would rather run the tests yourself, start with the redeemer check this week. It takes an hour with your order export, and it will tell you more than a year of dashboard screenshots.
