Shopify Rollouts vs. Intelligems

Yesterday Shopify shipped its Spring '26 Edition, with more than 150 updates spanning agentic commerce, checkout, retail, and the developer platform. Buried in the release was the update experimentation-minded merchants had been waiting on: Rollouts now supports scheduled publishing, gradual launches, and A/B testing for themes, checkout, and customer accounts pages.

If Shopify is letting me phase in theme and checkout changes natively, and even run an A/B test on them, why would I still pay for Intelligems?

It's a fair question. And it deserves a real answer.

The short version: Rollouts is a deployment-and-experimentation tool built around themes, checkout, and accounts. Intelligems is a profit optimization platform built around the question of whether any change you ship makes you more money, across the whole store. Both can live on the same site. But if your goal is a robust profit optimization strategy, Rollouts on its own won't get you there.

What Shopify Rollouts Actually Does

Rollouts is one product inside the Shopify admin (under Markets > Rollouts) with three modes you can choose for any given change.

Mode 1: Scheduled publish. Push a new theme, checkout configuration, or customer accounts setup live at a specific date and time. Useful if you want a redesign to go live the moment a campaign starts, without anyone clicking publish at midnight.

Mode 2: Gradual launch. Roll the change out to a configurable percentage of eligible visitors in selected markets. You can dial reach up over time, and you can set an end date that auto-reverts the change if something looks wrong.

Mode 3: Experiment. Run an A/B test between a control (your current setup) and a treatment (the change). Shopify allocates traffic between the two and reports results inside the Rollouts analytics view. This is the mode most people are picturing when they ask whether Rollouts replaces a testing platform.

A few constraints to know up front:

• Experiments require Grow plan or higher. Scheduled publishing and gradual launches appear available on lower tiers.

• The full scope is themes, checkout, and customer accounts pages. Pricing, shipping, subscriptions, offers, and any non-theme element are outside the Rollouts surface.

• One change per resource type per rollout.

• Editing the control or treatment copy while an experiment is live can affect the results, so the variants need to be locked in before launch.

• The analytics layer is Shopify's standard reporting. Profit per visitor isn't part of the readout.

• Only specific roles (store owners, org owners, admins, or users with explicit Markets > Rollouts permission) can manage rollouts.

That last constraint is a workflow detail. The first five shape what Rollouts can and can't do.

Where Rollouts Hits Friction

Here's what tends to come up when merchants try to run their full testing program on Rollouts alone.

The scope is narrow. Theme, checkout, and accounts pages are the surface. You can't natively test prices, shipping rates, free shipping thresholds, subscription configurations, multi-currency strategies, or any offer that isn't a theme edit. For anything outside the theme and checkout layer, Rollouts isn't a fit.

The granularity is coarse. Rollouts works at the theme or template level. If you want to test a single PDP element, a hero section, a quantity selector, or a single piece of copy in your cart drawer, you're either rebuilding a whole template to test one thing or you can't run the test at all.

Theme edits don't carry content with them. The Shopify documentation notes that edit-only theme changes don't support content modifications. So if your test involves swapping copy, images, or in-section content, you have to figure out a different workflow.

Concurrency is restricted. You can only run one change per resource type per rollout. If you want to test three things on your theme at once, you can't.

There's no profit lens. Shopify reports what its analytics layer reports, conversions, sessions, AOV. Profit per visitor, the metric that actually tells you whether the change made you more money after costs, isn't in the picture.

Audience targeting is limited. Rollouts splits by markets and a percentage of visitors. It doesn't natively target by traffic source, device, returning vs new customer, or by a Klaviyo segment. You can ship a global change to a percentage, but you can't run an experiment that says "this version only for paid social visitors who haven't bought before."

None of this is a knock on Rollouts. It's a deployment tool that recently grew an A/B mode. It's doing what it was built for.

The friction shows up when merchants try to use it as the foundation of a profit optimization program.

What Intelligems Does Instead

Intelligems is built around one question. For every change you ship, did profit per visitor go up?

That changes how the testing system has to work.

You can test the things that actually move profit. Prices, discounts, shipping rates, free shipping thresholds, subscription pricing, multi-currency strategies, checkout flows, theme variations, individual page sections, offers, gift-with-purchase mechanics, landing pages. Each one has a dedicated test type with the integrations and guardrails the test needs.

You can test at any granularity. A single PDP. A whole catalog. One section of one page. An entire theme. A single price point. Every product. Whichever level fits the question you're trying to answer.

Every test reports profit per visitor. Along with conversion rate, AOV, revenue, return rate, and any custom metric you wire in. So when a test wins, you know whether the win is real after costs, not just whether conversions went up.

**Audience targeting is built in.** Source, device, geography, returning vs new, Klaviyo segments, custom URL parameters, and customer attributes. You can split traffic the way the experiment actually needs to be split, not just by market or a flat percentage.

Personalization sits next to testing. Once a winning variant is found for a specific audience, you can keep that audience on the winning experience while testing something else against the rest of your traffic. The same platform handles both.

Multiple tests can run at once. Without stepping on each other and without needing to ration tests across resource types.

Winners apply with one click. When a test wins, you push it live to all traffic from inside Intelligems. There's no CSV export, no theme rebuild, no manual reconfiguration.

Eligibility isn't gated. Intelligems runs on any Shopify plan, with any product count, at any sales volume. The decision about whether to test isn't made for you.

What About Smart Pricing?

Worth a brief detour. Rollouts is Shopify's tool for theme and checkout experimentation. Shopify's other native testing product, Smart Pricing, is a separate app focused on price recommendations and price A/B testing for select US merchants.

They're sibling products covering different slices of the surface area. If you're specifically trying to understand how Smart Pricing compares to Intelligems on the pricing side, we wrote that comparison up separately.

The point worth making here: even running both Rollouts and Smart Pricing, you're using two limited tools to cover two narrow slices, and you're still missing shipping tests, subscription tests, multi-currency tests, offer tests, audience targeting, personalization, and a profit-per-visitor readout that ties it all together.

Side by Side

Why the Metric Difference Matters

Two scenarios show up over and over in real merchant data.

A theme test goes live. The new template has a bigger hero, fewer product cards above the fold, and a more aggressive CTA. Conversion rate goes up 6%. AOV goes down 4% because customers are buying the hero product more often instead of bundling. Net profit per visitor is roughly flat, maybe slightly negative once you factor in the higher promo mix on the hero SKU.

If you're watching Shopify's standard analytics, you ship the change. Conversions are up. Looks like a win.

If you're watching profit per visitor, you might not ship it. Or you ship it only for the audience where the math actually works.

The opposite scenario is just as common. A simplified checkout test drops conversion by 2%. But the customers who do convert have higher AOV, higher attach rate on subscriptions, and a meaningfully lower return rate. Profit per visitor goes up.

A pure conversion-rate readout sends you back to the old checkout. A profit-per-visitor readout sends you forward.

Neither of these reads correctly without the profit lens. Rollouts doesn't have one.

When Shopify Rollouts Might Be Enough

Rollouts on its own can be the right call if all of the following are true:

• You're only changing the theme, checkout, or accounts pages

• You don't need to test pricing, shipping, offers, subscriptions, or any non-theme element

• Conversion rate and standard Shopify analytics are enough to make the call

• You're fine phasing a change in or running a single experiment, not many in parallel

• You don't need audience-specific personalization

For brands shipping a single theme refresh and wanting to do it carefully, Rollouts is a useful tool. It's safer than flipping a switch.

What it doesn't give you is a way to know whether the new theme made you more money.

When You Need a Real Profit Test

Reach for Intelligems when any of these apply:

• The change involves pricing, discounts, shipping, free shipping thresholds, or any offer mechanic

• You want to measure the impact in profit per visitor, not just conversions

• You want to test subscription pricing or multi-currency

• You want to target the experiment to a specific audience

• You want to test multiple things in parallel

• You want personalization to live alongside testing on the same platform

• You're on Basic or Shopify plan and don't want Grow-tier eligibility deciding what you can test

These aren't edge cases. They're the bread and butter of a profit optimization program.

How long the test runs, how to set the audience, how to interpret the results, and what to do once a winner emerges all need a system that's actually built for testing. Not a deployment tool with an A/B mode.

@https://www.youtube.com/watch?v=SSUa-09-nEk

From IntelliJAMS EP 040, on the experimentation mindset for 2026 and why best practices alone don't get you to a profit answer.

Using Both, If That's the Right Call

Rollouts and Intelligems don't have to be either/or. A reasonable setup looks like this:

• Use Rollouts when you're shipping a confirmed change and want to schedule it or phase it in to manage risk

• Use Intelligems when you're trying to decide whether a change is worth shipping at all, and when the change is anywhere outside theme and checkout

The deployment job and the experimentation job are different jobs. Rollouts now does both, but only for theme, checkout, and accounts. Intelligems handles experimentation across the whole store and turns winners into personalized experiences for the audiences they apply to.

The trap is using Rollouts as the experimentation layer for your whole profit optimization strategy. That's where merchants find out months later that their "winning" theme test was a wash on profit, or that the checkout change they shipped to 50% of traffic never actually beat the old one.

Your Data Points the Way

Four questions to ask yourself:

• Do I want to ship changes carefully, or do I want to prove they work?

• Are conversions and AOV enough, or do I need to know what happened to profit?

• Is theme and checkout the full surface area I want to optimize, or do I also care about price, shipping, offers, and subscriptions?

• Do I need to personalize the winning experience by audience, or is one global rollout enough?

Rollouts is the right answer to some of these. Intelligems is the right answer to others. Knowing which question you're trying to answer is the part that changes everything.

FAQs

I'm already an Intelligems customer. Should I migrate my tests to Rollouts?

For tests that involve price, shipping, offers, subscriptions, multi-currency, or anything outside the theme and checkout layer, Rollouts isn't a fit. For straightforward theme phase-ins where you don't need a profit readout or audience targeting, Rollouts can sit alongside Intelligems and handle the deployment job. The two work fine on the same store.

Does Shopify Rollouts replace Intelligems Content Testing?

It overlaps at the surface but not in depth. Rollouts can A/B test a theme or a checkout. Intelligems Content Testing can test full themes, individual templates, single sections, single elements, copy variants, image swaps, and landing page variants, with profit per visitor as the readout and audience targeting built in. If your testing program is one theme refresh a year, Rollouts may be enough. If it's an ongoing program, the surface area and the measurement gap show up fast.

I'm on Shopify Basic. Can I run experiments?

Rollouts experiments require Grow plan or higher. Scheduled publishing and gradual launches inside Rollouts appear available on Basic. Intelligems runs on any plan, so if you're on Basic and want to A/B test, you don't need to upgrade your Shopify plan to do it.

What does Shopify charge for Rollouts?

Rollouts is included with Shopify plans. Experiment mode requires Grow plan or higher, which has its own monthly cost. Intelligems pricing isn't tied to your Shopify plan tier.

Can I test pricing with Shopify Rollouts?

Not natively. Rollouts targets theme, checkout, and accounts pages. Price changes are handled separately. Shopify's Smart Pricing app covers price recommendations and price A/B testing for select US merchants, and we covered how that compares to Intelligems in its own post. Anywhere outside that US-eligible cohort, or for any pricing test that needs profit-per-visitor measurement or audience targeting, price testing in Intelligems is the path.

Can Rollouts measure profit per visitor?

No. Rollouts reports through Shopify's standard analytics layer. Profit per visitor requires cost data per SKU, attribution back to the variant a visitor saw, and a measurement framework that ties the two together. That's the core of how Intelligems analytics work.

What happens to my running Intelligems tests if I set up a Shopify Rollout on the same theme?

The two systems aren't aware of each other, so if you're running an Intelligems theme test and you launch a Shopify Rollout that swaps the underlying theme, you'll create a confound. Pick one system per theme test at a time. If you want to run both deployment phasing and experimentation, run the experiment first to get the answer, then use Rollouts to schedule or phase the winner in.

For partner agencies, how should I explain the difference to clients?

Frame it as deployment vs. experimentation, with overlap on theme and checkout. Rollouts is how Shopify wants you to ship a confirmed change carefully, and now also a way to test one inside that narrow scope. Intelligems is how you decide what to ship in the first place across the whole store. Clients running a real profit optimization program need both jobs covered, and Rollouts only covers a narrow slice of one of them.

My client is on Grow plan and wants to run an A/B test on their checkout. Are Rollouts experiments enough?

They can be, for that single test, if the only metric they care about is what Shopify's standard analytics report. The friction shows up when they want to run the next test, target an audience, measure profit, or test anything outside the checkout. At that point the platform decision starts shaping the program.

Will Shopify expand Rollouts beyond themes and checkout?

Spring '26 brought scheduled publishing, gradual launches, and A/B testing to themes, checkout, and customer accounts. Whether the scope expands later to pricing, shipping, offers, and audience-level personalization isn't something Shopify has signaled publicly. For now, that surface area sits outside Rollouts and outside Smart Pricing both.

If I'm just getting started with testing, should I use Rollouts or Intelligems first?

If your first test is on price, shipping, or an offer, Intelligems is the only one of the two that supports it. If your first test is a theme change and you don't care about profit-per-visitor measurement yet, Rollouts is a low-effort way to phase it in. For a real testing program, the measurement gap and the surface-area gap catch up with you quickly. Starting with offer testing on a profit-aware platform is usually a better foundation.

Want to build a profit optimization program that holds up past your first theme test? Join GEM Academy for free courses and a community of brands sharing what's working.