From Dashboards to Decisions: The Weekly Question Framework for eCommerce Teams

Someone in r/shopify posted this in June: “Am I the only one who opens Shopify Analytics every Monday and has no idea what to do with it?” The thread got 140 replies. Almost none of them were disagreements.

Dashboard fatigue is not a tool problem. Every major analytics platform (Shopify, GA4, Amplitude, even the custom BI build your previous hire left behind) produces dashboards. The dashboards have data. The data is often accurate. And you still walk away from Monday morning without a clear action.

The reason is structural. A dashboard answers the question “what happened?” It does not answer “what should I do about it?” That second question requires judgment, context, and a deliberate decision-making habit that most stores never build. They get better dashboards instead.

This post replaces the dashboard habit with something more useful: three specific questions, asked in order, every week. Each question has a natural home in Stormly’s product-level analytics. Each takes less than ten minutes to answer. The output is not a report. It is a single action item for the week.

Why the reporting habit produces paralysis

When a store operator opens an analytics dashboard, they are typically looking at six to twelve metrics at once: revenue, sessions, conversion rate, AOV, bounce rate, new vs. returning, traffic by source, top products by revenue. These numbers coexist without hierarchy. None of them tells you what to fix first.

“Are we problem solvers or just reporters?” is how one r/analytics thread framed it. The distinction is exact. Reporting is the act of observing what happened. Problem-solving is the act of deciding what to change.

The gap between them is not intelligence or effort. It is a question. Without a specific question going into the Monday session, you observe. With the right question, you decide.

The three questions below were chosen because they cover the three places where eCommerce stores lose the most revenue week over week: product performance, customer retention, and checkout conversion. Each maps to data a product-level analytics tool can surface in a single view. The broader architecture of how to make your dashboards more actionable covers the structural redesign behind this approach in more detail.

The three questions

Question 1: Which product needs attention right now?

Not “which product made the most revenue last week.” Revenue is a lagging indicator: a product that sold well last week often peaked two weeks ago. The leading question is: which product is showing a meaningful shift in velocity relative to its own recent baseline?

In a Stormly product performance view, this is a weekly velocity ranking by product. For each SKU, it shows the current week’s units compared to the trailing four-week average, flagged with a directional indicator.

Here is what this looks like in practice. A candle store with 87 active SKUs. In the Monday view, most products are within 15% of their four-week average, which is normal weekly variance. One SKU stands out: a beeswax taper candle showing a 38% velocity drop over the past two weeks, despite being the store’s second-highest margin product and having no change in price or stock level.

A 38% drop on a high-margin item is not noise. The possible causes are narrow: a product page problem, a traffic source shift, a review issue, or the seasonal demand curve simply moving. Any of them are findable in twenty minutes. Without the velocity view, this product would have looked fine in the revenue ranking for another two or three weeks before the total became large enough to notice.

The action from Question 1 this week: investigate the beeswax taper page. Check the traffic source breakdown for that SKU. Look at the conversion rate week-over-week. One of those three checks will identify the problem.

That is a decision. The dashboard told you which product. You found the cause. You fix it.

Question 2: Which customers are most at risk of not returning?

Most stores think about retention as a trailing metric: what was our 90-day retention rate last quarter? That framing is too slow for a weekly decision cycle. The forward-looking version is: which cohort of customers, acquired recently, is showing a weaker retention trajectory than the store average?

In Stormly, the retention curve by first-purchase cohort shows repeat-purchase probability over time, broken down by which product the customer bought first. The weekly question is not about the aggregate curve. It is about whether any specific cohort is tracking below the historical pattern at the same point in their lifecycle.

A skincare store with 12,000 active customers found a consistent pattern in this view. Customers who first purchased a daily cleanser had a 52% 60-day repurchase rate. Customers who first purchased a one-time gift set had a 14% rate. That gap was known and expected. What was not expected: in the most recent two-week cohort, even the cleanser-first customers were tracking 9 percentage points below their historical baseline at the 30-day mark.

A 9-point drop at day 30 in a cohort that normally recovers to 52% by day 60 is a signal worth acting on before the 60-day window closes. The store ran a product-specific re-engagement email to that cohort, with a replenishment prompt for the cleanser SKU, within the same week. The 60-day retention for that cohort ended up at 48%, below the 52% historical average but well above the trajectory the early data suggested.

You cannot make that call on a revenue dashboard. You need the cohort curve by first-purchase product. How eCommerce customer retention analytics works and which product-category signals actually predict churn covers the measurement methodology in detail.

Run your first weekly retention check in Stormly. Start a free trial.

Question 3: Where is my checkout leaking this week?

Your overall checkout conversion rate is an average. The average hides which specific products are dragging it down. A store with a 61% overall checkout completion rate may have three products completing at 28% while the rest of the catalog sits above 70%. The aggregate looks acceptable. The product-level view shows three specific problems.

This question should be answered at the product level, not the site level.

In Stormly, the checkout funnel breakdown shows completion rates at each step (add to cart, begin checkout, payment, purchase) disaggregated by product. The weekly version asks: which products moved out of normal range this week, in either direction?

A fitness apparel store ran this check and found a resistance band set with a 71% cart-to-checkout drop, against a 22% category average. The product had not changed. The price had not changed. The traffic source mix had not changed. What had changed: a link from a popular YouTube review was sending traffic directly to an “add from bundle page” URL, which loaded without the store’s standard checkout trust signals. The resistance band was the only SKU in the bundle link. Once the store added a quantity selector and free-shipping threshold note to that page, checkout completion for that SKU returned to 67% within a week.

That finding took eleven minutes to surface in the product checkout view. It would have been invisible in a site-level conversion rate report for months.

What makes this work: the question comes first

The framework is three questions, in order, before you open a single dashboard. The sequence matters.

• Product velocity first: catches the fastest-moving problems before they compound
• Retention cohort second: identifies customers you can still recover this week, not last quarter
• Checkout funnel third: finds conversion losses that are usually fixable without any marketing spend

Each question points you at one specific view. You spend your time diagnosing and deciding, not scrolling through reports looking for something that seems important.

The habit is closer to a weekly health check than a reporting exercise. A physician does not open a chart and read every number. They check the three indicators that have changed since the last visit and investigate the one that is out of range.

The benefits of dashboards for data analysts are real, but those benefits materialize only when the dashboard is connected to a specific decision, not when it serves as a general-purpose reading room.

Where agentic analytics fits

The three-question framework assumes you know which question to ask and where to look for the answer. An agentic analytics layer changes the starting point: instead of you opening a view and scanning for anomalies, the tool surfaces the anomaly and tells you which question is already answered.

Stormly’s insight feed does this for the product-level views above. It flags when a product’s velocity has shifted outside its historical variance, when a retention cohort is tracking below baseline, and when a checkout funnel step has moved more than a threshold in either direction. You open the Monday session and the three questions are partially pre-answered: “This week, the beeswax taper velocity dropped 38%. The September cleanser cohort is 9 points below baseline at day 30. The resistance band checkout is at 71% drop, up from 29% last week.”

That is not a dashboard. That is a brief. The remainder of your Monday session is diagnosis and action, not search.

Self-serve analytics for eCommerce teams covers the broader transition from waiting on a BI team to getting answers directly in the tool. The agentic insight layer is what makes the three-question framework sustainable at scale: you do not have to build the habit of checking every view every week. The tool tells you which view matters this week.

Setting up the Monday routine

The framework takes about 25 to 35 minutes once the views are configured. Here is how to structure it.

Before 9 AM Monday:

1. Open the product velocity view. Identify any product with a week-over-week movement greater than 20% in either direction. For any flagged product, note whether the change is in traffic (sessions to the product page), conversion (add-to-cart or purchase rate), or both. Each of those points to a different fix.

2. Open the retention cohort by first-purchase product. Check the most recent two-week acquisition cohort against the historical baseline for the same point in the retention curve. If a cohort is tracking more than 8 percentage points below the historical average at day 21, add that cohort to a re-engagement sequence before the week is out.

3. Open the checkout funnel by product. Look at the ten products with the highest abandonment rate at the payment step this week. Compare each to their trailing four-week average. Any product that moved more than 15 percentage points warrants a five-minute audit: check the product page, the checkout URL, and whether a traffic source changed.

The output of these three checks is a list of no more than three specific action items for the week. Not a report. Not a reading list. Three actions.

That is the shift from reporter to problem solver.

One number that changes how you run the week

In a typical Shopify store with more than 50 SKUs, the spread between the highest and lowest performing products by checkout completion rate is usually 30 to 50 percentage points. The overall checkout rate tells you nothing about this spread. The product-level view makes it immediate.

Once you see your catalog through a product-level lens for the first time, the aggregate metrics start to feel like averages that are technically accurate and practically useless. Revenue was $48,000 last week. One product drove $11,000 of that and converted at 2.1%. Another drove $3,400 and converted at 14.7%. Those two numbers tell completely different stories about where to invest next week.

What eCommerce analytics actually covers and where the most useful data lives goes into this split in more detail: marketing analytics (sessions, attribution, ROAS) tells you the acquisition picture; product analytics tells you the catalog and merchandising picture. Most stores have the first and are missing the second.

The weekly question framework is the practical expression of that second layer. Three questions. Three views. One decision per question. Thirty minutes, every Monday.

Start your first weekly decision session in Stormly. Free trial here.