Supplier scorecard: the six metrics a small store can actually compute
A small-store supplier scorecard you can build from PO records: 6 metrics, formulas, a weighted template, and a worked quarterly review of 3 suppliers.
InventoryX is in development and not available today. ClearStock is the live Shopify app.
A supplier scorecard is a one-page tally of five or six performance metrics per supplier, computed from your own purchase order history, scored once a quarter. You do not need procurement software to build one — your PO records and a spreadsheet do the whole job. Most scorecard content online is written for companies with procurement departments and ERP systems; this is the version a Shopify store buying from three to ten suppliers can actually run, in about an hour per quarter.
Why bother? Because right now your supplier decisions are made on vibes: "they've been late a couple of times," "quality seems fine." Vibes fail exactly when you need them — when a supplier asks for a price increase, when you're deciding which of two suppliers gets the bigger order, or when you're staring at a late delivery wondering if this is a pattern or bad luck. A scorecard turns those arguments into numbers. It also feeds the rest of your inventory math: lead time and reorder decisions get more accurate when you know which suppliers actually deliver, and it slots straight into your wider inventory KPIs.
The six metrics worth scoring
Ignore enterprise supplier KPI libraries with thirty metrics. These six cover what actually hurts a small store, and every one is computable from PO records:
| # | Metric | Formula | What it tells you |
|---|---|---|---|
| 1 | On-time delivery % | POs received by promised date ÷ total POs | The headline reliability number |
| 2 | Fill rate | Units received ÷ units ordered | Whether you get the full order |
| 3 | Lead time: average vs. p80 | Average and 80th percentile of actual lead times | How variable the supplier really is |
| 4 | Defect / rejection rate | Rejected units ÷ units received | Quality, in the only currency that matters |
| 5 | Price stability | Unit price change over the quarter/year | Whether costs drift without warning |
| 6 | Responsiveness | Business days to acknowledge a PO | How the supplier behaves when something goes wrong |
Each metric maps to a specific field in your PO history:
- On-time delivery. You need two dates per PO: the promised delivery date (from the supplier's confirmation — more on why that confirmation matters here) and the date the units were actually available to sell. Score it as a simple count: received on or before promised ÷ total POs, over the quarter. Score partial credit for a PO that lands within 3 days of the promise if you want a kinder number — but pick the rule once and apply it to every supplier.
- Fill rate. Sum units ordered vs. units actually received per PO, per quarter. A supplier who ships 480 of 600 units on time didn't deliver on time — they delivered 80% of your order and left you short.
- Lead time average vs. p80. Lead time = received date − order date, in calendar days, per PO. The average is your planning number; the p80 (the value below which 80% of your POs fall — in a spreadsheet,
=PERCENTILE.EXC(range, 0.8)) is your variability number. Two suppliers can both average 30 days while one has a p80 of 33 and the other 47. The gap between average and p80 is the single most under-tracked number in small-store procurement. - Defect / rejection rate. Units rejected at receiving (or refunded/credited by the supplier afterward) ÷ units received. Your inventory counts record the write-offs; keep the reason with the count.
- Price stability. For each SKU the supplier sells you, compare this quarter's unit price to the same quarter last year, or to your rolling 12-month average.
=MAX(prices)/MIN(prices) − 1over the period gives you the drift range. A supplier who raises prices twice a year unannounced scores worse than one who raises once, with notice — even at the same average price. - Responsiveness. Days from when you sent the PO (or a question) to their first substantive reply. This is the easiest one to log: timestamp on your sent email, timestamp on theirs. It matters most when things go wrong, which is exactly when you can't afford slow replies.
A weighted template
Not every failure hurts equally. On-time delivery and lead-time variability hit your ability to keep bestsellers in stock; price stability hits margin; responsiveness mostly predicts how painful a dispute will be. A workable weighting for a typical small store:
| Metric | Weight | Rationale |
|---|---|---|
| On-time delivery % | 25% | Late goods cascade into every downstream decision |
| Fill rate | 15% | Short orders are stealth delays |
| Lead time p80 (variability) | 20% | Variability, not the average, forces buffer stock |
| Defect rate | 15% | Refunds and returns cost twice: cash and trust |
| Price stability | 15% | Unannounced increases poison the relationship |
| Responsiveness | 10% | Predicts dispute handling |
Score each metric 1–5 (5 = excellent) using simple bands, multiply by the weight, sum to 100. Example bands for on-time delivery: 95–100% → 5; 90–94% → 4; 80–89% → 3; 65–79% → 2; below 65% → 1. Set your own bands once, then never move them mid-quarter — a scorecard whose bands shift is just vibes with a spreadsheet attached.
Worked quarterly review: three suppliers
A store buying ceramics, textiles, and packaging runs this for Q3:
Nova Ceramics (overseas, 40-day quoted lead time). 8 POs in the quarter. 7 of 8 landed on or before the promised date → on-time 87.5% → band score 3. Fill rate: 2,410 of 2,500 units = 96.4% → 4. Lead times: 38, 41, 40, 44, 39, 37, 40, 42 days → average 40.1, p80 ≈ 42.4. Gap is only ~2 days → 5. Defects: 61 of 2,410 received units rejected = 2.5% → 3. Prices flat all quarter → 5. Responsiveness: 2 business days average → 4.
Weighted: 3×.25 + 4×.15 + 5×.20 + 3×.15 + 5×.15 + 4×.10 = 0.75 + 0.60 + 1.00 + 0.45 + 0.75 + 0.40 = 3.95 / 5 (79/100). A strong score despite the "slow" reputation — Nova is slow but ruthlessly consistent, which is the trait that matters.
Bluefin Textiles (domestic, 12-day quoted). 9 POs. 5 on time → 55.6% → band 1. Fill rate: 1,120 of 1,300 = 86.2% → 2. Lead times: 11, 14, 12, 19, 15, 12, 21, 13, 16 → average 14.8, p80 = 19. Quoted 12, planning on 15 would still miss 1 in 5 cycles. Variability gap of 7 days → 2. Defects: 22 of 1,098 = 2.0% → 3. Prices: one unannounced 6% mid-quarter increase on two SKUs → 2. Responsiveness: 4 days, sometimes a week → 2.
Weighted: 1×.25 + 2×.15 + 2×.20 + 3×.15 + 2×.15 + 2×.10 = 0.25 + 0.30 + 0.40 + 0.45 + 0.30 + 0.20 = 1.90 / 5 (38/100). "Domestic and fast" was the story; the scorecard says "domestic and unreliable." This is exactly the case the scorecard exists to catch — proximity feels like safety while the fill rate quietly bleeds.
Northpack Packaging (local distributor). 6 POs, 6 on time → 100% → 5. Fill rate 99.7% → 5. Lead times: 4–6 days, p80 6 → 5. Defects 0.3% → 5. Prices +3.5% across the board in Q3, announced in advance → 3. Responsiveness: same-day → 5.
Weighted: 1.25 + 0.75 + 1.00 + 0.75 + 0.45 + 0.50 = 4.70 / 5 (94/100).
The quarterly output is one line per supplier plus a sentence on the driver: Nova = excellent, keep consolidating volume. Bluefin = failing, see below. Northpack = excellent, the announced price increase is fair and worth tolerating.
What to do with a failing score
Escalate in order, and don't skip steps:
- Conversation. Send the supplier their own numbers — "5 of your last 9 orders arrived late, average 15 days against a 12-day quote, and two shipped short." Factual, unemotional, impossible to argue with. Ask what happened and what changes. Many suppliers are genuinely surprised; small buyers rarely tell them why they're losing volume.
- Adjust your own math while behavior changes. Don't wait for them to fix it — re-plan against their measured p80, not their quote, and size buffers accordingly. You can run lean against Nova's 2-day gap; you need real safety stock against Bluefin's 7-day one.
- Dual-source. Split the category: keep the failing supplier for non-critical items or as overflow, move your bestsellers to the higher scorer. This also fixes the concentration risk of one supplier per category.
- Exit. Two consecutive failing quarters after a direct conversation, with the numbers shared, is a decision, not a feeling. End cleanly, pay what's owed, and don't rescind on a whim — the credibility of your scorecard depends on it meaning something.
Why nothing automates this (yet)
Honest note: no mainstream tool for small stores — including us — automates supplier scorecards. Shopify's native purchase orders record order, expected, and received dates per the Shopify Help Center (accessed October 2026), and the Purchase Order Read API shipped September 17, 2026 exposes POs, line items, and supplier names to apps (shopify.dev changelog) — so the raw data is readable. But promised-date tracking, fill-rate logging at receiving, and rejection reasons are still mostly manual records, which is why this article is a spreadsheet method and not an app review. (For where native records end, see what Shopify purchase orders can and can't do.) InventoryX builds on the same PO-derived numbers for lead times and buying lists; the scorecard layer you'll still do by hand. An hour a quarter is a fair price for knowing which of your suppliers is quietly costing you weeks.
Common questions
What is a supplier scorecard? A one-page, per-supplier tally of performance metrics — typically on-time delivery, fill rate, lead-time variability, defect rate, price stability, and responsiveness — computed from your own PO records and scored on a regular schedule. It exists to turn "they've been unreliable lately" into a number you can act on, defend, and track across quarters.
What metrics should a small business track for suppliers? The six that matter most: on-time delivery %, fill rate, lead-time average vs. p80, defect/rejection rate, price stability, and responsiveness (days to acknowledge a PO). Every one is computable from PO history — order date, promised date, received date, units ordered vs. received, and rejected units. Anything more exotic than that is procurement-department territory.
How often should I review supplier performance? Quarterly for the full scorecard, with a quick monthly glance at on-time delivery for your critical suppliers. Quarterly is long enough to have statistically meaningful PO counts (aim for 6+ per supplier) and short enough to catch a deteriorating supplier before two bad quarters become a lost season.
How do I calculate on-time delivery rate for a supplier? Count the POs in the period where the units were available to sell on or before the promised delivery date, divide by total POs, multiply by 100. In the worked example above, 7 of 8 Nova POs arrived on time = 87.5%. Score partial credit within 3 days if you prefer, but apply the same rule to every supplier.
What do I do if a supplier keeps failing? Escalate in order: share their numbers in a factual conversation, re-plan against their measured lead-time variability in the meantime, dual-source your bestsellers, then exit after two consecutive failing quarters post-conversation. Skipping the conversation costs you leverage; skipping the exit after it costs you the whole next season.
Next step
Pick your highest-volume supplier and pull their last six POs — order date, promised date, received date, units ordered vs. received. Score the first three metrics tonight; the whole template above fits in one sheet. When you're ready to act on the numbers, this is also the moment to renegotiate terms or, for the suppliers that score well, consolidate more volume with them.