How to Test an Amazon Agency on One ASIN Before You Sign
A pilot is a small, fixed-price job on one product that shows how an agency works before you hand over the account. It only proves something if you write down the starting numbers, the targets and the dates first. This is the buyer's method, step by step.
Short answer
To test an Amazon agency before a retainer, hire it for one fixed-price project on one ASIN for 60 days. Before day 1, record 30 days of sessions, conversion, ACoS, TACoS, total sales and organic rank. Agree two or three targets in writing, a dated change plan, and reports at day 30 and day 60. Evolve Media Agency's pilot is any single project from its price list: listing copy $495, five images $295, a PPC audit $995, or single-ASIN PPC management at $745 a month.
Why One ASIN Tells You More Than Three Sales Calls
Every agency sounds good on a call. A pilot shows whether it hits dates, explains its changes, reports the right numbers and tells you when something is not working. Those habits carry over to the full account.
One ASIN (Amazon's product ID, one listing) keeps the test clean. With five products, a win on one hides a loss on another.
A pilot is not a free trial. Free work is usually a sales asset: a quick audit built to find problems, not to fix them. A paid, fixed-price project with a written scope makes the agency do real work on your real listing, on a deadline.
Pick a fixed-price project, not a discounted retainer
A retainer at a pilot price trains you to judge the wrong thing. Buy a real project at its real price, so the cost of the next step is already known.
Pick the job you would hire them for
If you want PPC help, pilot PPC. If your listing is the problem, pilot listing copy. A pilot on the wrong job proves nothing about the right one.
Keep it to 60 days
Long enough for the data to settle, short enough that a bad fit costs two months, not a year.
Choose the ASIN and the Job
The pilot ASIN matters as much as the agency. Choose one that can show a change. A good pilot product sells every day, has a steady price, and has at least 60 days of stock. It also has an obvious gap: a weak main image, thin bullets, or an ad account that has not been cleaned in months.
Avoid a product that is brand new (no baseline), seasonal, about to run out, or about to change price.
Then match the project to the problem. Here is what a pilot looks like on our price list; ask other agencies for the same shape.
Listing copy: $495
Keyword research plus a new title, bullets and backend search terms for one ASIN. Best when traffic is fine but the page does not convert or does not rank. Start from our free listing audit to see if copy is the gap.
Listing images: $295
Our design tier of five main images. Best when the main image or image stack is the weak point.
PPC audit and restructure: $995 one time
10 hours of work on the ad account, aimed at the pilot ASIN's campaigns. Best when ad spend is high and results are flat.
Single-ASIN PPC management: $745 a month
Month-to-month ad management for one product. Our $1,000 a month minimum ad spend applies, and Amazon bills you for ad spend directly. Two months is a full 60-day pilot.
All Evolve projects
Paid 100% upfront, no long-term contract. See every option on our pricing page and the full listing optimization service.
Record These Numbers Before Day 1
The baseline is the before picture. Without it, the agency's day 60 report is a story, not a measurement. Pull it yourself from Seller Central, for the 30 days before the pilot, and send the agency a copy. Pull the same numbers for a comparison ASIN that the agency will not touch.
Also pull the same weeks from last year if you can. If they rose 15% with no agency, a 15% rise now is the season.
Write the Targets Before the Work Starts
Pick two or three numbers and set a target for each. Use the ones that match the job. A listing pilot should move conversion and organic rank. A PPC pilot should move TACoS and total sales. Write the targets in the scope, signed off by both sides, before the first change goes live.
Use relative change for conversion: 10% to 11% is a 10% relative lift. Pair any ad number with a sales number, because ACoS can fall while total sales fall too.
Listing pilot targets
Conversion rate up 8% to 15% relative versus baseline, organic rank up on at least three of the five keywords, and total sales up, all against the comparison ASIN.
PPC pilot targets
TACoS down one to three points at the same or higher total sales, or total sales up 10% or more at the same TACoS. Less wasted spend on search terms that never convert.
Process targets (every pilot)
Plan on time, changes on the dates promised, both reports on time, every change logged, and problems raised before you find them.
Worked example (made-up numbers). Baseline month: 6,000 sessions, 10% conversion, 600 units at $30, so $18,000 in total sales. Ad spend $2,700 and ad sales $9,000, so ACoS 30% and TACoS 15%. Last 30 days of the pilot: 6,300 sessions at 11.2% conversion is about 706 units, or $21,180. Ad spend $2,800 and ad sales $9,800 gives ACoS 28.6% and TACoS 13.2%. Conversion rose 12% relative, total sales rose $3,180 a month, and TACoS fell 1.8 points. If the comparison ASIN was flat, that is a pass on a 10% conversion target.
Is it worth it? At an example 30% margin before ad cost, $3,180 in extra sales is $954 in extra profit, minus $100 more in ad spend, so about $854 a month. The $1,985 pilot pays back in a little over two months if the lift holds.
From Baseline to Decision in 60 Days
Week before day 1: record the baseline
Download 30 days of sessions, conversion, ad spend, ad sales and total sales for the pilot ASIN, plus a comparison ASIN. Screenshot organic rank for five keywords. Send the agency a copy.
Days 1 to 7: agree the plan in writing
The agency audits the ASIN and sends a written list of what it will change, on what date, and which numbers it expects to move. Nothing goes live before you approve it.
Days 8 to 21: changes go live
New copy, images or campaign structure go live on the dates in the plan. The agency logs every change with the date, so you can line results up against it later.
Days 22 to 35: let the data settle
No new listing changes. Bid and budget changes only. Amazon needs a few weeks of traffic before conversion and rank show a real trend.
Day 30: midpoint report
The agency reports every baseline number against the first 30 days, explains what moved and what did not, and says what it will adjust in the second month.
Days 36 to 59: tune and hold
Bid, keyword and budget work continues. Any second listing change is agreed in writing first, so you can still tell which change did what.
Day 60: final read and decision
Compare the last 30 days to the baseline and to the comparison ASIN. Score the agency on the sheet below, then expand, extend or end the pilot.
15 Questions to Ask Before You Pay for a Pilot
1. Which ASIN would you pick for the pilot, and why?
Good answer: a reason tied to your numbers. A good pilot ASIN has steady sales, at least 60 days of stock, a price you will not change, and room to improve.
2. What exactly is in scope, and what is not?
Good answer: a short written list. For example: "keyword research and new title, bullets and backend terms; Sponsored Products restructure; no images, no A+ Content." If it is not on the list, assume it is not included.
3. What is the fixed price, and is anything billed on top?
Good answer: one number, paid once or monthly, with ad spend named as a separate cost. Ours are on our pricing page.
4. What baseline will you record, and will I get a copy before day 1?
Good answer: "Yes, here is the sheet." It should list sessions, conversion, ACoS, TACoS, total sales and organic rank, all for the same 30 days.
5. Which numbers define success, and what are the targets?
Good answer: two or three numbers with a target each, written down before work starts. For example: conversion up 10% relative, TACoS down two points, total sales up 10%. One number alone is easy to game.
6. What will you change, and on what dates?
Good answer: a dated plan. Changes on known dates let you line up the data.
7. What access do you need?
Good answer: the narrowest role Amazon allows for the job, added as a user in your Seller Central account. You keep ownership and can remove them on day 61. Campaigns stay in your ad account.
8. Who does the work, and how many hours is it?
Good answer: a name and a number. Ask how many other accounts that person carries.
9. What will you report, and when?
Good answer: a midpoint report at day 30 and a final one at day 60, both against the baseline, with a dated log of changes.
10. How will you separate your work from seasonality and price changes?
Good answer: they suggest a comparison ASIN, a check against the same weeks last year, or both.
11. What happens on day 61?
Good answer: nothing automatic. The pilot ends, you decide, and any monthly fee stops unless you renew. Read the terms for auto-renewal before you pay.
12. If the pilot misses the target, what then?
Good answer: a plain explanation of why, using the data, and what they would do differently. Beware an agency that promises to "keep working for free."
13. What would make you stop or pause the pilot?
Good answer: real examples. The ASIN runs low on stock, it gets suppressed, a price war starts, or reviews drop.
14. What do I keep when the pilot ends?
Good answer: the copy is live on your listing, the campaigns are in your ad account, and you get the keyword research as a file. Ask for this in writing.
15. If it works, what does the full account cost?
Good answer: the price for the next step, quoted now. A cheap pilot followed by a surprise retainer is a sales tactic, not a test.
Listen for numbers, names and dates. "Conversion up 10% by day 60" is an answer. "We will optimize the listing" is not. A vague answer on the call should turn specific in the written scope. If it does not, the scope is what you will get.
How to Compare Two Agencies Fairly
You cannot pilot two agencies on the same ASIN at once, and running them one after another puts them in different seasons. There are three fair ways to compare.
Two similar ASINs, same 60 days
Give each agency a product with similar sales, price and category. Same dates, same baseline sheet, same targets. Compare relative change, not raw sales, since the products differ.
One pilot, one written proposal
Pilot your first choice. Ask the second for a written plan for the same ASIN and the same targets. At day 60, see whose plan matched what the data showed.
Compare the process, not just the result
An agency that hit every date and explained a miss clearly is a safer bet than one that got lucky and reported late.
Five Signs a Pilot Is Set Up to Fail
No baseline before work starts
If the agency is happy to start without recording the before numbers, it is planning to report whatever looks best at the end.
A pilot that rolls into a retainer automatically
Read the terms. A pilot should end on day 60 with a decision. An auto-renewing contract is a retainer with a short first month.
A guaranteed rank or ACoS
Rank and ACoS depend on competitors, price and reviews. A guarantee usually means a target set low enough to hit, or ad spend cut to hit it.
Access you cannot remove
Campaigns built in an agency-owned account, or a request for your main login, mean you cannot cleanly walk away. Add them as a user in your account.
Results only in ad metrics
A report with ACoS, clicks and impressions but no total sales or TACoS cannot show whether the product grew.
Score the Pilot Out of 20
Fill this in at day 60. Score each line 0, 1 or 2. Zero means it did not happen; one means late or partial; two means on time and in writing. Use the same sheet for every agency, including us.
When a Pilot Will Not Tell You Much
You are under about $20,000 a month in sales
Fees are a large share of margin at this size, and a single ASIN may not get enough sessions for a change to show. Use the free audit and fix the listing yourself first.
The pilot would run through Prime Day, Black Friday or your peak season
Event traffic swamps whatever the agency changed. Start after the event, or compare to the same weeks last year.
You plan to change price, run deals or launch a variation
Each one moves conversion on its own. Hold them until day 60 or the result means nothing.
The ASIN is low on stock or suppressed
Fix supply and listing health first. An agency cannot win on a product that cannot sell.
About the Author
Ian Smith
Founder, Evolve Media Agency
Ian has run Evolve Media Agency since 2017 and has spent more than a decade in ecommerce, building and selling his own brands and producing listing content, video and ads for Amazon, Shopify and TikTok Shop brands. He writes every page on this site himself and the prices quoted for Evolve are the prices you pay.
Agency Pilot Questions
How do I test an Amazon agency before signing a retainer?
What should a trial project with an Amazon agency include?
How long should an Amazon agency pilot last?
Which metrics should I use to judge an agency pilot?
What is the difference between ACoS and TACoS?
Should an agency pilot be free?
Can I test two agencies at the same time?
What does Evolve Media Agency charge for a pilot?
Run the Pilot Method on Us
Book a free 30-minute call. Bring one ASIN and its last 30 days of numbers. We will tell you which project fits it, what we would change and when, and the targets we would sign up to. If a pilot will not tell you much yet, we will say so.