Hello and welcome to our third and final post in my series on using Claude to manage my Amazon Ads. If you’re just joining me for the first time, in Part 1 I gave an overview of Amazon Ads and ran through my initial exploration of using Claude Code to handle reporting on my ad performance and finding new targets for ads. In Part 2, I took what I learned in Part 1 and turned the processes into skills that I could use on my other brands.
Today I’m going to talk about a few more pieces of functionality I’ve added to Claude’s Amazon Ads repertoire and some of the infrastructure needed to keep the whole thing humming along. I’ll wrap it all up with some thoughts on what this all might tell us about how AI might impact the broader economy.
Creating Campaigns
In Part 2, I broke out Claude’s skills into a one-time audit that finds new targets to advertise against and a regular review skill. The latter still needs some polishing (more on that later), but Claude is really good at coming up with new targets.
Given that, the next step was to have it start actually making changes to my ads account in order to create all of the new campaigns it proposed. Always a little nerve-wracking to let your LLM start updating things in critical systems, but once more, unto the breach and all that.
I wanted to be as thorough as possible here to avoid any potential pitfalls, so I went through the process of creating a new campaign via the Amazon UI, recording every field and the appropriate value for it. This is really, really important, because if Claude doesn’t have specific instructions it will not infrequently skip things or guess at what it should do and not inform you until you ask.
With Amazon Ads, there are settings like this one:
When you’re creating a campaign by hand, you realize that of course you don’t want it spending a bunch of money to advertise your products outside of Amazon in places where they perform poorly. But Claude, as discussed, will not necessarily show that level of common sense and is liable to think that of course you want your ads to reach the masses. Thus, we need specific instructions covering every single setting.
Having gone through that, I had Claude create a new campaign with my detailed instructions then checked it carefully. Mostly good, except in some cases where Amazon did not have a recommended bid for targets (for simplicity I just told Claude to start at the recommended bid), it decided to use numbers that were bizarrely high. This is the sort of thing we very much want to avoid!
Otherwise, all good, so I added an instruction on how to handle targets with no recommendation, then I had it build out a campaign creation skill. One important note, though — for each campaign to be created, Claude gives me a list of all campaign-level settings, all keyword/ASIN targets, Amazon’s recommended bid for each and the bid that Claude was going to use. Only once I approve does it actually create anything.
The first couple of times I used it, I did thorough checks of everything it created, and when those were all good I moved to spot checks, and now I’m just lazy and check nothing. It’s funny how quickly it goes from being a little unnerving to let an LLM make changes to something that’s critical to my business to just being completely accustomed to it. The human capacity to adapt to a new normal is truly something.
Ongoing Campaign Monitoring
The other piece that I’d like Claude to handle is the day-to-day management of the ads, because that’s more time consuming and frankly boring. Unfortunately, Claude has no intuitive sense for the task, and it turns out that trying to document everything I do creates a skill that’s too long for it to follow reliably.
The problem is there’s just too much variance in what you need to do in order to analyze the performance of a given keyword depending on the context. You need to look back far enough to have enough data, which could be 7 days or 30 days or 60 days depending on how frequently-searched it is. But if the 30 days you’re looking at has a sale period like Prime Day in it, that’s going to throw things off. You also have to balance changing both the bid for the keyword itself and the placement adjustments. You must consider the impact of changes outside of that campaign entirely, whether that’s new campaigns you’re running that are pulling traffic away or new competitors that have popped up.
This is all just too much for Claude, which is somehow both surprising and not. None of the logic is complicated, there’s just a lot of it. I could perhaps codify all of it into a deterministic system and have Claude script that, but this is one of those times where it feels like the best approach is to ignore the problem and wait for a better model to come along that will magically solve it.
That said, its limitations have led me to think more carefully about what I can scope clearly and succinctly enough for it to accomplish. One tedious but important task is checking on new campaigns regularly to make sure they’re spending money. Typically, when you launch a new target at Amazon’s recommended bid, that bid won’t be high enough for it to get any impressions. I haven’t the foggiest why Amazon’s recommendations are so frequently bad, but that’s neither here nor there.
Thus, after launching a new ad campaign, you have to go in every couple of days, check bids, and increase the bids of any target that hasn’t spent anything by 10% or so. Claude can handle that! So I now have another skill just for that, and it’s not glamorous but it saves me an hour every couple of days.
I also had Claude wire up a dashboard for me that helps me more efficiently go through all of the things that I need to check to determine if I should modify bids or not. Amazon’s own UI is terrible and slow and presents information in ways that feel like they were designed by someone who has never actually used the ad system. Mine is very ugly, but it’s perfectly laid out for the way that I need to use it, which is an incredible benefit of being able to get custom software made.
On the one hand, getting a dashboard when I really wanted a digital intelligence to do the job of managing my ads is somewhat disappointing. On the other hand, the dashboard I have would have cost thousands of dollars if I were hiring a developer to build it in the pre-LLM days, so I can’t really complain.
Adding Some Infrastructure
As I started working on this project, Claude frequently hit Amazon’s rate limits. They’re not especially generous to start with, and because your Amazon Ads API key is tied to your user account (as opposed to their other APIs, for which it’s tied to the actual Seller Central account), I was sharing one set of rate limits across every brand.
The other very annoying thing is that Amazon will only let you pull ads data for the last 90 days. There are many cases where I’d like to look farther back, but even though I can see lifetime data in the UI, Claude was out of luck.
So I set off on a task that took much longer and was more involved than I anticipated but ultimately worth it: saving all of my historical ads data to a database, starting with a backfill of the 90 days available and then running a job that adds to it daily. Having it stored outside of Amazon would not only save Claude from having to hit the APIs whenever it needed to find something, but also allow me to build up a longer-term historical record.
I’m going to compress this slightly, because it really was a saga, but it’ll give you a pretty good idea of what it’s like to work with Claude on this kind of project.
First, I told it to pull all available historical data, save it to a SQLite database on my hard drive and back that file up to GitHub. Neither Claude nor I gave this plan the level of consideration that it warranted before it got started.
The backfill progressed slowly. Painfully slowly. I left that Claude Code tab open for six hours and it had made shockingly little progress. I asked Claude what was going on, and it said it was dealing with rate limits.
That did not feel like it was a sufficient explanation for how glacially slow things were moving. I prodded. Eventually Claude told me that unfortunately given the rate limits and the fact that it had to retrieve data one day at a time across a number of brands meant that this was going to take a while.
“Is there really no way to get more than one day of data with each API call?” I asked. Claude had not considered this. It investigated. Turns out you can get 30 days of data with each API call. I couldn’t decide whether I should feel frustration at its ineptitude or joy at knowing that for at least a little while longer, I could be smarter than a SOTA LLM.
Next problem: GitHub has a 100MB file size limit that we hit pretty quickly. Claude did some research and found that Turso should be able to handle something much larger for free. I’d never heard of it, but if it’s good enough for Claude, it’s good enough for me! Probably! Unless something goes wrong!
Finally we got the whole backfill completed.
From there, we moved onto building a daily job that pulls yesterday’s data and adds it. Claude wrote the script and set it to run via GitHub Actions. I went to bed, and the next morning it had worked as expected, though it took several hours. That seemed too long, but it wasn’t actually a problem for me, so I let it be.
Later in the month, when I was notified by GitHub that I only get a certain number of free GitHub Actions minutes and had exceeded my quota, it became a problem for me, so I investigated.
It’s been a little while since I dealt with this, so I don’t remember the full order of things that went wrong here. They included:
Claude was writing the data to a local SQLite DB then pushing it to Turso. Which was fine, except it was pushing an update every single time it added something instead of just waiting for the full daily update to be complete. Turso kept closing the connections because they were open for so long, and Claude did not handle that well.
At one point I built a dashboard on top of this data. Claude decided it would be most efficient to have the data in a local DB for the dashboard to use rather than read directly from Turso. So it created one and started to sync the data from Turso daily. You may recall from the previous bullet point that there was already a local DB. Claude had created a second one and was now pulling data from Amazon, storing it on the first local DB, pushing it to Turso, then later syncing it back to the second local DB.
I only figured out the two DB thing because as I continued to make changes to things touching the DBs, Claude got confused about which one to use and started writing to the backup one, which was not syncing to Turso.
Eventually I finally spent half a day figuring out what was actually going on instead of just telling Claude to fix whatever the random problem of the day was. We got one coherent flow of data in place and everything that used the data pointing at the same local DB, and to boot I actually went through what was being captured and got rid of a bunch of stuff that was duplicative.
There is probably a lesson in here. Certainly the software engineers reading this are feeling rightfully smug about the fact that this sort of comedy of errors is what you get when you just start YOLOing things with LLMs. Honestly, if I had spent a little more time reading the stuff that Claude was writing before telling it to start building, I probably would’ve seen some of these issues coming.
But if we’re being honest here, I know that at some point in the not-so-far future, LLMs and their harnesses will have improved to the point that they will just get this sort of thing right the first time. As such, I decline to learn any lessons.
Where Things Stand and Next Steps
The long-term goal here is to fully delegate the management of my ads to Claude. As soon as I take a brand over, it audits the account, makes whatever changes it sees fit, and then checks back in at a regular cadence.
We are definitely not at a point where that’s anything close to a good idea. I’ve got Claude doing campaign creation, because that is definitely the place where it’s closest. But a big part of that is the fact that when you’re creating new campaigns, it’s a combination of two things that Claude Code is well suited to handle: rote, rules-based analysis and making stuff up.
The former is what it does when looking at existing campaigns to find keyword targets. It checks for search terms that are leading people to buy my products but that I’m not already targeting. This just involves pulling reports and doing some straightforward analysis. Claude isn’t actually doing the work; it just wrote the scripts that do the work.
Finding new products to advertise against is decidedly less formulaic, and I have actually been surprised at some of the positive results stemming from Claude’s creativity on that front.
I still haven’t really given it instructions much more specific than “look at Amazon’s product catalog and find products to advertise against,” and yet it finds things that I would not have considered. I have a brand that sells baby mobiles, and it created several campaigns targeting other mobiles by theme. One of these themes is mobiles that play music. The mobiles I sell do not play music. And yet, for some reason, that’s the best performing campaign.
Similarly, I sell a binder that is designed to hold seed packets. Before I bought the brand, the previous owner had an ad strategy that mirrored my typical approach — she targeted directly competing binders that hold seed packets. Claude decided to target stakes and trowels and anything related to gardening, and I’ll be damned if a bunch of those targets weren’t quite lucrative.
In fairness, a bunch of them were not lucrative, but that’s how these things go. You try a bunch of targets, some of them don’t work and you turn those off, but the ones that do make you money for hopefully a long time. I didn’t think creativity was where Claude would outdo me in my business, but there you are. LLMs are weird!
Eventually I’d like Claude to take over the day-to-day management of my ads, but we’re just not there yet. The challenges there really highlight the jagged frontier of LLM capabilities plus the limits of the value of raw intelligence. There’s nothing required that Claude isn’t smart enough to do; the problem is long-context instruction following.
I think we’ll see similar cases as AI rolls out more broadly through the economy. There will be places where sheer intellect will be useful, but I strongly suspect that the greater impact will be the ability to deploy an infinite number of pretty smart, very fast, always working agents.
Most of the work done in the world today doesn’t require genius-level intelligence, after all. Even in disciplines where we think of intelligence as being fundamentally important, it’s often not used with great frequency. My wife is a scientist, and while she and her peers spend a lot of time using their brains to come up with hypotheses and reason over data, they spend a lot more of it sitting in the lab pipetting and centrifuging (I don’t know if these are actually the things my wife does in the lab but they are the lab tools that I’m familiar with, so we’ll pretend), attending meetings and creating presentations. I was a product manager, which I choose to believe people think requires intelligence, but I spent a lot of time updating JIRA tickets and spreadsheets. It would have increased my productivity much more to have someone handle those tasks than to have them take over the time I spent pondering product direction and priorities.
Overall, I have deeply enjoyed this project both because it’s taught me a lot about how best to work with Claude and also because it has made me a meaningful amount of money. Three of the brands I own had their best months of revenue and profit after I unleashed Claude on their ads. Most of the ones that didn’t are seasonal and not at peak season, but they showed substantial YoY improvements. I can see the new ad targets generating additive revenue, and I can also see some of the cleanup work that Claude has done making other campaigns more efficient.
The impact of all of this in terms of dollars in my pocket is orders of magnitude more than my lifetime LLM spend so far, and the increase in profit should continue on an ongoing basis. I say that largely to highlight how different the impact of AI looks in large-scale economic numbers, where it seems to be ambiguous at best, compared to its totally unambiguous effect on my business. A big part of the reason I decided to start writing this Substack is that I think a tiny, simple business like mine is a useful leading indicator of how AI will affect the economy as a whole, and if you agree with that, then you won’t be surprised if we see some substantial changes in the next few years.