Describe a rule in plain language, review the draft

The AdOps AI rule builder turns a plain-language goal into draft rule tasks. AdOps injects its live 45-metric catalogue into the prompt so the model works from the real grammar, then appends the returned tasks to the rule you are editing with a confidence score from 0 to 100. The draft is a starting point you review, not an approved rule.

What does the AI rule builder actually do?#

You open the Generate with AI modal from the rule form and describe the outcome you want. The modal carries example goals to calibrate the detail level, and a staged progress bar narrates six steps while the request runs, from Analyzing your goals through Finalizing recommendations. The modal locks while a generation is in flight so a half-written rule cannot be closed out from under you.

Behind that, AdOps builds a system prompt from the metric catalogue stored in its database, sends it with your text to the AI service, and reads the tasks and confidence score off the response.

Why is the metric catalogue in the prompt?#

Because a model asked to write rules for a system it cannot see will invent field names. AdOps avoids that by serialising its current catalogue on every request: each of the 45 metrics with its id, display name, category, unit, whether it supports a lookback period, its Meta API field and the exact operators it allows.

The prompt states the constraint plainly, requiring the model to use only metric ids that exist in the supplied schema and only the operators listed for each metric. Because the catalogue is a database collection rather than a hardcoded list, deactivating a metric removes it from the next generation with no prompt maintenance.

The AI service itself runs on a Claude model through LangChain with extended thinking enabled and a default budget of 10,000 thinking tokens, so the model reasons through the schema before emitting JSON rather than answering in one pass. Model name, token ceiling and thinking budget are all environment settings.

What the AI rule builder does not do#

This is the part worth reading twice.

AdOps does not validate the model’s output against the metric grammar. The response has its code fences stripped and is parsed as JSON; if parsing fails the raw text is returned instead. There is no schema check confirming that a returned metric id exists, that an operator is permitted for that metric, or that a period is valid for it. A generated rule is a draft, and the reason it lands in the ordinary editor is so that you can read every condition before setting it live.

The model’s reasoning trace is captured by the service and stored, but it is not rendered in the interface. What you see is the tasks and the confidence score.

The confidence score is the model’s own estimate. It is a useful triage signal for deciding which drafts to read carefully, and it is not a prediction of campaign outcomes.

How does a generated rule reach the editor?#

Generated tasks are not a separate artifact living in a preview pane. They are built into the same task list the rule form uses for hand-written tasks, with the same action picker, the same action frequency control, the same condition rows and the same value fields.

Each generated task renders with an AI Generated badge and a confidence chip showing the score as a percentage, and it stays editable. Change a threshold, swap the period, delete a condition, add one of your own. When you save, it is simply a rule.

The practical workflow is short: describe the goal, read every condition the draft produced, correct the thresholds against your own economics, save as draft, then set live once the estimated match count looks right.

What gets logged?#

Every generation writes an AI log row before the call and updates it after, storing the request, the response body, the response time in milliseconds, the confidence score and a status. Generations made while editing an existing rule are attributed to that rule, so the log can be filtered to one rule’s generation history.

That log is queryable per user, newest first, 10 rows per page by default. It answers what was asked, what came back, how long it took and how confident the model claimed to be.