The regime-matched redo — what survives when budget, hours and cohort age are all held

1The result in one line

Nothing in this account establishes which of these two creatives is better, and four separate attempts to say otherwise have each failed for a different reason.

The geography hypothesis #47 raised is refuted: inside the $875 regime bikini drew less India than bodysuit — 31.8% against 39.3%, p = 0.007 — and the value of that mix difference is −0.9%, 95% CI [−21.2%, +27.5%], indistinguishable from zero. There is no geography correction to make, in either direction.

But the gap it was meant to explain cannot be read either. Revenue per install has a coefficient of variation of 6.68; separating two creatives at the observed gap needs about 19,000 installs per arm and the best window this account has produced had 732 and 488, with bootstrap ROAS 1.18× [0.64–1.82] against 1.45× [0.54–2.58].

And the delivery-side verdict does not survive either. On every stage a creative actually controls — CTR, click→install, payer rate — the two are indistinguishable. The only measure that separates them is CPM, which is set by the auction rather than the video, and which appears when Meta ramps an ad and disappears when it stops (0.98× at $1.6/hour, 1.43× above $20/hour, R² = 0.79 on the ad's own hourly spend).

Underneath all of it: Meta's allocation is not evidence. At the $875 step it had two parked arms, reactivated the one with 1 purchase over the one with 13, and escalated it to 76% of budget while its cumulative ROAS sat at 0.21. That was capacity — one arm could not spend $875/day — not judgement.

2The test #47 called for

Regime R4 ($875/day, 07-31 17:21 → 08-01 04:10 PT) is the one window where both creatives delivered at volume, on the same ten hours. Install-level country comes from the Lum registration channel; each install is attributed to the delivery hour that produced it.

bikini skewed away from India, not into it — and the difference is not worth a measurable amount. The India gap itself is solid, on 732 and 488 installs. Its price is not. The country values it would have to be converted with are themselves estimates: India's revenue per install rests on a few hundred installs, the US on a few dozen, and a single large payer moves either. Resampling the pooled installs and rebuilding the country table from each resample puts the value gap at −0.9%, from −21% to +28% — spanning zero, and spanning it widely.

That is a better answer than either direction would have been: the creative comparison needs no geography correction, because the geography difference cannot be shown to be worth anything.

The exposure confound was nonetheless real — which is why the question was worth asking:

bikini lived almost entirely inside the $875 regime while bodysuit spread across four. So every pooled lifetime comparison of the two was confounded exactly as #47 suspected. Standardising both onto the account's regime distribution moves bikini from 31.5% to 23.6% India and bodysuit the other way. The confound was there; its sign was the opposite of the one assumed.

3Pricing the mix — and what it is worth

A shift in country shares is only a confound if it moves money. Each ad's own mix is valued at pooled country prices built from the same cells: shares from the ad, prices from the pool. An ad's revenue inside one country is not measurable here at all, so it is never used.

Two things fall out, and one non-thing:

1. The entire ROAS gap is acquisition cost. bikini paid 21% more per install than its own audience implies while bodysuit paid 7% less — a 30% swing on the cost side, against a revenue side where both sit below their mix expectation by a similar margin.
2. The revenue side does not separate them. The bootstrap intervals overlap across almost their whole length, which §6 shows is not a property of this window but of the measure.
3. There is no mix correction to apply. Both arms' expected revenue per install land within 4% of each other, and that 4% carries an interval from −21% to +28%. A "mix-neutral ROAS" computed from a single country table reads −14% or −22% depending on which population the table is built from; neither number is real. The raw −19% stands as the gap, uncorrected, because geography has nothing measurable to say about it.

4Where the cost difference sits — and why it is not the creative's

CPI = CPM/1000 × 1/CTR × 1/(click→install). Over the same ten hours:

The hook is not the problem: CTR differs by 3% and click-to-install is 6% better for bikini. The whole gap is impression price. Two controls narrow it and one dissolves it.

Hour mix — standardised onto pooled impressions per hour the gap is +38%, and bikini is the dearer in nine of the ten hours. Placement — day-level only, but on 07-31 each ad's CPM sits within 5% of what its own placement mix implies (0.95 and 1.02). Geography — pricing each arm's own country mix at pooled country CPMs, geography predicts bikini should run +11%; it runs +35%. That leaves +21%.

But bikini was being spent 1.65× harder in those hours, and the premium tracks the push:

log(CPM ratio) on log(bikini's own hourly spend): b = +0.132, se 0.019, R² = 0.79. Doubling the push raises its CPM 10%. The trajectory shows it directly: in its first hour back, at $1.6/h, bikini's CPM is 0.98× bodysuit's — level. Meta ramps it to $39/h and it runs 1.3–1.7×. Spend collapses to $4.5/h at 05:00 and it returns to 1.01×. The premium appears with the ramp and vanishes when the ramp stops, twice.

One hour has the two at genuinely matched spend — 07-31 19:00, $19.0 against $18.0 — and it shows 1.31×. So a residual cannot be ruled out. But n = 1, because the two ads were almost never run at comparable spend in the same hour: Meta was continuously shifting budget from one to the other.

CPM is a price the auction sets, not a property of the video. In a CBO the bid comes from Meta's own predicted conversion rate, so a higher CPM can mean Meta bidding up for an ad it favours or charging more for one it rates poorly — opposite mechanisms, and Meta's own quality, engagement and conversion-rate rankings are blank on every export for these ads, withheld below a volume threshold none of them reached. It cannot be read as a creative verdict in either direction.

5Every stage a creative actually controls

Paired by clock hour with spend depth regressed out:

The raw CTR difference is −0.25pp on 34,345 against 23,398 impressions, p = 0.275 — not resolvable even at that volume, because it is genuinely tiny. On every stage the creative controls, the two are indistinguishable. The only thing that separates them is the price of impressions, which the creative does not set.

The intervals still bound it: CTR cannot differ by more than ~25%, click→install by more than ~34%. Whatever separates these two videos is small — which is what the stage-2 DACH proposal predicted when it established that all 252 clips are one shot, with face and top the only things that have ever varied.

6Meta's allocation is not a quality signal

At the $875 step Meta had two parked arms and this on its books:

pool5 was still converting while parked — 2 further purchases worth $18.73 on zero spend, the 7-day-click tail. Meta reactivated bikini and gave pool5 zero for the remainder of the run. The obvious mechanical explanation fails too: pool5 is 15s against the others' 10s, but on 07-30 it ran in all the same placements at comparable shares, so there was no eligibility wall.

Then it escalated bikini through a stretch where the evidence was against it:

bodysuit at that same 18:00 decision held 25,075 impressions, 11.29% CTR, 73 purchases, ROAS 1.81. Meta moved to 51% on one hour of 211 impressions and zero purchases, and kept raising while bikini's cumulative ROAS sat at 0.21–0.29.

What changed at 17:21 was the budget, not the evidence. $875/day needs $36.5/hour sustained; bodysuit's peak hour before the step was $28.5, and even running completely alone the next day it averaged $28.1. One arm could not hold the budget, so Meta opened a second outlet — and the arm it opened was not chosen on performance.

Consequence: allocation carries no information about creative quality, in either direction. It was equally wrong for this account to read "Meta parked bikini" as evidence against it.

7The finding that governs the rest — revenue cannot referee this

The R4 window is the best-powered creative comparison this account has run: one regime, matched hours, both cohorts fully mature, the two largest install cohorts on record. Bootstrapping over installs:

The intervals overlap across almost their whole length. Revenue per install is a near-zero vector with rare large entries — 4.2% of installs pay anything — so its dispersion is enormous relative to its mean:

n = 3,179 · payers 4.2% · mean $0.454 · sd $3.029 · coefficient of variation 6.68

Resolving the −19% actually observed needs ~19,000 installs per arm. The window had 732 and 488 — 4% and 3% of the requirement. No ROAS comparison between two creatives in this account has ever been close to resolvable, including the ones computed in this document. The delivery side, measured on tens of thousands of impressions rather than tens of payers, resolves a 0.5pp CTR difference on a single night.

Read the requirement as an order of magnitude, not a figure. CV falls as the horizon lengthens and more installs have paid, so the 20% requirement runs ~18,600 at H = 6, ~17,500 at H = 12 and ~11,600 at H = 24. Beyond that the account cannot measure it — only 40 vid_ww_* installs are 48 h old and one of them has paid. It does not drop below ~10,000 per arm anywhere in that range, and 732 is 4–6% of it throughout.

Horizon coverage is the other half of this. A ROAS quoted at a horizon says nothing about the spend sitting in cells too young to have reached it:

Parenthesised is the share of that ad's lifetime spend old enough to have reached the horizon. bodysuit's whole-life figures describe 63% of its money at H = 12 and 40% at H = 24, because most of its spend is recent; bikini's H = 24 number describes 1% and is not a number at all. Only the R4 window above has both arms at full coverage, which is why it is the only comparison quoted.

So creative selection runs on CPM and CTR. ROAS sizes the account, not the creative.

8The account, event by event

Budget changes are not the only events. An asset starting or stopping delivery changes the auction for every other asset in the ad set, so a window straddling one compares two different competitive fields. Cutting at the union of both — every budget step, every ad's first and last delivering hour, every gap where the optimiser parked one — gives the run's actual shape.

⚠ All three creatives were live from launch. The only operator switch in this window is bikini at 06:55 on 08-01; bodysuit followed at ~17:00, past the data cut. Everything else below is Meta reallocating inside the CBO ad set — an ad at zero delivery is still live and still competing, it simply got no budget that hour. That distinction is the Gen 5 finding: the disjoint delivery windows were the optimiser's doing, not anyone's hand. Operator on/off is in no export either; Ad delivery is stamped at export time, reading active for the same hour in a file pulled while the ad ran and inactive in one pulled after it was paused.

← marks the segment in which that budget took effect. Audience value is the segment's own country mix priced at pooled country rates — the only column not confounded by which ad earned it.

Meta parked two of three creatives on its own, at $150/day, within six hours of launch — and brought bikini back 39 minutes after the $875 step. No one touched the ads. That is why the three arms have disjoint delivery windows, and why every per-asset figure computed across them describes Meta's schedule rather than the creative.

What the regions did. India tracks the budget and reverts, exactly as #47 found: 18% at $250 → 30% at $500 → 39% at $875 → 29% back at $500, holding the creative (bodysuit) fixed throughout. The composition effect is unambiguous.

What it was worth is a different matter. Across the whole run the audience's value moves between $0.316 and $0.657 — but the extremes are the two thinnest segments (43 and 38 installs), and across the four large ones it sits in $0.445–$0.576, a spread of about 25%. Over the same span CTR falls from 12.0% to 7.6% and comes back to 8.8% — a 37% swing on a quantity measured over tens of thousands of impressions rather than hundreds of installs.

So the region story is the smaller half of the story. The budget does move the geography, the geography does move the audience's value, and that channel is worth roughly a quarter at the extremes. The larger and far better-measured effect is that at $875 Meta bought cheaper impressions from much lower-intent people inside the same countries — CPM fell 60% while CTR halved, so cost per install rose anyway.

The one real off-switch. After bikini was switched off at 06:55 on 08-01, bodysuit alone at $500 shows CPM $8.74 against $6.62 in the preceding both-live segment, on a near-identical India share (29% against 28%). It inherited the budget bikini had been absorbing and had to buy further up its own cost curve. ⚠ That segment is also daytime where the previous one is overnight, and §6 shows hour-of-day moves CPM on its own, so this is consistent with a real effect rather than evidence of one — the clean version needs an off-switch held at matched clock hours.

9The budget, re-examined

#47 established that the budget moves the country mix and read the account's decline off that. The mix moves; the question left open was what it is worth. Pricing every regime's pooled mix at account-wide country values:

The mix swings hugely and is worth very little. India runs 14.9% to 34.6% across the regimes — a 20pp swing — while the value of the resulting audience spans only 1.45 to 1.67, and the $875 regime prices out 6% above the $500 one. The countries that replace India are not systematically better: India's revenue per install ($0.410) is only 12% below account average, and its cost per install ($0.347) is 7% above, so it is a worse market by about a fifth, not by a multiple.

The budget's real damage is visible when the same creative is held on the same clock hours:

Tripling the budget did not buy the same people dearer — it bought far more, far cheaper impressions from a far lower-intent audience. CPM fell 60% while CTR halved, so cost per install still rose 15%. Country mix accounts for only −12 of the −34 percentage points of lost revenue per install; the rest is audience quality within the same countries, which is what the CTR collapse is measuring. Country targeting would recover about a third of what the budget step cost.

The budget is a cap; realised spend is the treatment. "Same $500/day" hours one day apart differed by 1.34× in what Meta actually spent, and CTR fell 17.7% across them. Pairing by clock hour and regressing on the spend ratio: spend elasticity −0.724 [−1.31, −0.14], residual day effect +1.3% [−16.8%, +19.4%]. Spend predicts −18.8% against −17.7% observed. There is no decay — every comparison must match realised spend, not the dial.

⚠ The revenue columns in that table are not reliable and the delivery columns are. An equal-budget control — same ad, same clock hours, $500 on both days — moves observed RPI by −75% and ROAS by −73%, as much as the budget contrast, while CTR moves only −7% and India only +2.7pp. At ~150-install cells the revenue side is noise. The CPM, CTR and mix figures are the findings here; the ROAS figures are illustration.

10pool5, before it is read again

pool5's 3.01× was a projected cohort ROAS on 151 installs. Two days on, it can be checked against realised money instead of re-argued. Its 07-30 cohort is now essentially complete — 100% of its revenue arrived within 48 h, 50% within the first hour.

Not a mix artifact. The $150 regime it ran in prices at 1.48 expected ROAS against the current $500 regime's 1.45 — a 2% difference. The regime was not unusually generous, so the number is not explained by where it was shown.

Also not a result. Eleven payers, one of whom — a Cambodian account making four purchases — is a quarter of the revenue. Against §6's table, 151 installs resolves nothing at all. pool5 has been running alone at $500/day since ~17:00 on 08-01 and none of that run is in this data; the Telegram reconstruction stops at the 08-01 16:38 export.

11What this changes

1. The cut of ww_bikini_10s had no evidential basis — in either direction. Not the 0.82× (a calendar-day artifact), not 1.13× vs 1.45× (unresolvable), and not the CPM premium, which is a function of how hard Meta was pushing it. The cut may have been right; it was not justified, and those are different things. The one comparison that holds spend, hours and region — payer rate, Mantel-Haenszel OR 3.17, p = 0.096, higher in all four shared countries — leans the other way, and bodysuit's ROAS edge is three payers in Belgium, Slovenia and Switzerland worth 55% of its R4 revenue.
2. Rank creatives on CTR, and only CTR. It is the one creative-controlled metric that resolves at buyable volumes. CPM describes the auction. Revenue needs ~17,500 installs per arm — $6,100 of media each — for a 20% read.
3. Never read allocation as quality. Meta's spend split is driven by budget capacity and its own exploration state, and at the one moment we can audit it, it chose the worse of two arms.
4. Condition on realised spend, not the budget setting. The budget is a cap; what Meta actually spent in the hour is the treatment. "Same $500/day" hours differed by 1.34× in real spend, and correcting for that removes the apparent day-over-day decay entirely (residual +1.3%, CI [−16.8%, +19.4%]). #47's "the campaign never decayed" stands.
5. #47's mechanism is right and its weight was wrong. The budget moves the country mix near-deterministically and reversibly — bodysuit alone, same clock hours, $250 → $875: India 19.1% → 41.7%, CPM 14.01 → 5.60. But priced at account country values the mix channel is worth ≤15% of expected ROAS, and about a third of the CTR fall; the rest is within-country audience quality. And it was not bikini that brought the cheap Indian traffic — bikini was the less-Indian, higher-CPM arm, and adding it diluted the account's India share.
6. The design cannot answer the question, and no analysis will fix that. Meta selects within every stratum we can observe, on a predicted-conversion signal absent from every export. Only randomising the audience — Meta's split test — identifies a creative effect. Equal budgets in separate ad sets do not, because each ad set still optimises internally.

12Reproduce

Everything above is one command. The unit is (ad, delivery hour); every table is a selection and a grouping over it (ad_ops/asset_estimator/cohorts.py, views in views.py).

cd C:/Projects/VideoGenAI_Related
L=_runs/2026-08-01_account_recheck

python -m ad_ops.asset_estimator hourly     --meta ad_ops/_data/b1_meta_exports/B1-Ads-by-hour-Jul-30.csv            ad_ops/_data/b1_meta_exports/B1-Ads-Jul-31-2026-Jul-31-2026.csv            $L/ads_hourly_aug1.csv     --lum $L/lum --fx $L/fx_rates.json     --ads "ww_bikini_10s,ww_bodysuit_10s,ww_pool5" --horizon 12

It refuses rather than guesses in two places: an export with no Time of day column raises (folding a day into one cell is the error the unit exists to remove), and a cohort younger than the horizon is excluded and its spend reported as missing coverage rather than truncated into the total.

The scaling test (§3), the matched-clock-hour budget contrast (§6) and the pool5 cohort reconstruction (§7) remain one-off probes in _runs/2026-08-01_regime_matched/; they answered a mechanism question once and are not part of a cycle.

Data gaps this ran into, in the order they bind:

• No hourly export for 07-30 after 19:00, so the $150 regime is only 5 of its 7 hours. The installs that produced most of pool5's revenue sit in the missing two, which is why §7 uses Meta's full-day spend rather than the cohort table.
• Delivery stopped entirely for the whole 04:00 hour on 08-01, at the $875 → $500 step — all three exports agree on zero impressions. 15 installs registered inside it, carried back to the 03:00 cell that produced them. 79 installs across the run needed that carry-back; 58 more had no delivering hour within three hours and are excluded.
• Placement breakdown is day-level only; it cannot be cut by regime. An hourly x placement export would close the last open mechanism for the CPM gap.
• adjust_sink is still dead, so all of this rests on a manual Telegram export with a 2026-07-11 floor and an 08-01 16:38 ceiling.