“Video increases conversion by 80 percent” has been circulating for more than a decade. Nobody can produce the study. The figure traces back to a vendor page that no longer exists and has been recycled through thousands of articles since.

That mattered less when a hero video cost four thousand dollars and three weeks of production. Teams tested it carefully because the spend demanded justification. Now that a usable clip costs a few dollars and four minutes, teams ship video on faith and never check whether it did anything.

I have run or reviewed roughly ninety video tests on landing pages over the past three years. The honest summary is that video lifts conversion about a third of the time, does nothing at all about half the time, and measurably hurts in the remainder. The variable that predicts which bucket you land in is almost never the creative.

The confound that invalidates most video tests

Adding video makes the page slower. That single fact ruins more experiments than anything else, and almost nobody controls for it.

Suppose your control is a static hero and your variant is the same page with an autoplaying clip. The variant converts 6 percent worse. What did you learn? Nothing, if the variant also loads 500 milliseconds slower, because a half-second delay on its own is enough to produce a loss of that size. You have run a speed test and labeled it a creative test.

Controlling for this is not complicated, but it has to be deliberate. Serve the first frame as a preloaded poster image so that the largest contentful paint is identical across both variants. Keep the video out of the critical request chain with a deferred preload value. Then measure real-user load times for both arms and report them next to the conversion numbers. My working rule is that if the 75th-percentile LCP gap between variants exceeds 150 milliseconds, the test is void and has to be rebuilt.

Two smaller confounds deserve the same discipline. Novelty inflates early results, so a variant that wins in week one and fades by week three was never a winner. And traffic mix shifts, so a test that spans a promotional email send is measuring the email, not the video.

Design the test before you generate anything

Most video tests fail at the arithmetic stage, before a frame exists.

Work out the minimum detectable effect first. If your landing page converts at 3 percent and you want to detect a 15 percent relative lift with 80 percent power at 95 percent confidence, you need roughly 14,000 visitors per variant. At 1,500 sessions per week, that is nine weeks per arm. If you cannot fund nine weeks, you cannot run that test, and the correct decision is to test something with a larger expected effect.

Video tests also need longer runtimes than copy tests for a structural reason. Video introduces higher variance in the visitor experience, because some people watch, some never trigger playback, and some receive a static fallback. Higher variance means wider confidence intervals and a slower path to significance.

Set a fixed horizon and refuse to peek. Calling a test the first morning it crosses 95 percent is the single fastest way to accumulate a portfolio of imaginary wins. If you need early reads, use a sequential testing method that accounts for repeated looks rather than checking a fixed-horizon test daily and pretending you did not.

What actually lifts

Across the tests I have seen, the effects cluster in a predictable order.

• Product demonstration on pages selling something hard to picture. Software interfaces, multi-step services, and physical products with a non-obvious mechanism. This is the largest and most repeatable win, often in the 10 to 25 percent range on primary conversion, because the video is doing work that copy genuinely cannot.
• Video placed near the decision point rather than at the top. On long pages, a clip sitting just above the pricing table or the form consistently outperforms the same clip in the hero. The visitor who scrolled that far has intent and is looking for a reason to commit.
• A human face on high-trust offers. Consultative services, high-ticket purchases, and anything requiring a visitor to hand over sensitive information. The effect is modest but stable.
• Short loops showing a single mechanism. A five-second clip of one feature working, placed inline next to the copy describing that feature, quietly outperforms a full explainer on feature-heavy pages. Visitors read the claim and see the proof in the same glance.
• Ambient background loops. Effect on conversion is close to zero in most tests. They lift time on page and they make a brand feel more considered, which are legitimate goals, but they should not be sold internally as conversion work.

What does nothing, or actively hurts

Autoplaying narrative video with sound is the most reliable loser I have measured. It startles people, it fails to play at all in most browsers, and the sessions where it does play show elevated bounce.

Hero video on mobile that pushes the headline below the fold hurts almost every time. The visitor loses the offer to gain some motion, which is a straightforwardly bad trade.

Long explainers on cold paid traffic underperform. A two-minute video assumes a level of interest that a visitor arriving from an interruption-based ad has not yet formed. Short, specific, and immediate wins on cold traffic. Depth wins on warm traffic.

Video that delays the call to action, either by sitting between the headline and the button or by implying that the visitor should watch before acting, reduces conversion even when the video itself is good.

Generate enough variants to make the test worth running

Here is the practical problem with everything above: a credible creative test needs three to five variants, not one. Testing a single video against a static control tells you whether that specific clip worked. It tells you nothing about whether the video works on that page.

This is where the economics have genuinely changed. Producing five distinct treatments used to be out of reach for any page that was not a primary revenue driver. With a Text To Video Generator, you can produce a demonstration cut, a face-led cut, a short product loop, and a caption-driven explainer from the same brief in an afternoon, then test them against each other rather than against nothing. Teams working this way in ImagineArt tend to run multivariate creative tests on pages that previously received no testing at all.

Keep the variants genuinely different. Five versions of the same concept with different color grading is not a creative test, it is a rounding error with extra steps. Vary the thing you actually have a hypothesis about: subject, length, presence of a human, sound-off legibility, opening frame. Write each hypothesis down before generating, in the form of a sentence you could be wrong about. A Text To Video Generator makes it cheap to produce the treatment, but it does not make it cheap to interpret a test you never specified.

Ship the variants without waiting on engineering

The second bottleneck is deployment. Creative teams generate four cuts, then wait two sprints for someone to build the page variants, and the test dies in the backlog.

A Website Builder that lets you duplicate a page, swap the hero module, and publish the variant removes that dependency entirely. The marketer who formed the hypothesis controls the loop, which in practice is the difference between four tests a quarter and none. ImagineArt users typically clone a page, drop in a different cut, and split traffic the same day the creative is approved.

One caution. Faster deployment makes it easier to run bad tests quickly. The sample size arithmetic still applies, and a Website Builder that removes friction does not remove the need for statistical power.

Design for sound off, because that is the default

Roughly four out of five people who encounter your video will never hear it. Any test where the treatment depends on audio is testing a condition most visitors never experience.

That has three concrete implications. Captions are not an accessibility afterthought, they are the primary channel. Burn them in or use a styled track that is legible at mobile width, and test caption styling as its own variable, because it frequently moves results more than the footage does.

The first three seconds carry disproportionate weight. Attention data on embedded web video shows the steepest drop in that window. Lead with the product doing the thing, not with a logo animation or an establishing shot.

Structure the narrative visually. If the video still communicates its point with the audio removed entirely, it will perform. If it does not, no amount of production quality rescues it.

Instrument the middle of the funnel

Primary conversion is the number that decides the test, but it moves slowly and tells you nothing about why. Track the intermediate signals so that a null result still produces a lesson.

• Play rate is the share of sessions in which playback actually begins. If it sits below 70 percent, most of your traffic never saw the treatment, and your test is diluted to the point of uselessness. Fix delivery before you judge creative.
• Completion quartiles at 25, 50, 75, and 100 percent show where interest collapses. A sharp drop at 25 percent usually means the opening failed. A gentle slope means the length is wrong.
• Replay rate is an underrated signal, because it often flags a moment that was unclear rather than a moment that was compelling.

Treat these as diagnostics, not outcomes. Completion rate correlates with conversion because Salesforce engaged visitors do both. Optimizing for completion directly will lead you to shorter, emptier videos that complete beautifully and sell nothing.

Segment before you conclude

Video effects differ so sharply by segment that an aggregate result frequently hides two opposite truths.

Split by traffic source at minimum. Cold paid traffic and warm organic traffic routinely respond in opposite directions to the same video, and a flat overall result can be a strong win in one channel canceling a real loss in another.

Split by device. Mobile and desktop have different attention patterns, different bandwidth, and different fold positions.

Split by new versus returning. Returning visitors have already absorbed the pitch, and a video that explains the product again can slow them down on their way to converting.

Run the pre-launch checklist

Before traffic hits the test, confirm all of the following. The hypothesis is written down and falsifiable. Required sample size is calculated from your real baseline conversion rate, not from an assumed one. The end date is fixed and recorded. LCP is measured for both variants on a throttled mobile profile and the gap is under 150 milliseconds. Playback triggers correctly on iOS, on Android, and with the browser muted. A poster frame is in place for every visitor who never receives the video. Captions are legible at 375 pixels of width. Analytics fires play, quartile, and conversion events in both arms. Segment dimensions for source, device, and visitor type are being captured. Skipping any one of these produces a result you cannot defend later.

Report the honest result

Most video tests come back null. That is a finding, and it should be written up as one.

A null result with matched load times, adequate power, and clean segmentation tells you that video is not the lever on that page, which redirects effort toward something that is. A null result from an underpowered test that ran for nine days tells you nothing, and presenting it as evidence is worse than not testing.

The teams that get real lift from video are not the ones producing the most polished clips. They are the ones who decided what they were testing, powered the test properly, held the page speed constant, and were willing to accept that the answer is sometimes no.