Doing nothing for visual regression testing

Varya Stepanova

Doing nothing for visual regression testing

Brought to you by Varya Stepanova, From The Front, 15th September 2016

Bug hunt

Testings

Customer context

Does my website make sense for the customer right now?

Spot the difference

Old version

New version

Spot the difference

Visual regression

Perceptual diffs

  1. Take reference screenshots
  2. Provide changes to the local copy
  3. Take local copy screenshots
  4. Compare with image diff

Before and after

Diff

Right way to use

What a little margin can do

Big diff

Impressive & useless

Atomic approach

Visual regression testing not for whole pages but for isolated components gives better understanding of what actually happened.

Button sizes

Alterated margin

Icons

Unlinked icon font

Affected colors

Typography

before

Typography

after

Typography

difference

Typography

affection onto components

How to make it work

Developers make a separate page for every single component, fix them every time they change code, write tests for each and make testing infrastructure.

SC5 Style Guide

Informative and easily navigable live style guide which renders every component separately.

Example: varya.me/styleguide

Wonderful for perceptual diffs!

Add on

SC5 Style Guide addition

npm install sc5-styleguide-visualtest

Take reference pics

var visTest = require('sc5-styleguide-visualtest');

gulp.task("test:visual:update", function() {
  gulp.src('path/to/styleguide', { read: false })
    .pipe(visTest.gather({
      // configuration params
    }));
});

Current state test

gulp.task("visual:test", function(done) {
  gulp.src(styleGuidePath, { read: false })
    .pipe(visTest.test({
      // configuration params
    }));
});

Console report

> gulp test:visual

Human friendly report

open gemini-report/index.html

Features

Custom tests

The dev flow

When refactoring

Test particular component

> gulp test:visual --section 7.3.4 --section 8.5

Test all

> gulp test:visual

When making new

Update one component reference

> gulp test:visual:update --section 8.5 --from local

Update all from production

> gulp test:visual:update

Real life stories

Apply for

The practises you did not dare to use

Pictorial pull request

Psychology

Insurance

Devs must be accurate but nothing checks the result.
No, code reviews don’t.

Business

Making money out of it

Grazie!

Varya Stepanova

The tool: github.com/sc5/sc5-styleguide-visualtest

Working code: github.com/varya/varya.github.com

Slides: varya.me/fromthefront-2016