Doing nothing for visual regression testing

Varya Stepanova, SC5 Online

Doing nothing for visual regression testing

Brought to you by Varya Stepanova and generated with Jekyller

Bug hunt

Testings

  • performance testing
  • stress testing
  • conformance testing
  • acceptane testing
  • smoke testing
  • regression testing
  • white box testing
  • functional testing
  • system testing
  • black box testing
  • load testing
  • compatibility testing
  • unit testing

Customer context

Does my website make sense for the customer right now?

Spot the difference

Old version

New version

Spot the difference

Visual regression

Perceptual diffs

  1. Take reference screenshots
  2. Provide changes to the local copy
  3. Take local copy screenshots
  4. Compare with image diff

Before and after

Diff

Right way to use

What a little margin can do

Big diff

Impressive & useless

Atomic approach

Visual regression testing not for whole pages but for isolated components gives better understanding of what actually happened.

Button sizes

Wrong margin

Wrong margin

Icons

Missing icon font

Affected colors

Typography before

before

Typography after

after

Typography difference

difference

Typography affection onto components

affection onto components

How to make it work

Developers make a separate page for every single component, fix them every time they change code, write tests for each and make testing infrastructure.

SC5 Style Guide

Informative and easily navigable live style guide which renders every component separately.

Example: varya.me/styleguide

Wonderful for perceptual diffs!

Add on

SC5 Style Guide addition

npm install sc5-styleguide-visualtest

Take reference pics

var visTest = require('sc5-styleguide-visualtest'); gulp.task("test:visual:update", function() {  gulp.src('path/to/styleguide', { read: false })    .pipe(visTest.gather({      // configuration params    }));});

Current state test

gulp.task("visual:test", function(done) {  gulp.src(styleGuidePath, { read: false })    .pipe(visTest.test({      // configuration params    }));});

Console report

> gulp test:visual

Human friendly report

open gemini-report/index.html

Features

  • Out-of-box solution
  • One embedded test suitable for all the components
  • Optional custom tests with interactions and logic

Custom tests

  • Different screen sizes
  • Ignore dynamic parts
  • Wait for element to appear
  • Make interactions (e.g. clicks)
  • Run custom JavaScript

The dev flow

When refactoring

Test particular component

> gulp test:visual --section 7.3.4 --section 8.5

Test all

> gulp test:visual

When making new

Update one component reference

> gulp test:visual:update --section 8.5 --from local

Update all from production

> gulp test:visual:update

Real life stories

  • Missing icon font (sad but true)
  • Changing font linking
  • Typography affection
  • Refactoring of atom components

Apply for

The practises you did not dare to use

  • Continuous integration
  • Continuous deployment
  • Removing code
  • Intensive refactoring

Pictorial pull request

Psychology

Insurance

Devs must be accurate but nothing checks the result.
No, code reviews don’t.

Business

Making money out of it

  • Introduce style guide approach
  • Provide very stable UI
  • Work faster and do more

THANK YOU!

Varya Stepanova, SC5 Online

The tool: github.com/sc5/sc5-styleguide-visualtest

Working code: github.com/varya/varya.github.com

Slides: varya.me/helsinkijs-2015