Understanding Lost Pixel: An Open-Source Visual Regression Testing Tool

Visual regression testing is crucial for maintaining the visual integrity of web applications. It ensures that new changes do not introduce unexpected visual defects. Lost Pixel is an innovative, open-source visual regression testing tool that stands as a robust alternative to commercial solutions like Percy and Chromatic. This technical blog post aims to provide an in-depth understanding of Lost Pixel, its features, integrations, setup processes, and the value it brings to development workflows.

What is Lost Pixel?

Lost Pixel is an open-source tool designed for visual regression testing. It allows developers to run visual tests on Storybook and Ladle stories, as well as application pages, ensuring that visual changes in the codebase do not introduce regressions. Lost Pixel consists of two primary components:

1. lost-pixel: The core engine responsible for executing visual regression tests. This component can be used standalone and is highly customizable.
2. Lost Pixel Platform: A user interface and CI/CD helpers that provide a seamless experience for managing visual regression tests. This includes a specialized regression UI, team collaboration features, and an easy approval/rejection process for screenshots.

Key Features of Lost Pixel

Lost Pixel offers a rich set of features that cater to modern development needs:

• First-Class GitHub Integration: Setting up Lost Pixel with GitHub is straightforward. It provides automatic GitHub status checks and a smooth GitHub Actions experience.
• Power of Composition: Lost Pixel can leverage existing tools like Storybook, Next.js, and Playwright to create comprehensive visual tests, offering a holistic view of UI components and application pages.
• Streamlined Review Flow: The platform provides a user-friendly UI for reviewing, approving, and rejecting visual changes, making it suitable for projects of various sizes and stacks.
• Mask Elements: Developers can mask unnecessary elements to exclude them from screenshot comparisons, reducing noise in test results.
• Execute Browser Code: Lost Pixel allows adjustments to page CSS and HTML before capturing screenshots, providing control over the testing environment.
• Set Thresholds: Fine-grained control over screenshot comparison thresholds is available, both at global and per-screenshot levels.
• Multiple Browsers and Breakpoints: Lost Pixel supports testing across multiple browsers and device widths, ensuring comprehensive coverage.
• Parallel Execution: Tests can be run in parallel to speed up the test suite, crucial for large projects.
• Approval Flow: A robust approval system for visual changes, with the ability to comment on screenshots.
• Fight Flakiness: Features like flaky test retries and custom wait times help mitigate test flakiness.

Getting Started with Lost Pixel

Setting up Lost Pixel is designed to be simple and quick, especially with its tight integration with GitHub. Here are some examples of how to configure Lost Pixel for different setups:

Example: Storybook Setup

1. Create a lostpixel.config.ts file at the root of your project:

    import { CustomProjectConfig } from 'lost-pixel';
   
    export const config: CustomProjectConfig = {
      storybookShots: {
        storybookUrl: './storybook-static',
      },
      generateOnly: true,
      failOnDifference: true,
   
      // Uncomment the following for Lost Pixel Platform
      // lostPixelProjectId: "xxxx",
      // process.env.LOST_PIXEL_API_KEY,
    };
   

2. Add a GitHub action configuration in .github/workflows/lost-pixel-run.yml:

    on: [push]
   
    jobs:
      build:
        runs-on: ubuntu-latest
   
        steps:
          - name: Checkout
            uses: actions/checkout@v3
   
          - name: Setup Node
            uses: actions/setup-node@v3
            with:
              node-version: 18.x
              cache: 'npm'
   
          - name: Install dependencies
            run: npm ci
   
          - name: Build Storybook
            run: npm run build-storybook
   
          - name: Lost Pixel
            uses: lost-pixel/lost-pixel@v3.17.0
   

Example: Ladle Setup

1. Create a lostpixel.config.ts file:

    import { CustomProjectConfig } from 'lost-pixel';
   
    export const config: CustomProjectConfig = {
      ladleShots: {
        ladleUrl: 'http://172.17.0.1:61000',
      },
      generateOnly: true,
      failOnDifference: true,
   
      // Uncomment the following for Lost Pixel Platform
      // lostPixelProjectId: "xxxx",
      // process.env.LOST_PIXEL_API_KEY,
    };
   

2. Add scripts to package.json:

    "scripts": {
      "serve": "npx serve build -p 61000",
      "build": "ladle build"
    },
   

3. Add a GitHub action configuration:

    on: [push]
   
    jobs:
      build:
        runs-on: ubuntu-latest
   
        steps:
          - name: Checkout
            uses: actions/checkout@v3
   
          - name: Setup Node
            uses: actions/setup-node@v3
            with:
              node-version: 18.x
              cache: 'npm'
   
          - name: Install dependencies
            run: npm install
   
          - name: Build Ladle
            run: npm run build
   
          - name: Serve Ladle
            run: npm run serve &
   
          - name: Lost Pixel
            uses: lost-pixel/lost-pixel@v3.17.0
   

Lost Pixel for Various Use-Cases

Lost Pixel is versatile and supports a wide range of use cases:

1. Design Systems: Test UI components built with Storybook or Ladle to ensure design consistency across the application.
2. Marketing Pages: Verify that marketing pages render correctly across different browsers and devices.
3. Application UI: Test dynamic pages and flows within modern frontend frameworks like Next.js, Gatsby, and Remix.
4. Custom Integrations: Use custom screenshots from Cypress or Playwright for advanced testing scenarios.

Benefits of Using Lost Pixel

Open-Source Flexibility

Lost Pixel is an open-source project, providing transparency and flexibility. Developers can customize and extend its functionality to fit their specific needs. The open-source nature also means there is a community around it, contributing to its continuous improvement.

Cost-Effective Solution

Compared to proprietary visual regression testing tools, Lost Pixel offers a cost-effective alternative without compromising on features. The pricing is designed to be accessible, ensuring that teams can benefit from visual regression testing without worrying about exceeding budgets.

Seamless GitHub Integration

Lost Pixel's integration with GitHub streamlines the CI/CD workflow. Developers can set up visual regression tests within minutes, and the automatic status checks help catch visual bugs before they reach production.

Comprehensive Testing Capabilities

With support for multiple browsers, device breakpoints, and custom configurations, Lost Pixel ensures comprehensive testing coverage. This helps in catching visual regressions across various environments, reducing the risk of visual bugs slipping through.

Real-World Testimonials

Lost Pixel is trusted by engineering teams at several companies for its reliability and ease of use. Here are some testimonials from satisfied users:

• Prisma: "Lost Pixel is an amazing tool for visual regression testing. We use it to make sure our Prisma Documentation looks good, and that changes to design or content do not introduce any regressions or problems. Lost Pixel is easy to set up, use, and the team is continuously improving based on our feedback. It’s been a great addition to our workflow!"
• Design System Team: "Visual regression testing has become very important for our teams - especially the design-system team - to move fast with confidence, and Lost Pixel is the best tool for the job: fast, reliable, and affordable. 10/10."

Support and Community

Lost Pixel offers robust support options:

• Ask Questions: Developers can ask questions and initiate discussions on the GitHub Discussions page.
• Bug Reports: Report issues directly on the GitHub Issues page to help improve the tool.
• Feature Requests: Submit feature requests to suggest enhancements or new features.
• Community Engagement: Join the Lost Pixel community on Discord to chat with like-minded individuals and get faster support.

Pricing

Lost Pixel offers competitive pricing that makes sense for development teams. It ensures that you can ship your projects confidently without worrying about exceeding monthly quotas. The open-source nature of Lost Pixel also allows for free usage in non-commercial open-source projects, providing additional flexibility.

Conclusion

Lost Pixel is a powerful, open-source visual regression testing tool that provides a comprehensive solution for maintaining the visual integrity of web applications. With its robust feature set, seamless GitHub integration, and flexible pricing, Lost Pixel stands out as a valuable tool for developers looking to ensure their UI components and application pages remain visually consistent across changes. By integrating Lost Pixel into your development workflow, you can ship frontend changes with confidence, knowing that visual regressions will be caught before reaching your users.