Progressive Front-end CI on Github Actions. How to integrate Linting and Testing automation into workflows.

original image rights — github.com

“First, solve the problem. Then, write the code” — by John Johnson

GitHub Actions make it easy to automate all your software workflows, now with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make reviews, branch management, and issue triaging work the way you want.

…but this sounds like an intro from GitHub welcome page, isn’t it 😳? So, what we actually need here is pointing out real-world CI pipeline configuration processes for the modern Web.

🌈 That’s what we’re going do today!

Introduction

It’s not a secret or fairy tale that modern front-end is standing not just on HTML + CSS + JS combination, stored & served by back-end. There is much complex architecture for well-quality code delivering to the end-user. Which we should handle day-by-day in fact.

Nowadays, as Front-end engineers we’re responsible for keeping a lot of preprocessors/pipelines/staging scripts along with business codebase 💪. As well as linting/testing/security conventions. This allows us to take focus out from quality monitoring and put all forces on writing the code instead.

💡Note. We don’t discuss any continious deployment (CD) processes here because it’s a different part of CI/CD ecosystem. Our goal is to figure out how to automate annoying routine tasks during coding or staging PR.

Let’s imagine there’s no one with a particular DevOps role on the project. Well, that’s a pity. Sometimes it’s normal for business because even in that case, you’re still able to design some sort of pipelines, via ESLint/TSLint/Prettier/Jest or etc.

While it’s pretty easy to do locally based on documents and JavaScript knowledge, it’s still quite hard to figure out how to deal with external CI workflows. As well as YML-based code language at all.

That’s where we’re digging further.

Basic GitHub CI Configuration

God bless GitHub with their passion for creating templates for everything around: pull requests, issues, funding and etc. So eventually configuring CI/CD doesn’t look much harder.

Here what we have by activating basic NodeJS workflow on GitHub:

# This workflow will do a clean install of node dependencies, build the source code, and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
name: Node.js CIon:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [10.x, 12.x, 14.x]

steps:
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- run: npm ci
- run: npm run build — if-present
- run: npm test

As you can see there isn’t much logic to investigate. Therefore, you can already point out 3 pretty familiar npm commands: ci, build and test.

💡There are no differences in which way manage you packages either npm or yarn. Personally, I prefer to use yarn so the further examples will be on top of it.

We need to understand here that among all the fields like: name, on, jobs, run-on, strategy, steps, build — the only one is valuable for us to level up this basic template. Its name is — steps. 👉 Steps provide us with an ability to add/delete/modify any further scripts(actions) over the whole workflow.

Besides, as far as you see there is no word about how to automate Linting, Testing, or any other staging flows 🔮.

Custom CI Configuration

Once we figured out how to set up GitHub Actions, let’s move forward and create our own CI configuration. 🎉

The very first problem of understanding how CI works is an inability to believe that we could run the same scripts equal to those on our local machines run, like yarn eslint ./someAwesomeFile.ts or npm run jest ./someAwesomeFile.test.js and etc.

The only main difference is to figure out how to run them for staged files over the post-commit stage. Eventually, it works similar to the way like lint-staged and husky composition doon the pre-commit hook. But… this is what we’ll discuss at the end.

Creating Testing CI workflow

So, let’s take a deep dive into CIs! Here we go, a truly workable Testing workflow:

name: Unit + UI Testingon:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
unit-ui-testing:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [12.x]

steps:
- uses: actions/checkout@v2
- name: Staring Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- name: Bootstraping packages
run: yarn install
- name: Testing Shared Utils
if: always()
run: yarn jest ./shared
- name: Testing Storybook UI
if: always()
run: yarn storybook:build

💡Taking in the view that everything above “Starting Node.js” server (including itself) seems familiar to us from GitHub template, we could put our focus on the main testing scripts (described below) only.

The way GitHub Action CI works requires the same pre-starting configuration as we usually have before starting our repo locally.

E.g. bootstrap itself along with all packages required to:

- name: Bootstraping packages
run: yarn install

Setting up Jest + Enzyme testing

Okay, now we’re standing one step away from creating our own Testing configuration. Let’s make some noise already:

- name: Testing Shared Utils
if: always()
run: yarn jest **/*.test.*

👉 You’re wondering what does if: always() flag do? Well, that’s easy to answer. Once you have several independent steps, but at the same time bonded over common abstraction, they should still be able to process even if some of them fall apart.

💡YML if: conditions along with internal GitHub API give us a flexible interface for such a scenarious.

The main run: yarn jest **/__test__/*.test.* command is a way we run testing over the entire repo. Our personal goal here is to rely not on the committed files (even they have or not some own test), but, for their impact, on the global ones. So we test everything around each time.

💡It’s always easy to switch the configuration on the particular files checking only. Up to you!

Setting up Storybook UI testing

Note, you can omit this step if you don’t use Storybook UI testing at all.

- name: Testing Storybook UI
if: always()
run: yarn storybook:build

We personally keep this stuff here just to ensure that our UI shared codebase wasn’t occasionally impacted by some new feature on commit.

So, generally, by getting a successfully inform from run: yarn storybook:build run we hope that compilation is good to go!

Creating Linting CI workflow

Moving further to the Linting CI. That’s where we finally roll up our sleeves and get more fun for sure! It’s time to face real CI configuration over Prettier + ESLint + TSLint + StyleLint pipeline.

So far, so good. The same like with testing workflow, we start with an example:

name: Lintingon:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
linting:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [12.x]
steps:
- uses: actions/checkout@v2
- name: Staring Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- name: Bootstraping packages
run: yarn install
- name: Get file changes
id: get_file_changes
uses: trilom/file-changes-action@v1.2.4
with:
output: ' '
- name: Echo file changes
id: hello
run: |
echo Added files: ${{ steps.get_file_changes.outputs.files_added }}
echo Changed files: ${{ steps.get_file_changes.outputs.files_modified }}
echo Removed files: ${{ steps.get_file_changes.outputs.files_removed }}
- name: Prettier Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn prettier --config ./prettier.config.js --ignore-path ./.prettierignore ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
- name: ESLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn eslint --config ./.eslintrc.js --ignore-path ./.eslintignore ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
- name: TSLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn tslint --config ./tslint.json -e "**/*.(js|jsx|css|scss|html|md|json|yml)" ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
- name: StyleLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn stylelint --config ./.stylelintrc --ignore-path ./.stylelintignore --allow-empty-input ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
- name: Commit changes
if: always()
uses: stefanzweifel/git-auto-commit-action@v4.1.2
with:
commit_message: Apply formatting changes

As usual, by omitting everything above Bootstraping (including itself) section moving further to discuss the rest part.

📌Just to clarify, the main point is to lint staged files only added on the commit. There is no point in linting all the repo every time. Otherwise, it would take us quite a long time to check and fix every particular file in the repo. Nonsens.

Setting Up File Changes

We shall gather somehow all the files from commit for further operations. Hopely for us there is a pretty useful action trilom/file-changes-action which could lift up all the affected(changed) file names.

- name: Get file changes
id: get_file_changes
uses: trilom/file-changes-action@v1.2.4
with:
output: ' '
- name: Echo file changes
id: echo_file_changes
run: |
echo Added files: ${{ steps.get_file_changes.outputs.files_added }}
echo Changed files: ${{ steps.get_file_changes.outputs.files_modified }}
echo Removed files: ${{ steps.get_file_changes.outputs.files_removed }}

In its turn, along with setting up echo these files on the next step, we could create a reachable string consisted of file paths. Eventually, create an ability to throw further all the files into eslint and other pipelines!

💡So generally Get file changes and Echo file changes bound is our strategic point for all further checking steps.

Setting up Prettier checking

All right! Once we got how to gather files that are exactly needed to process. Now it’s time to put them through the first lining pipeline.

And #1 step would be Prettier optimization:

- name: Prettier Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn prettier --config ./prettier.config.js --ignore-path ./.prettierignore ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix

Heh, that looks a bit scary code 😨, isn’t it? Besides on closer look, it appears that we have only if statement and run script here. Not too much to become scary 😅 in fact.

Let’s start from if: then. It seems a bit complicated in comparison with those equal inside Testing CI workflow. There are no surprises, here we need to check particular files only, while in the testing workflow checking is going over the whole codebase.

So if: condition indicated to start checking only if there are some fresh new or modified files present. In the case of deleted ones nothing to check for sure. By the way, it should start even if some related steps will fall (the same as with Jest/Storybook steps).

As for run: command, it’s quite the same as the command you run locally (based on Prettier CLI schema).

Setting up ESLint checking

The same as in a previous step. Just its own interface to play with:

- name: ESLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn eslint --config ./.eslintrc.js --ignore-path ./.eslintignore ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix

Setting up TSLint checking

One more 😊. The same as in the previous one. Just its own interface to play with:

- name: TSLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn tslint --config ./tslint.json -e "**/*.(js|jsx|css|scss|html|md|json|yml)" ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix

The only difference here is that you can’t use global .tslintignore file, because it doesn’t invent by TSLint creators yet. We literally need to put all the excludes rules among CLI arguments.

Setting up StyleLint checking

Everything looks the same in comparison with ESLint and Prettier steps. Just its own run: interface to play with:

- name: StyleLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn stylelint --config ./.stylelintrc --ignore-path ./.stylelintignore --allow-empty-input ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix

Setting up Commit changes

🔥 Phew, we did it! We’ve done with the whole pipelines so far! Consisted of 4 independent checks. Great job!

Now, it’s time to move forward and investigate ways to save processed changes (files). Hopely for us there is quite a useful package called stefanzweifel/git-auto-commit-action:

- name: Commit changes
if: always()
uses: stefanzweifel/git-auto-commit-action@v4.1.2
with:
commit_message: Apply formatting changes

It aims to automate the commit process for CI-staged files. So generally it saves all the pipeline changes at once.

And that it! The second workflow file has been created and described. Now you have all the power in your hands to automate and enhance Front-end codebase in the way you like.

Bonuses — Yarn Caching and Slack Notifications

Just in case you’re looking further, for some more optimization/enhance features I would recommend integrating a few more actions.

Enable Yarn caching

As well as locally, it took some time to collect and install all the packages of your repo. There is a way to reduce the bootstrapping speed from 15 to 1 minute.

Of course, we could make caching for every installed package on the CI side. To do the trick simply replace the Bootstrapping step previously added with actions/cache@v2 action before.

- name: Restoring Yarn cache
uses: actions/cache@v2
with:
path: '**/node_modules'
key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}
- name: Bootstraping packages
if: steps.yarn-cache.outputs.cache-hit != 'true'
run: yarn install

Enable Slack Notification

In case you use Slack as a source for communication at the project, it’s essential to have notifications over the ongoing processes of your CI/CD.

Thanks to 8398a7/action-slack it’s easy to manage which sort of notification messages and their texts you need to have. Just put this step right after everything above in your CI workflows.

- name: Slack Notification
uses: 8398a7/action-slack@v3.8.0
if: failure()
with:
status: custom
fields: workflow,job,commit,repo,ref,author,took
custom_payload: |
{
username: 'Awesome-CI',
icon_emoji: ':react:',
author_name: 'Linting Test',
attachments: [{
color: '${{ job.status }}' === 'success' ? 'good' : '${{ job.status }}' === 'failure' ? 'danger' : 'warning',
text: `CI Task: ${process.env.AS_WORKFLOW}\ncommit: (${process.env.AS_COMMIT}) ${{ github.event_name }} ${{ job.status }}. Initiated by ${process.env.AS_AUTHOR} in ${process.env.AS_TOOK}`,
}]
}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
MATRIX_CONTEXT: ${{ toJson(matrix) }}

Must say that to be able to use Slack CI Notification you should create a Webhook secret and set up its data right into Slack preferences.

The full (final) Testing and Linting CI workflows could be founded here:

Linting final real-world workflow

Testing final real-world workflow

Wrapping Up

Today we’ve created real-world CI workflows that could handle a lot of routine stuff without spending personal time.

So far we got that it’s easy to manage Linting, Testing, and any other pipelines of your application simultaneously.

By blessing modern GitHub CI you almost don’t need any 3rd-party software/hooks/actions. Try to use your scripts, which are in sync with package.json only. It appears that at some point we could live free without any custom committing actions further.

Check out my previous articles:

Senior Front-end Engineer | React enthusiast | Open Source Contributor | Love traveling and snowboarding.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store