GitHub Actions CI config for MySQL, Redis, Elasticsearch in the Ruby on Rails project with RSpec tests

0


Knapsack Pro Hacker Noon's profile picture

@backpackBackpack Pro

Speed ​​up your testing. Run 1 hour of test suite in 2 minutes with optimal parallelization on the CI server.

You will learn how to configure the Ruby on Rails project on GitHub actions. This specific Rails project has a MySQL and Redis database. There is also an Elasticsearch service running on CI. If your project is close to this configuration below, the yaml configuration of GitHub actions will allow you to run tests on the GitHub CI server.

If you’re using PostgreSQL, you can check out our previous article on configuring the Rails app with Postgres on GitHub Actions.

Configuring GitHub Actions for Rails

In your repository, you need to create a file

.github/workflows/main.yaml

. Thanks to this, GitHub Actions will run your CI builds. You can find the results of executed CI builds in the Actions tab of your GitHub repository page.

In that case, Rails the application has MySQL, Redis, and Elasticsearch data base. You must configure services with a Docker container to run each of them. In the setup below, there is also a step to check the health of MySQL and Elasticsearch to make sure that both are up and running before you can start running tests.

Tests are run on parallel tasks using the matrix feature in GitHub Actions and the Knapsack Pro ruby ​​gem which will automatically balance the distribution of tests between tasks. Self-balancing tests using Knapsack Pro’s queue mode will ensure that each parallel job completes at a similar time. Thanks to this, there is no bottleneck (no slow job with too many tests to run) and you can enjoy fast IC build time as you get optimal tests spread over tasks parallels.

# .github/workflows/main.yaml
name: Main

on: [push]

jobs:
  vm-job:
    name: CI
    runs-on: ubuntu-latest

    # If you need DB like MySQL then define service below.
    # Example for PostgreSQL and Redis can be found here:
    # https://github.com/actions/example-services/tree/master/.github/workflows
    services:
      # How to use MySQL
      mysql:
        image: mysql:5.7
        env:
          MYSQL_ROOT_PASSWORD: root
        ports:
        - 3306
        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
      # How to use Redis
      redis:
        image: redis
        ports:
        - 6379/tcp
      # How to use Elasticsearch
      elasticsearch:
        image: elasticsearch:6.8.3
        ports:
        - 9200/tcp
        options: -e="discovery.type=single-node" --health-cmd="curl http://localhost:9200/_cluster/health" --health-interval=10s --health-timeout=5s --health-retries=10

    # https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix
    strategy:
      fail-fast: false
      matrix:
        # Set N number of parallel jobs you want to run tests on.
        # Use higher number if you have slow tests to split them on more parallel jobs.
        # Remember to update ci_node_index below to 0..N-1
        ci_node_total: [5]
        # set N-1 indexes for parallel jobs
        # When you run 2 parallel jobs then first job will have index 0, the second job will have index 1 etc
        ci_node_index: [0, 1, 2, 3, 4]

    steps:
    - name: Checkout
      uses: actions/[email protected]
      with:
        fetch-depth: 1
        
    - name: Set up Ruby 2.6
      uses: actions/[email protected]
      with:
        ruby-version: 2.6.5
        
    - uses: actions/[email protected]
      with:
        path: vendor/bundle
        key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
        restore-keys: |
          ${{ runner.os }}-gems-
    - name: Verify MySQL connection from host
      run: |
        sudo apt-get install -y mysql-client libmysqlclient-dev
        mysql --host 127.0.0.1 --port ${{ job.services.mysql.ports[3306] }} -uroot -proot -e "SHOW GRANTS FOR 'root'@'localhost'"
        mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql --host 127.0.0.1 --port ${{ job.services.mysql.ports[3306] }} -uroot -proot mysql
    - name: Verify Elasticsearch connection from host
      env:
        ELASTIC_SEARCH_URL: http://localhost:${{ job.services.elasticsearch.ports[9200] }}
      run: |
        echo $ELASTIC_SEARCH_URL
        curl -fsSL "$ELASTIC_SEARCH_URL/_cat/health?h=status"
    - name: Bundle Install and Create DB
      env:
        RAILS_ENV: test
        DB_PASSWORD: root
        # tell Rails to use proper port for MySQL
        DB_PORT: ${{ job.services.mysql.ports[3306] }}
      run: |
        cp .env.sample .env
        cp config/database.yml.ci config/database.yml
        gem install bundler --version 1.17.3 --no-ri --no-rdoc
        bundle config path vendor/bundle
        bundle install --jobs 4 --retry 3 --path vendor/bundle
        # create DB for Rails 5.x
        bin/rails db:setup
        # create DB for Rails 6.x
        bin/rails db:prepare
    - name: Run tests
      env:
        DB_PASSWORD: root
        DB_PORT: ${{ job.services.mysql.ports[3306] }} # get randomly assigned published port
        REDIS_PORT: ${{ job.services.redis.ports[6379] }}
        ELASTIC_SEARCH_URL: http://localhost:${{ job.services.elasticsearch.ports[9200] }}
        RAILS_ENV: test
        # You need to set API tokens in GitHub repository Settings -> Secrets
        KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC }}
        KNAPSACK_PRO_TEST_SUITE_TOKEN_CUCUMBER: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_CUCUMBER }}
        KNAPSACK_PRO_TEST_SUITE_TOKEN_MINITEST: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_MINITEST }}
        KNAPSACK_PRO_TEST_SUITE_TEST_UNIT: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_TEST_UNIT }}
        KNAPSACK_PRO_TEST_SUITE_TOKEN_SPINACH: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_SPINACH }}
        KNAPSACK_PRO_CI_NODE_TOTAL: ${{ matrix.ci_node_total }}
        KNAPSACK_PRO_CI_NODE_INDEX: ${{ matrix.ci_node_index }}
        # if you use Knapsack Pro Queue Mode you must set below env variable
        # to be able to retry CI build and run previously recorded tests
        KNAPSACK_PRO_FIXED_QUEUE_SPLIT: true
      run: |
        # run tests in Knapsack Pro Regular Mode
        bundle exec rake knapsack_pro:rspec
        bundle exec rake knapsack_pro:cucumber
        bundle exec rake knapsack_pro:minitest
        bundle exec rake knapsack_pro:test_unit
        bundle exec rake knapsack_pro:spinach
        # you can use Knapsack Pro in Queue Mode once recorded first CI build with Regular Mode
        bundle exec rake knapsack_pro:queue:rspec
        bundle exec rake knapsack_pro:queue:cucumber
        bundle exec rake knapsack_pro:queue:minitest

How Knapsack Pro Queue Mode works

In this video, you will learn how the dynamic test suite spread over parallel jobs (parallel CI nodes) works.

How Knapsack Pro Regular Mode works

Knapsack Pro also has a deterministic way to split tests. Tests are only split once before running the tests. This is the easiest way you will first try to save your CI build before entering queue mode.

You can read more about Knapsack Pro in the installation guide.

summary

If you want to learn more about GitHub actions, check out our previous article on testing the Rails app with PostgreSQL on GitHub actions. There is also a video showing him in action.

Learn how to break down slow RSpec spec files by test examples on Github actions.

And if you’re still planning to use GitHub actions, our comparison of GitHub actions against other CIs may be of use to you. The most frequently viewed comparisons are: Github vs Circle CI Actions, Github vs Jenkins Actions, and Github vs Travis CI Actions.

Previously posted at https://docs.knapsackpro.com/2019/github-actions-ci-config-for-ruby-on-rails-project-with-mysql-redis-elasticsearch-how-to-run-parallel-tests

Knapsack Pro Hacker Noon's profile picture

Keywords

Join Hacker Noon

Create your free account to unlock your personalized reading experience.



Source link

Leave A Reply

Your email address will not be published.