Implementing Testing Strategies

1. Running Unit Tests

Option	Values	Description
script	`npm test`	Command to run unit tests
artifacts	`reports/junit.xml`	Path to JUnit XML report for test results

Example: Unit Test Configuration

unit_test:
  stage: test
  script:
    - npm install
    - npm test
  artifacts:
    reports:
      junit: reports/junit.xml

2. Running Integration Tests

Setting	Description	Value
script	Command to run integration tests	`npm run test:integration`
artifacts	Artifacts to store after test run	`test-reports/`
services	Services to use during test execution	`postgres:13`

Example: Integration Test Job Configuration

integration_test:
  stage: test
  script:
    - npm install
    - npm run test:integration
  artifacts:
    reports:
      junit: test-reports/*.xml
  services:
    - postgres:13

Note: Ensure database services are properly configured to avoid test flakiness.

Warning: Integration tests may take longer to run than unit tests.

3. Running End-to-End Tests

Option	Values	Notes
script	`npm run e2e`	Command to execute end-to-end tests
artifacts	`reports/e2e`	Directory to store test reports
coverage	`coverage/e2e`	Directory to store coverage reports

Example: End-to-End Test Configuration

stages:
  - test

e2e_test:
  stage: test
  image: node:18
  script:
    - npm install
    - npm run e2e
  artifacts:
    reports:
      junit: reports/e2e/junit.xml
    paths:
      - reports/e2e/
  coverage: coverage/e2e/coverage.xml

Note: Ensure that your E2E tests are configured to run in a headless browser environment for CI.

4. Using Test Fixtures

Fixture Type	Description	Usage Example	Notes
Test Data	Predefined data used to test specific scenarios in a controlled environment.	`test_data: user: name: "John Doe" email: "john@example.com"`	Ensures consistent and repeatable test results
Mock Services	Simulated services or APIs to isolate the system under test from external dependencies.	`mock_service: url: "http://mock-api.example.com" responses: - status: 200 body: '{"status": "ok"}'`	Useful for testing without real backend services
Database Snapshots	Pre-configured database states used to simulate real-world data conditions.	`database_snapshot: name: "users_and_orders" file: "snapshots/users_orders.sql"`	Improves test performance and reliability

Example: Setting Up a Mock Service in GitLab CI

services:
  - name: mock-api:latest
    alias: api-mock

before_script:
  - echo "Starting mock service..."
  
test_job:
  script:
    - curl http://api-mock:8080/users

Note: Fixtures should be lightweight and not affect test execution times significantly.

Warning: Avoid using live external services in tests as they can introduce flakiness.

5. Parallel Test Execution

Example: Running Tests in Parallel Jobs

test:
  stage: test
  script:
    - npm run test:unit
  parallel:
    matrix:
      - TEST_SUITE: unit
      - TEST_SUITE: integration

Option	Values	Description
parallel	`matrix`	Enables parallel execution using matrix strategy
parallel.matrix	`[{TEST_SUITE: unit}, {TEST_SUITE: integration}]`	Defines the matrix of jobs to run in parallel

Note: Parallel jobs run simultaneously, improving test execution time.

Warning: Ensure tests are isolated to prevent interference between parallel jobs.

Best Practices for Parallel Test Execution

Use parallel.matrix to define job variations
Ensure test isolation to avoid flaky results
Monitor resource usage to prevent overloading the runner

6. Configuring Test Coverage

Option	Values	Notes
coverage_format	`json`, `xml`, `lcov`	Format of the coverage report
coverage_path	`coverage.xml`, `coverage/lcov.info`	Path to the coverage report file
coverage_pattern	`**/coverage.xml`	Glob pattern to match coverage files

Example: Configuring test coverage in GitLab CI

test:
  script:
    - npm test
  coverage: '/Coverage: \d+.\d+%/i'

Note: Coverage reporting requires a compatible test runner that supports coverage output in one of the supported formats.

Format	Description	Supported by
`json`	JSON formatted coverage report	Most modern test runners
`xml`	XML formatted coverage report (e.g., Cobertura)	Jest, Mocha
`lcov`	LCov format for line coverage	Jest, Istanbul

7. Setting Coverage Regex

Property	Values	Description
coverage.regex	`string`	Regular expression to match coverage reports
coverage.format	`string`	Format of the coverage report (e.g., `lcov`, `cobertura`)

Example: Configuring Coverage Regex

coverage:
  regex: ".*\.cov"
  format: lcov

Note: The regex must match the path to your coverage report file(s).

8. Splitting Tests by Timing

Test Type	Description	Timing	Best Practices
Unit Tests	Test individual functions or components in isolation	Fast execution, typically run on every commit	Use mocking to isolate dependencies; run in parallel for speed
Integration Tests	Test interactions between modules or services	Medium execution time, run after unit tests	Ensure proper data flow and communication between components
End-to-End (E2E) Tests	Test the full application flow from user input to output	Slower execution, run less frequently (e.g., on merge)	Use real or simulated environments; focus on user workflows
Performance Tests	Measure application performance under load or stress	Longer execution, typically run in dedicated environments	Use tools like JMeter or Gatling; define thresholds for pass/fail
Security Tests	Verify application security vulnerabilities and compliance	Varies by tool, often run periodically or on major changes	Use automated tools like OWASP ZAP or SonarQube; integrate into CI

Example: Splitting Test Jobs in GitLab CI

unit_tests:
  stage: test
  script:
    - npm run test:unit
  artifacts:
    reports:
      junit: junit.xml

integration_tests:
  stage: test
  script:
    - npm run test:integration
  artifacts:
    reports:
      junit: junit.xml

e2e_tests:
  stage: test
  script:
    - npm run test:e2e
  artifacts:
    reports:
      junit: junit.xml

Timing-Based Test Strategy

Unit Tests: Fast, frequent execution; ideal for continuous feedback
Integration Tests: Moderate runtime; ensure component interaction
E2E Tests: Slower, less frequent; simulate real user behavior
Performance Tests: Long-running, dedicated stages; measure scalability
Security Tests: Periodic or triggered on changes; ensure compliance

9. Implementing Test Retries

Option	Values	Notes
retry	`0`, `1`, `2`, `3`	Number of retry attempts for a job. Default is `0`.
retry.max	`1`, `2`, `3`	Maximum number of retry attempts allowed. Default is `1`.
retry.when	`runner_system_failure`, `stuck_or_canceled_job`, `job_execution_timeout`	Conditions under which a job will be retried. Can be a list of conditions.

Example: Retry configuration in GitLab CI

test_job:
  script:
    - npm test
  retry:
    max: 2
    when:
      - runner_system_failure
      - stuck_or_canceled_job

Note: Retries are not supported for jobs that have artifacts or cache defined.

Warning: Excessive retries can lead to increased resource usage and longer pipeline times.

10. Generating Test Reports

Property	Description	Values	Notes
artifacts	Defines artifacts to be stored after job completion	`reports`	Used to store test reports in GitLab
reports.junit	Path to JUnit XML report file	`reports/junit.xml`	Used for test results in JUnit format
reports.cobertura	Path to Cobertura XML report file	`reports/cobertura.xml`	Used for code coverage reports

Example: Configuring JUnit Test Reports

test:
  script:
    - mvn test
  artifacts:
    reports:
      junit: "target/surefire-reports/TEST-*.xml"

Example: Configuring Cobertura Code Coverage Reports

coverage:
  script:
    - mvn cobertura:cobertura
  artifacts:
    reports:
      cobertura: "target/site/cobertura/coverage.xml"

Note: GitLab CI/CD supports JUnit and Cobertura formats for test reports. Ensure the paths match your project's output directory.