nas-tools/TESTING.md

Testing Guide

This document describes how to run and write tests for the NAS Dashboard project.

Overview

The project has comprehensive test coverage for both backend (Python/FastAPI) and frontend (Svelte 5):

• Backend: pytest with unit and integration tests
• Frontend: Vitest with unit and component tests
• CI: Automated testing via Gitea Actions

Backend Testing

Setup

cd dashboard/backend
pip install -r requirements.txt
pip install -r requirements-dev.txt

Running Tests

# Run all tests
pytest

# Run with coverage
pytest --cov=. --cov-report=term --cov-report=html

# Run only unit tests
pytest tests/unit/ -v

# Run only integration tests
pytest tests/integration/ -v

# Run specific test file
pytest tests/unit/test_auth.py -v

# Run specific test
pytest tests/unit/test_auth.py::TestPasswordHashing::test_hash_and_verify_password -v

Test Structure

dashboard/backend/tests/
├── conftest.py              # Shared fixtures
├── unit/
│   ├── test_auth.py         # JWT, TOTP, password hashing
│   ├── test_rbac.py         # Role-based access control
│   └── test_config.py       # IP parsing, env validation
└── integration/
    ├── test_auth_flow.py    # Login, refresh, logout flows
    ├── test_docker.py       # Container operations
    └── test_files.py        # File operations

Key Fixtures

• mock_config: Mocked configuration with test values
• temp_auth_file: Temporary auth.json for testing
• temp_rbac_file: Temporary rbac.json for testing
• valid_access_token: Valid JWT access token
• admin_headers: Headers with admin authentication
• mock_docker_client: Mocked Docker SDK client

Writing Tests

Example unit test:

def test_hash_and_verify_password(self):
    """Test that password hashing and verification works."""
    password = "test_password_123"
    hashed = get_password_hash(password)

    assert hashed != password
    assert verify_password(password, hashed)

Example integration test:

def test_login_success_no_2fa(self, test_app, mock_config, temp_auth_file):
    """Test successful login without 2FA."""
    password = "test_password"
    hashed = get_password_hash(password)
    save_password_hash(hashed)

    response = test_app.post(
        "/api/auth/login",
        json={"username": "testadmin", "password": password, "totp_code": ""}
    )

    assert response.status_code == 200
    assert response.json()["user"] == "testadmin"

Frontend Testing

Setup

cd dashboard/frontend
npm install

Running Tests

# Run all tests
npm test

# Run with coverage
npm run test:coverage

# Run in watch mode
npm test -- --watch

# Run specific test file
npm test -- tests/unit/api.test.js

# Run with UI
npm run test:ui

Test Structure

dashboard/frontend/tests/
├── setup.js                 # Global test setup
├── unit/
│   └── api.test.js          # API client tests
├── components/
│   └── Login.test.js        # Login component tests
└── integration/
    └── (future tests)

Mocked APIs

The test setup (tests/setup.js) provides mocks for:

• fetch - HTTP requests
• WebSocket - WebSocket connections
• SpeechRecognition / speechSynthesis - Web Speech API
• navigator.credentials - WebAuthn
• localStorage - Local storage
• ResizeObserver - Resize observer
• MutationObserver - Mutation observer

Writing Tests

Example unit test:

it('should set and get token', () => {
  setToken('test-token-123');
  expect(getToken()).toBe('test-token-123');
});

Example component test:

it('should render login form', () => {
  render(Login);

  expect(screen.getByLabelText(/username/i)).toBeTruthy();
  expect(screen.getByLabelText(/password/i)).toBeTruthy();
});

Coverage Goals

• Backend: 80%+ overall, 95%+ for critical paths (auth, RBAC)
• Frontend: 70%+ overall, 80%+ for critical components

CI Integration

Tests run automatically on:

• Push to main branch
• Pull requests to main

The CI workflow (.gitea/workflows/test.yml) runs:

1. Backend unit tests
2. Backend integration tests
3. Frontend tests with coverage
4. Uploads coverage reports as artifacts

Viewing CI Results

1. Go to the Gitea repository
2. Click on "Actions" tab
3. Select the workflow run
4. Download coverage artifacts

Common Issues

Backend Tests

Issue: ModuleNotFoundError: No module named 'fastapi' Solution: Install dependencies: pip install -r requirements.txt -r requirements-dev.txt

Issue: RuntimeError: SECRET_KEY must be set Solution: Tests use mocked config, ensure conftest.py is loaded

Issue: Permission denied on temp files Solution: Check that /tmp is writable

Frontend Tests

Issue: Cannot find module '@testing-library/svelte' Solution: Install dependencies: npm install

Issue: ReferenceError: WebSocket is not defined Solution: Ensure tests/setup.js is loaded (check vitest.config.js)

Issue: Component tests fail with Svelte 5 runes Solution: Use @testing-library/svelte v5.2.3+ which supports Svelte 5

Best Practices

Backend

1. Use fixtures: Leverage shared fixtures from conftest.py
2. Mock external services: Always mock Docker, SSH, HTTP clients
3. Test edge cases: Invalid inputs, expired tokens, permission errors
4. Atomic tests: Each test should be independent
5. Clear test names: Use descriptive names that explain what's being tested

Frontend

1. Test behavior, not implementation: Focus on user interactions
2. Mock API calls: Use vi.fn() to mock fetch responses
3. Test accessibility: Use getByRole, getByLabelText from testing-library
4. Avoid testing internal state: Test what the user sees
5. Keep tests simple: One assertion per test when possible

Debugging Tests

Backend

# Run with verbose output
pytest -vv

# Run with print statements visible
pytest -s

# Run with debugger on failure
pytest --pdb

# Run specific test with full output
pytest tests/unit/test_auth.py::TestPasswordHashing -vv -s

Frontend

# Run with verbose output
npm test -- --reporter=verbose

# Run in UI mode for debugging
npm run test:ui

# Run single test file in watch mode
npm test -- tests/unit/api.test.js --watch

Adding New Tests

Backend

1. Create test file in appropriate directory (tests/unit/ or tests/integration/)
2. Import necessary fixtures from conftest.py
3. Write test classes and methods
4. Run tests to verify

Frontend

1. Create test file in appropriate directory (tests/unit/, tests/components/, or tests/integration/)
2. Import necessary mocks and utilities
3. Write test cases using describe and it
4. Run tests to verify

Resources

• pytest documentation
• Vitest documentation
• Testing Library documentation
• FastAPI testing guide
• Svelte testing guide