johnpccd/image_management_api
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

cp

Browse Source
This commit is contained in:
johnpccd 2025-05-24 13:57:58 +02:00
parent 59fcf08696
commit 485eb5ccec
7 changed files with 1396 additions and 727 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

97
README.md
View File

@ -11,6 +11,7 @@ SEREACT is a secure API for storing, organizing, and retrieving images with adva
- Metadata extraction and storage - Metadata extraction and storage
- Image processing capabilities - Image processing capabilities
- Multi-team support - Multi-team support
- **Comprehensive E2E testing with real database support**
## Architecture ## Architecture
@ -21,7 +22,8 @@ sereact/
│ ├── cloud-run/ # Google Cloud Run configuration │ ├── cloud-run/ # Google Cloud Run configuration
│ └── terraform/ # Infrastructure as code │ └── terraform/ # Infrastructure as code
├── docs/ # Documentation ├── docs/ # Documentation
│ └── api/ # API documentation │ ├── api/ # API documentation
│ └── TESTING.md # Comprehensive testing guide
├── scripts/ # Utility scripts ├── scripts/ # Utility scripts
├── src/ # Source code ├── src/ # Source code
│ ├── api/ # API endpoints and routers │ ├── api/ # API endpoints and routers
@ -42,7 +44,7 @@ sereact/
│ ├── models/ # Model tests │ ├── models/ # Model tests
│ ├── services/ # Service tests │ ├── services/ # Service tests
│ ├── integration/ # Integration tests │ ├── integration/ # Integration tests
│ └── test_e2e.py # End-to-end workflow tests │ └── test_e2e.py # **Comprehensive E2E workflow tests**
├── main.py # Application entry point ├── main.py # Application entry point
├── requirements.txt # Python dependencies ├── requirements.txt # Python dependencies
└── README.md # This file └── README.md # This file
@ -183,12 +185,12 @@ Refer to the Swagger UI documentation at `/docs` for detailed endpoint informati
pytest pytest
``` ```
### End-to-End Testing ### **Comprehensive End-to-End Testing**
SEREACT includes comprehensive end-to-end tests that cover complete user workflows: SEREACT includes a comprehensive E2E testing suite that covers complete user workflows with **completely self-contained artificial test data**:
```bash ```bash
# Run all E2E tests with mocked services (recommended for development) # Run all E2E tests (completely self-contained - no setup required!)
python scripts/run_tests.py e2e python scripts/run_tests.py e2e
# Run unit tests only (fast) # Run unit tests only (fast)
@ -204,14 +206,83 @@ python scripts/run_tests.py all
python scripts/run_tests.py coverage python scripts/run_tests.py coverage
``` ```
The E2E tests cover: #### **E2E Test Coverage**
- **Team Management**: Create, update, and manage teams
- **User Management**: User creation, roles, and permissions Our comprehensive E2E tests cover:
- **API Authentication**: API key generation and validation
- **Image Workflows**: Upload, metadata management, and downloads **Core Functionality:**
- **Search Functionality**: Text and tag-based search - ✅ **Bootstrap Setup**: Automatic creation of isolated test environment with artificial data
- **Multi-team Isolation**: Ensuring data privacy between teams - ✅ **Authentication**: API key validation and verification
- **Error Handling**: Validation and error response testing - ✅ **Team Management**: Create, read, update, delete teams
- ✅ **User Management**: Create, read, update, delete users
- ✅ **API Key Management**: Create, list, revoke API keys
**Image Operations:**
- ✅ **Image Upload**: File upload with metadata
- ✅ **Image Retrieval**: Get image details and download
- ✅ **Image Updates**: Modify descriptions and tags
- ✅ **Image Listing**: Paginated image lists with filters
**Advanced Search Functionality:**
- ✅ **Text Search**: Search by description content
- ✅ **Tag Search**: Filter by tags
- ✅ **Advanced Search**: Combined filters and thresholds
- ✅ **Similarity Search**: Find similar images using embeddings
- ✅ **Search Performance**: Response time validation
**Security and Isolation:**
- ✅ **User Roles**: Admin vs regular user permissions
- ✅ **Multi-team Isolation**: Data privacy between teams
- ✅ **Access Control**: Unauthorized access prevention
- ✅ **Error Handling**: Graceful error responses
**Performance and Scalability:**
- ✅ **Bulk Operations**: Multiple image uploads
- ✅ **Concurrent Access**: Simultaneous user operations
- ✅ **Database Performance**: Query response times
- ✅ **Data Consistency**: Transaction integrity
#### **Test Features**
**🎯 Completely Self-Contained**
- **No setup required**: Tests create their own isolated environment
- **Artificial test data**: Each test class creates unique teams, users, and images
- **Automatic cleanup**: All test data is deleted after tests complete
- **No environment variables needed**: Just run the tests!
**🔒 Isolated and Safe**
- **Unique identifiers**: Each test uses timestamp-based unique names
- **No conflicts**: Tests can run in parallel without interference
- **No database pollution**: Tests don't affect existing data
- **Idempotent**: Can be run multiple times safely
**⚡ Performance-Aware**
- **Class-scoped fixtures**: Expensive setup shared across test methods
- **Efficient cleanup**: Resources deleted in optimal order
- **Real database tests**: Optional performance testing with larger datasets
- **Timing validation**: Response time assertions for critical operations
#### **Advanced Test Modes**
**Standard E2E Tests (No Setup Required)**
```bash
# Just run them - completely self-contained!
python scripts/run_tests.py e2e
```
**Integration Tests with Real Services**
```bash
# Enable integration tests with real Google Cloud services
export E2E_INTEGRATION_TEST=1
pytest -m integration
```
**Real Database Performance Tests**
```bash
# Enable real database tests with larger datasets
export E2E_REALDB_TEST=1
pytest -m realdb
```
For detailed testing information, see [docs/TESTING.md](docs/TESTING.md). For detailed testing information, see [docs/TESTING.md](docs/TESTING.md).

653
docs/TESTING.md
View File

@ -1,449 +1,236 @@
# SEREACT Testing Guide # SEREACT Testing Guide
This document provides comprehensive information about testing the SEREACT API, including different test types, setup instructions, and best practices. This document provides comprehensive information about testing the SEREACT API, including unit tests, integration tests, and end-to-end tests.
## Test Types ## Test Types
SEREACT uses a multi-layered testing approach to ensure reliability and maintainability: SEREACT includes several types of tests to ensure code quality and functionality:
### 1. Unit Tests ### 1. Unit Tests (`unit`)
- **Purpose**: Test individual components in isolation - **Purpose**: Test individual components in isolation using mocks
- **Speed**: Fast (< 1 second per test) - **Speed**: Fast (< 1 second per test)
- **Dependencies**: Use mocks and stubs - **Dependencies**: None (uses mocks)
- **Coverage**: Functions, classes, and modules - **Location**: `tests/` (excluding `test_e2e.py`)
- **Location**: `tests/` (excluding `tests/integration/`)
### 2. Integration Tests ### 2. Integration Tests (`integration`)
- **Purpose**: Test interactions with real external services - **Purpose**: Test component interactions with real services
- **Speed**: Moderate (1-10 seconds per test) - **Speed**: Medium (1-5 seconds per test)
- **Dependencies**: Real Firestore database - **Dependencies**: Real database connections
- **Coverage**: Database operations, service integrations
- **Location**: `tests/integration/` - **Location**: `tests/integration/`
### 3. End-to-End (E2E) Tests ### 3. End-to-End Tests (`e2e`)
- **Purpose**: Test complete user workflows - **Purpose**: Test complete user workflows from API to database
- **Speed**: Moderate to slow (5-30 seconds per test) - **Speed**: Medium to slow (2-10 seconds per test)
- **Dependencies**: Full application stack (mocked or real) - **Dependencies**: **Self-contained with artificial test data**
- **Coverage**: Complete API workflows
- **Location**: `tests/test_e2e.py` - **Location**: `tests/test_e2e.py`
## Test Structure ### 4. Real Database Tests (`realdb`)
- **Purpose**: Test performance and scalability with real database
``` - **Speed**: Slow (5-30 seconds per test)
tests/ - **Dependencies**: Real database with artificial test data
├── conftest.py # Global test configuration - **Location**: `tests/test_e2e.py` (marked with `@pytest.mark.realdb`)
├── test_e2e.py # End-to-end workflow tests
├── api/ # API endpoint tests
│ ├── conftest.py # API-specific fixtures
│ ├── test_auth.py # Authentication tests
│ ├── test_teams.py # Team management tests
│ ├── test_users.py # User management tests
│ ├── test_images.py # Image management tests
│ └── test_search.py # Search functionality tests
├── auth/ # Authentication module tests
├── db/ # Database layer tests
├── integration/ # Integration tests
│ ├── __init__.py
│ └── test_firestore_integration.py
├── models/ # Data model tests
└── services/ # Business logic tests
```
## Running Tests ## Running Tests
### Prerequisites
1. **Virtual Environment**: Ensure you're in the project's virtual environment:
```bash
# Windows (Git Bash)
source venv/Scripts/activate
# Linux/macOS
source venv/bin/activate
```
2. **Dependencies**: Install test dependencies:
```bash
pip install -r requirements.txt
```
### Quick Start ### Quick Start
Use the test runner script for convenient test execution:
```bash ```bash
# Run unit tests only (fast, recommended for development) # Run all tests (recommended for development)
python scripts/run_tests.py unit
# Run end-to-end tests with mocked services
python scripts/run_tests.py e2e
# Run integration tests (requires real database)
python scripts/run_tests.py integration
# Run all tests
python scripts/run_tests.py all python scripts/run_tests.py all
# Run tests with coverage report # Run only unit tests (fastest)
python scripts/run_tests.py unit
# Run E2E tests (completely self-contained)
python scripts/run_tests.py e2e
# Run with coverage report
python scripts/run_tests.py coverage python scripts/run_tests.py coverage
``` ```
### Direct pytest Commands ### Using pytest directly
For more control, use pytest directly:
```bash ```bash
# Unit tests only # Run all tests
pytest -m "not integration and not e2e" -v pytest
# End-to-end tests # Run specific test types
pytest -m e2e -v pytest -m unit # Unit tests only
pytest -m integration # Integration tests only
pytest -m e2e # End-to-end tests only
pytest -m realdb # Real database tests only
# Integration tests # Run specific test files
FIRESTORE_INTEGRATION_TEST=1 pytest -m integration -v pytest tests/test_e2e.py # All E2E tests
pytest tests/api/ # All API tests
# Specific test file # Run specific test methods
pytest tests/test_e2e.py -v pytest tests/test_e2e.py::TestE2EWorkflows::test_bootstrap_and_basic_workflow
# Specific test function
pytest tests/test_e2e.py::TestE2EWorkflows::test_complete_team_workflow -v
# Run with coverage
pytest --cov=src --cov-report=html --cov-report=term
``` ```
## End-to-End Test Coverage ### Test Combinations
The E2E tests cover the following complete workflows: ```bash
# Run unit and integration tests (skip E2E)
pytest -m "not e2e and not realdb"
### 1. Team Management Workflow # Run all tests except real database tests
- Create a new team pytest -m "not realdb"
- Retrieve team details
- Update team information
- List all teams
- Verify team isolation
### 2. User Management Workflow # Run only E2E tests that don't require real database
- Create admin and regular users pytest -m "e2e and not realdb"
- Assign users to teams ```
- Update user roles and permissions
- List team members
- Verify user access controls
### 3. API Key Authentication Workflow ## End-to-End Test Setup
- Generate API keys for users
- Authenticate requests using API keys
- Test protected endpoints
- Manage API key lifecycle (create, use, deactivate)
- Verify authentication failures
### 4. Image Upload and Management Workflow **The E2E tests are now completely self-contained!** They automatically:
- Upload images with metadata
- Retrieve image details
- Update image metadata and tags
- List team images
- Download images
- Verify file handling
### 5. Search Workflow 1. **Create artificial test data** at the start of each test class
- Text-based search by description 2. **Run all tests** against this isolated test environment
- Tag-based filtering 3. **Clean up all test data** at the end automatically
- Combined search queries
- Search result pagination
- Verify search accuracy
### 6. Multi-Team Isolation ### No Setup Required!
- Create multiple teams
- Upload images to different teams
- Verify cross-team access restrictions
- Test search result isolation
- Ensure data privacy
### 7. Error Handling ```bash
- Invalid data validation # Just run the tests - no environment variables or API keys needed!
- Authentication failures python scripts/run_tests.py e2e
- Resource not found scenarios
- File upload errors
- Proper error responses
## Integration Test Setup # Or with pytest directly
pytest -m e2e
```
Integration tests require real external services. Follow these steps: ### Test Environment Creation
### 1. Firestore Setup Each test class automatically creates its own isolated environment:
1. **Create a test database**: - **Unique team** with timestamp-based naming to avoid conflicts
- Use a separate Firestore database for testing - **Admin user** with unique email addresses
- Database name should end with `-test` (e.g., `sereact-test`) - **API keys** for authentication
- **Test images** uploaded during tests
- **Additional users/teams** as needed for specific tests
2. **Set environment variables**: ### Automatic Cleanup
```bash
export FIRESTORE_INTEGRATION_TEST=1
export FIRESTORE_PROJECT_ID=your-test-project
export FIRESTORE_DATABASE_NAME=sereact-test
export FIRESTORE_CREDENTIALS_FILE=path/to/test-credentials.json
```
3. **Run integration tests**: At the end of each test class, all created resources are automatically deleted:
```bash
python scripts/run_tests.py integration
```
### 2. Full E2E Integration Setup - All uploaded images are removed
- All created users are deleted
- All created teams are removed
- All API keys are revoked
For testing with real cloud services: ### Advanced Test Modes
1. **Set up all services**: #### Integration Tests with Real Services
- Google Cloud Storage bucket For testing with real Google Cloud services:
- Firestore database
- Cloud Vision API
- Pinecone vector database
2. **Configure environment**: ```bash
```bash # Enable integration tests
export E2E_INTEGRATION_TEST=1 export E2E_INTEGRATION_TEST=1
export GCS_BUCKET_NAME=your-test-bucket
export VECTOR_DB_API_KEY=your-pinecone-key
# ... other service credentials
```
3. **Run E2E integration tests**: # Run integration tests
```bash pytest -m integration
python scripts/run_tests.py e2e --with-integration ```
```
#### Real Database Performance Tests
For testing with real database connections and larger datasets:
```bash
# Enable real database tests
export E2E_REALDB_TEST=1
# Run real database tests
pytest -m realdb
```
## E2E Test Coverage
The E2E tests cover the following workflows with artificial test data:
### Core Functionality
- ✅ **Bootstrap Setup**: Automatic creation of isolated test environment
- ✅ **Authentication**: API key validation and verification
- ✅ **Team Management**: Create, read, update, delete teams
- ✅ **User Management**: Create, read, update, delete users
- ✅ **API Key Management**: Create, list, revoke API keys
### Image Operations
- ✅ **Image Upload**: File upload with metadata
- ✅ **Image Retrieval**: Get image details and download
- ✅ **Image Updates**: Modify descriptions and tags
- ✅ **Image Listing**: Paginated image lists with filters
### Advanced Search Functionality
- ✅ **Text Search**: Search by description content
- ✅ **Tag Search**: Filter by tags
- ✅ **Advanced Search**: Combined filters and thresholds
- ✅ **Similarity Search**: Find similar images using embeddings
- ✅ **Search Performance**: Response time validation
### Security and Isolation
- ✅ **User Roles**: Admin vs regular user permissions
- ✅ **Multi-team Isolation**: Data privacy between teams
- ✅ **Access Control**: Unauthorized access prevention
- ✅ **Error Handling**: Graceful error responses
### Performance and Scalability
- ✅ **Bulk Operations**: Multiple image uploads
- ✅ **Concurrent Access**: Simultaneous user operations
- ✅ **Database Performance**: Query response times
- ✅ **Data Consistency**: Transaction integrity
## Test Data Management ## Test Data Management
### Fixtures and Test Data ### Unique Identifiers
All E2E tests use unique suffixes to avoid conflicts:
```python
unique_suffix = str(uuid.uuid4())[:8]
team_name = f"E2E Test Team {unique_suffix}_{int(time.time())}"
```
- **Shared fixtures**: Defined in `tests/conftest.py` ### Isolation Strategy
- **API fixtures**: Defined in `tests/api/conftest.py` Tests are completely isolated:
- **Sample images**: Generated programmatically using PIL - Each test class creates its own environment
- **Test data**: Isolated per test function - Uses timestamp-based unique identifiers
- No dependency on existing database state
- Can run in parallel without conflicts
### Automatic Resource Tracking
The test environment tracks all created resources:
```python
"created_resources": {
"teams": [team_id],
"users": [admin_user_id],
"api_keys": [api_key_id],
"images": []
}
```
### Cleanup Strategy ### Cleanup Strategy
Comprehensive cleanup at test completion:
- Images deleted first (to avoid orphaned files)
- Additional users deleted (preserving admin for team deletion)
- Additional teams deleted
- Main team deleted last (cascades to remaining resources)
- **Unit tests**: Automatic cleanup through mocking ## Environment Variables
- **Integration tests**: Manual cleanup in test teardown
- **E2E tests**: Resource tracking and cleanup utilities
## Best Practices ### No Variables Required for Basic E2E Tests!
The standard E2E tests now run without any environment variables.
### Writing Tests
1. **Test naming**: Use descriptive names that explain the scenario
```python
def test_user_cannot_access_other_team_images(self):
```
2. **Test structure**: Follow Arrange-Act-Assert pattern
```python
def test_create_team(self):
# Arrange
team_data = {"name": "Test Team"}
# Act
response = client.post("/api/v1/teams", json=team_data)
# Assert
assert response.status_code == 201
assert response.json()["name"] == "Test Team"
```
3. **Test isolation**: Each test should be independent
4. **Mock external services**: Use mocks for unit tests
5. **Use fixtures**: Leverage pytest fixtures for common setup
### Running Tests in Development
1. **Fast feedback loop**: Run unit tests frequently
```bash
pytest -m "not integration and not e2e" --tb=short
```
2. **Pre-commit testing**: Run E2E tests before committing
```bash
python scripts/run_tests.py e2e
```
3. **Coverage monitoring**: Check test coverage regularly
```bash
python scripts/run_tests.py coverage
```
### CI/CD Integration
For continuous integration, use different test strategies:
```yaml
# Example GitHub Actions workflow
- name: Run unit tests
run: python scripts/run_tests.py unit
- name: Run E2E tests
run: python scripts/run_tests.py e2e
- name: Run integration tests (if credentials available)
run: python scripts/run_tests.py integration
if: env.FIRESTORE_INTEGRATION_TEST == '1'
```
## Troubleshooting
### Common Issues
1. **Import errors**: Ensure you're in the virtual environment
2. **Database connection**: Check Firestore credentials for integration tests
3. **Slow tests**: Use unit tests for development, integration tests for CI
4. **Test isolation**: Clear test data between runs
### Debug Mode
Run tests with additional debugging:
### Optional for Enhanced Testing
```bash ```bash
# Verbose output with full tracebacks # Enable integration tests with real services
pytest -v --tb=long E2E_INTEGRATION_TEST=1
# Stop on first failure # Enable real database performance tests
pytest -x E2E_REALDB_TEST=1
# Run specific test with debugging # Custom test database (if different from main)
pytest tests/test_e2e.py::TestE2EWorkflows::test_complete_team_workflow -v -s TEST_FIRESTORE_PROJECT_ID="your-test-project"
TEST_GCS_BUCKET_NAME="your-test-bucket"
``` ```
### Performance Monitoring ## Continuous Integration
Monitor test performance:
```bash
# Show slowest tests
pytest --durations=10
# Profile test execution
pytest --profile
```
## Test Metrics
Track these metrics to ensure test quality:
- **Coverage**: Aim for >80% code coverage
- **Speed**: Unit tests <1s, E2E tests <30s
- **Reliability**: Tests should pass consistently
- **Maintainability**: Tests should be easy to update
## Contributing
When adding new features:
1. **Write tests first**: Use TDD approach
2. **Cover all scenarios**: Happy path, edge cases, error conditions
3. **Update documentation**: Keep this guide current
4. **Run full test suite**: Ensure no regressions
For more information about the SEREACT API architecture and features, see the main [README.md](../README.md).
## Running E2E Tests
### With Fresh Database
If you have a fresh database, the E2E tests will automatically run the bootstrap process:
```bash
pytest tests/test_e2e.py -v -m e2e
```
### With Existing Database
If your database already has teams and users (bootstrap completed), you need to provide an API key:
1. **Get an existing API key** from your application or create one via the API
2. **Set the environment variable**:
```bash
export E2E_TEST_API_KEY="your-api-key-here"
```
3. **Run the tests**:
```bash
pytest tests/test_e2e.py -v -m e2e
```
### Example with API Key
```bash
# Set your API key
export E2E_TEST_API_KEY="sk_test_1234567890abcdef"
# Run E2E tests
python scripts/run_tests.py e2e
```
## Test Features
### Idempotent Tests
The E2E tests are designed to be idempotent - they can be run multiple times against the same database without conflicts:
- **Unique identifiers**: Each test run uses unique suffixes for all created data
- **Graceful handling**: Tests handle existing data gracefully
- **Cleanup**: Tests create isolated data that doesn't interfere with existing data
### Test Data Isolation
- Each test run creates unique teams, users, and images
- Tests use UUID-based suffixes to avoid naming conflicts
- Search tests use unique tags to find only test-created data
## Test Configuration
### Environment Variables
- `E2E_TEST_API_KEY`: API key for E2E tests with existing database
- `E2E_INTEGRATION_TEST`: Set to `1` to enable integration tests
- `TEST_DATABASE_URL`: Override database for testing (optional)
### Pytest Configuration
The `pytest.ini` file contains:
- Test markers for categorizing tests
- Async test configuration
- Warning filters
## Best Practices
### Writing Tests
1. **Use descriptive names**: Test names should clearly describe what they test
2. **Test one thing**: Each test should focus on a single workflow or feature
3. **Use fixtures**: Leverage pytest fixtures for common setup
4. **Handle errors**: Test both success and error scenarios
5. **Clean up**: Ensure tests don't leave behind test data (when possible)
### Running Tests
1. **Run frequently**: Run unit tests during development
2. **CI/CD integration**: Ensure all tests pass before deployment
3. **Test environments**: Use separate databases for testing
4. **Monitor performance**: Track test execution time
## Troubleshooting
### Common Issues
#### "Bootstrap already completed"
- **Cause**: Database already has teams/users
- **Solution**: Set `E2E_TEST_API_KEY` environment variable
#### "No existing API key found"
- **Cause**: No valid API key provided for existing database
- **Solution**: Create an API key via the API or bootstrap endpoint
#### "Failed to create test team"
- **Cause**: Insufficient permissions or API key issues
- **Solution**: Ensure the API key belongs to an admin user
#### Import errors
- **Cause**: Python path or dependency issues
- **Solution**: Ensure virtual environment is activated and dependencies installed
### Getting Help
1. Check the test output for specific error messages
2. Verify environment variables are set correctly
3. Ensure the API server is running (for integration tests)
4. Check database connectivity
## CI/CD Integration
### GitHub Actions Example ### GitHub Actions Example
```yaml ```yaml
@ -459,43 +246,89 @@ jobs:
with: with:
python-version: 3.10 python-version: 3.10
- name: Install dependencies - name: Install dependencies
run: | run: pip install -r requirements.txt
pip install -r requirements.txt
- name: Run unit tests - name: Run unit tests
run: python scripts/run_tests.py unit run: python scripts/run_tests.py unit
- name: Run integration tests - name: Run E2E tests (self-contained)
run: python scripts/run_tests.py integration run: python scripts/run_tests.py e2e
env: # No environment variables needed!
E2E_INTEGRATION_TEST: 1
``` ```
### Docker Testing ## Troubleshooting
```dockerfile
# Test stage
FROM python:3.10-slim as test
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
RUN python scripts/run_tests.py unit
```
## Coverage Reports ### Common Issues
Generate coverage reports: #### "Cannot create isolated test environment" Error
```bash ```bash
python scripts/run_tests.py coverage # This is rare but can happen if database has conflicting constraints
# Solution: Check database state or use a clean test database
``` ```
View HTML coverage report: #### Tests Skipped Due to Missing Environment Variables
```bash ```bash
open htmlcov/index.html # Only affects integration and realdb tests
echo $E2E_INTEGRATION_TEST # Should be "1" for integration tests
echo $E2E_REALDB_TEST # Should be "1" for real database tests
``` ```
## Performance Testing #### Slow Test Performance
```bash
# Run only fast tests
pytest -m "not realdb and not integration"
For performance testing: # Run tests in parallel (requires pytest-xdist)
1. Use `pytest-benchmark` for micro-benchmarks pip install pytest-xdist
2. Test with realistic data volumes pytest -n auto
3. Monitor database query performance ```
4. Test concurrent user scenarios
### Debug Mode
```bash
# Run with verbose output
pytest -v -s tests/test_e2e.py
# Run single test with full output
pytest -v -s tests/test_e2e.py::TestE2EWorkflows::test_bootstrap_and_basic_workflow
```
## Best Practices
### Writing New Tests
1. **Use the test_environment fixture** for automatic setup/cleanup
2. **Track created resources** in env["created_resources"]
3. **Use unique identifiers** for all test data
4. **Test both success and failure** scenarios
5. **Use appropriate markers** (`@pytest.mark.e2e`, etc.)
### Test Organization
1. **Group related tests** in classes with shared fixtures
2. **Use descriptive test names** that explain the scenario
3. **Keep tests independent** - no shared state between methods
4. **Use class-scoped fixtures** for expensive setup
5. **Document test purpose** in docstrings
### Performance Considerations
1. **Use class-scoped fixtures** to share expensive setup
2. **Minimize database operations** in individual tests
3. **Clean up test data** automatically
4. **Run expensive tests** only when necessary
5. **Use artificial data** instead of real external dependencies
## Test Metrics
### Coverage Goals
- **Unit Tests**: > 90% code coverage
- **Integration Tests**: > 80% API endpoint coverage
- **E2E Tests**: > 95% user workflow coverage
### Performance Targets
- **Unit Tests**: < 1 second per test
- **Integration Tests**: < 5 seconds per test
- **E2E Tests**: < 10 seconds per test
- **Real DB Tests**: < 30 seconds per test
### Quality Metrics
- **Test Reliability**: > 99% pass rate
- **Test Maintainability**: Clear, readable test code
- **Test Coverage**: All critical paths tested
- **Test Documentation**: All test purposes documented
- **Test Isolation**: No dependencies between tests

6
pytest.ini
View File

@ -1,4 +1,4 @@
[tool:pytest] [pytest]
asyncio_mode = auto asyncio_mode = auto
asyncio_default_fixture_loop_scope = function asyncio_default_fixture_loop_scope = function
testpaths = tests testpaths = tests
@ -14,10 +14,12 @@ markers =
asyncio: marks tests as async (deselect with '-m "not asyncio"') asyncio: marks tests as async (deselect with '-m "not asyncio"')
integration: marks tests as integration tests requiring real database (deselect with '-m "not integration"') integration: marks tests as integration tests requiring real database (deselect with '-m "not integration"')
e2e: marks tests as end-to-end tests covering complete workflows (deselect with '-m "not e2e"') e2e: marks tests as end-to-end tests covering complete workflows (deselect with '-m "not e2e"')
realdb: marks tests as real database tests requiring actual database connections (deselect with '-m "not realdb"')
unit: marks tests as unit tests using mocks (default) unit: marks tests as unit tests using mocks (default)
# Test configuration # Test configuration
# To run unit tests only: pytest -m "not integration and not e2e" # To run unit tests only: pytest -m "not integration and not e2e and not realdb"
# To run integration tests: pytest -m integration # To run integration tests: pytest -m integration
# To run e2e tests: pytest -m e2e # To run e2e tests: pytest -m e2e
# To run real database tests: pytest -m realdb
# To run all tests: pytest # To run all tests: pytest

8
scripts/run_tests.py
View File

@ -79,7 +79,13 @@ def run_command(cmd, description):
os.chdir(project_root) os.chdir(project_root)
try: try:
result = subprocess.run(cmd, capture_output=False, text=True) # Use the same Python executable that's running this script
# This ensures we use the virtual environment
if cmd[0] == "python":
cmd[0] = sys.executable
# Inherit the current environment (including virtual environment)
result = subprocess.run(cmd, capture_output=False, text=True, env=os.environ.copy())
os.chdir(original_cwd) os.chdir(original_cwd)
return result.returncode == 0 return result.returncode == 0
except Exception as e: except Exception as e:

106
src/db/repositories/firestore_image_repository.py
View File

@ -1,4 +1,6 @@
import logging import logging
from typing import Optional, List
from bson import ObjectId
from src.db.repositories.firestore_repository import FirestoreRepository from src.db.repositories.firestore_repository import FirestoreRepository
from src.models.image import ImageModel from src.models.image import ImageModel
@ -29,6 +31,110 @@ class FirestoreImageRepository(FirestoreRepository[ImageModel]):
logger.error(f"Error getting images by team ID: {e}") logger.error(f"Error getting images by team ID: {e}")
raise raise
async def get_by_team(
self,
team_id: ObjectId,
skip: int = 0,
limit: int = 50,
collection_id: Optional[ObjectId] = None,
tags: Optional[List[str]] = None
) -> List[ImageModel]:
"""
Get images by team with pagination and filtering
Args:
team_id: Team ID
skip: Number of records to skip
limit: Maximum number of records to return
collection_id: Optional collection ID filter
tags: Optional list of tags to filter by
Returns:
List of images
"""
try:
# Get all images and filter in memory (for simplicity)
images = await self.get_all()
# Filter by team
filtered_images = [image for image in images if image.team_id == team_id]
# Filter by collection if specified
if collection_id:
filtered_images = [image for image in filtered_images if image.collection_id == collection_id]
# Filter by tags if specified
if tags:
filtered_images = [
image for image in filtered_images
if any(tag in image.tags for tag in tags)
]
# Apply pagination
return filtered_images[skip:skip + limit]
except Exception as e:
logger.error(f"Error getting images by team: {e}")
raise
async def count_by_team(
self,
team_id: ObjectId,
collection_id: Optional[ObjectId] = None,
tags: Optional[List[str]] = None
) -> int:
"""
Count images by team with filtering
Args:
team_id: Team ID
collection_id: Optional collection ID filter
tags: Optional list of tags to filter by
Returns:
Count of images
"""
try:
# Get all images and filter in memory (for simplicity)
images = await self.get_all()
# Filter by team
filtered_images = [image for image in images if image.team_id == team_id]
# Filter by collection if specified
if collection_id:
filtered_images = [image for image in filtered_images if image.collection_id == collection_id]
# Filter by tags if specified
if tags:
filtered_images = [
image for image in filtered_images
if any(tag in image.tags for tag in tags)
]
return len(filtered_images)
except Exception as e:
logger.error(f"Error counting images by team: {e}")
raise
async def update_last_accessed(self, image_id: ObjectId) -> bool:
"""
Update the last accessed timestamp for an image
Args:
image_id: Image ID
Returns:
True if successful
"""
try:
from datetime import datetime
update_data = {"last_accessed": datetime.utcnow()}
result = await self.update(image_id, update_data)
return result is not None
except Exception as e:
logger.error(f"Error updating last accessed: {e}")
return False
async def get_by_uploader_id(self, uploader_id: str) -> list[ImageModel]: async def get_by_uploader_id(self, uploader_id: str) -> list[ImageModel]:
""" """
Get images by uploader ID Get images by uploader ID

67
src/db/repositories/firestore_repository.py
View File

@ -1,19 +1,21 @@
import logging import logging
from typing import Dict, List, Optional, Type, Any, Generic, TypeVar from typing import TypeVar, Generic, Optional, List, Dict, Any
from bson import ObjectId
from pydantic import BaseModel from pydantic import BaseModel
from src.db.providers.firestore_provider import firestore_db from src.db.providers.firestore_provider import firestore_db
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
T = TypeVar('T', bound=BaseModel) T = TypeVar('T')
class FirestoreRepository(Generic[T]): class FirestoreRepository(Generic[T]):
"""Generic repository for Firestore operations""" """Base repository class for Firestore operations"""
def __init__(self, collection_name: str, model_class: Type[T]): def __init__(self, collection_name: str, model_class):
self.collection_name = collection_name self.collection_name = collection_name
self.model_class = model_class self.model_class = model_class
self.provider = firestore_db
async def create(self, model: T) -> T: async def create(self, model: T) -> T:
""" """
@ -30,10 +32,10 @@ class FirestoreRepository(Generic[T]):
model_dict = model.model_dump(by_alias=True) model_dict = model.model_dump(by_alias=True)
# Add document to Firestore # Add document to Firestore
doc_id = await firestore_db.add_document(self.collection_name, model_dict) doc_id = await self.provider.add_document(self.collection_name, model_dict)
# Get the created document # Get the created document
doc_data = await firestore_db.get_document(self.collection_name, doc_id) doc_data = await self.provider.get_document(self.collection_name, doc_id)
return self.model_class(**doc_data) return self.model_class(**doc_data)
except Exception as e: except Exception as e:
logger.error(f"Error creating {self.collection_name} document: {e}") logger.error(f"Error creating {self.collection_name} document: {e}")
@ -50,7 +52,7 @@ class FirestoreRepository(Generic[T]):
Model if found, None otherwise Model if found, None otherwise
""" """
try: try:
doc_data = await firestore_db.get_document(self.collection_name, str(doc_id)) doc_data = await self.provider.get_document(self.collection_name, str(doc_id))
if doc_data: if doc_data:
return self.model_class(**doc_data) return self.model_class(**doc_data)
return None return None
@ -60,17 +62,56 @@ class FirestoreRepository(Generic[T]):
async def get_all(self) -> List[T]: async def get_all(self) -> List[T]:
""" """
Get all documents Get all documents from the collection
Returns: Returns:
List of models List of model instances
""" """
try: try:
docs = await firestore_db.list_documents(self.collection_name) docs = await self.provider.list_documents(self.collection_name)
return [self.model_class(**doc) for doc in docs]
# Transform data to handle legacy format issues
transformed_docs = []
for doc in docs:
transformed_doc = self._transform_document(doc)
transformed_docs.append(transformed_doc)
return [self.model_class(**doc) for doc in transformed_docs]
except Exception as e: except Exception as e:
logger.error(f"Error getting all {self.collection_name} documents: {e}") logger.error(f"Error getting all {self.collection_name} documents: {e}")
raise raise
def _transform_document(self, doc: Dict[str, Any]) -> Dict[str, Any]:
"""
Transform document data to handle legacy format issues
Args:
doc: Raw document data
Returns:
Transformed document data
"""
# Handle tags field - convert string representation to list
if 'tags' in doc and isinstance(doc['tags'], str):
try:
# Try to parse as Python literal (list representation)
import ast
doc['tags'] = ast.literal_eval(doc['tags'])
except (ValueError, SyntaxError):
# If that fails, split by comma
doc['tags'] = [tag.strip() for tag in doc['tags'].split(',') if tag.strip()]
# Handle metadata field - convert string representation to dict
if 'metadata' in doc and isinstance(doc['metadata'], str):
try:
# Try to parse as Python literal (dict representation)
import ast
doc['metadata'] = ast.literal_eval(doc['metadata'])
except (ValueError, SyntaxError):
# If that fails, set to empty dict
doc['metadata'] = {}
return doc
async def update(self, doc_id: str, update_data: Dict[str, Any]) -> Optional[T]: async def update(self, doc_id: str, update_data: Dict[str, Any]) -> Optional[T]:
""" """
@ -89,7 +130,7 @@ class FirestoreRepository(Generic[T]):
del update_data["_id"] del update_data["_id"]
# Update document # Update document
success = await firestore_db.update_document( success = await self.provider.update_document(
self.collection_name, self.collection_name,
str(doc_id), str(doc_id),
update_data update_data
@ -115,7 +156,7 @@ class FirestoreRepository(Generic[T]):
True if document was deleted, False otherwise True if document was deleted, False otherwise
""" """
try: try:
return await firestore_db.delete_document(self.collection_name, str(doc_id)) return await self.provider.delete_document(self.collection_name, str(doc_id))
except Exception as e: except Exception as e:
logger.error(f"Error deleting {self.collection_name} document: {e}") logger.error(f"Error deleting {self.collection_name} document: {e}")
raise raise

1186
tests/test_e2e.py
View File

File diff suppressed because it is too large Load Diff