image_management_api/deployment/terraform

Terraform Configuration for Sereact

This directory contains Terraform configurations to provision the required Google Cloud resources for Sereact:

• Google Cloud Run service
• Google Container Registry (GCR)
• Firestore database
• Cloud Storage bucket

Prerequisites

1. Install Terraform (v1.0.0+)
2. Install Google Cloud SDK
3. Authenticate with Google Cloud:

   gcloud auth login
   gcloud auth application-default login
   

4. Create or select a Google Cloud project:

   gcloud projects create PROJECT_ID --name="Sereact Project" # optional
   gcloud config set project PROJECT_ID
   

Setup and Usage

1. Copy the example variables file and edit it with your values:

   cp terraform.tfvars.example terraform.tfvars
   # Edit terraform.tfvars with your project-specific values
   

2. Initialize Terraform:

   terraform init
   

3. Preview the changes:

   terraform plan
   

4. Apply the configuration:

   terraform apply
   

5. After provisioning, you'll see outputs including:

   • Cloud Run service URL
   • Storage bucket name
   • Firestore database ID
   • Container Registry URL

Managing Secrets

Secrets for environment variables (API_KEY_SECRET, VECTOR_DB_API_KEY, etc.) should be managed separately using Google Secret Manager:

# Create secrets
gcloud secrets create sereact-api-key-secret --replication-policy="automatic"
gcloud secrets create sereact-vector-db-key --replication-policy="automatic"

# Add secret versions
echo -n "your-secret-value" | gcloud secrets versions add sereact-api-key-secret --data-file=-
echo -n "your-secret-value" | gcloud secrets versions add sereact-vector-db-key --data-file=-

# Update Cloud Run service to use secrets
gcloud run services update sereact \
  --update-secrets=API_KEY_SECRET=sereact-api-key-secret:latest,VECTOR_DB_API_KEY=sereact-vector-db-key:latest

CI/CD Integration

To integrate this with CI/CD, store the terraform.tfvars securely in your CI/CD system and run Terraform as part of your deployment pipeline:

# Example GitHub Actions step
- name: Terraform Apply
  run: |
    cd deployment/terraform
    terraform init
    terraform apply -auto-approve
  env:
    GOOGLE_APPLICATION_CREDENTIALS: ${{ secrets.GCP_SA_KEY }}

Destroying Resources

To destroy all provisioned resources:

terraform destroy

Terraform Infrastructure for Sereact

This directory contains Terraform configurations to deploy the complete Sereact infrastructure on Google Cloud Platform, including automatic configuration of Qdrant VM IP addresses for Cloud Run.

Architecture Overview

The infrastructure includes:

• Cloud Run Service: Main application service
• Qdrant VM: Vector database running on Compute Engine
• Firestore: Document database
• Cloud Storage: File storage
• Automatic IP Configuration: Qdrant VM IP is automatically passed to Cloud Run

Qdrant VM IP Address Integration

The Terraform configuration automatically handles passing the Qdrant VM IP address to your Cloud Run service:

How It Works

1. VM Creation: Terraform creates a Compute Engine VM running Qdrant
2. IP Address Extraction: Terraform extracts the VM's external IP address
3. Environment Variable Injection: The IP is automatically set as QDRANT_HOST in Cloud Run
4. Dependency Management: Cloud Run waits for the VM to be created before deploying

Configuration Options

Static vs Dynamic IP

You can choose between static and dynamic IP addresses:

# In terraform.tfvars
use_static_ip = true   # Use static IP (recommended for production)
use_static_ip = false  # Use ephemeral IP (cheaper for development)

Environment Variables Set Automatically

The following environment variables are automatically configured in Cloud Run:

QDRANT_HOST=<vm-external-ip>      # Automatically set from VM
QDRANT_PORT=6333                  # HTTP port
QDRANT_API_KEY=<your-api-key>     # From terraform.tfvars

Quick Start

1. Prerequisites

# Install required tools
gcloud auth login
gcloud config set project YOUR_PROJECT_ID
terraform --version

2. Configuration

# Copy and edit configuration
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your values

3. Deploy Everything

# From the root directory
./deployment/deploy.sh --deploy --build

4. Get Qdrant Information

# Get Qdrant VM IP and endpoints
./deployment/terraform/scripts/get_qdrant_ip.sh

# Get internal IP (for VPC connections)
./deployment/terraform/scripts/get_qdrant_ip.sh --internal

Terraform Outputs

The configuration provides several outputs for integration:

# Get specific outputs
terraform output cloud_run_url              # Cloud Run service URL
terraform output vector_db_vm_external_ip    # Qdrant VM external IP
terraform output vector_db_vm_internal_ip    # Qdrant VM internal IP
terraform output qdrant_http_endpoint        # Full HTTP endpoint
terraform output cloud_run_qdrant_host       # IP configured for Cloud Run

# Get deployment summary
terraform output deployment_summary

File Structure

terraform/
├── main.tf              # Cloud Run and main resources
├── vm.tf                # Qdrant VM configuration
├── outputs.tf           # Output definitions
├── variables.tf         # Variable definitions
├── terraform.tfvars     # Your configuration values
├── scripts/
│   ├── get_qdrant_ip.sh # Helper script to get VM IP
│   └── install_qdrant.sh # VM startup script
└── README.md            # This file

Environment Variables Reference

Automatically Configured

These are set automatically by Terraform:

Variable	Source	Description
QDRANT_HOST	VM external IP	Qdrant server IP address
QDRANT_PORT	Static value	Qdrant HTTP port (6333)
QDRANT_API_KEY	terraform.tfvars	Qdrant authentication key
FIRESTORE_PROJECT_ID	terraform.tfvars	GCP project ID
GCS_BUCKET_NAME	terraform.tfvars	Storage bucket name

Manual Configuration Required

Set these in terraform.tfvars:

project_id = "your-gcp-project-id"
storage_bucket_name = "your-bucket-name"
qdrant_api_key = "your-qdrant-api-key"
use_static_ip = true  # or false

Networking Configuration

Firewall Rules

The configuration automatically creates firewall rules to allow:

• Qdrant HTTP traffic (port 6333)
• Qdrant gRPC traffic (port 6334)
• Access from Cloud Run to Qdrant VM

IP Address Types

External IP (Default)

• Used for Cloud Run → Qdrant communication
• Accessible from internet (with firewall rules)
• Can be static or ephemeral

Internal IP (Alternative)

• Available for VPC-native connections
• More secure but requires VPC configuration
• Use cloud_run_qdrant_host_internal output

Troubleshooting

Check Qdrant Connectivity

# Get IP and test connection
./deployment/terraform/scripts/get_qdrant_ip.sh

# Manual test
QDRANT_IP=$(terraform output -raw vector_db_vm_external_ip)
curl http://$QDRANT_IP:6333/health

Verify Cloud Run Environment

# Check Cloud Run service
gcloud run services describe sereact --region=us-central1

# Check environment variables
gcloud run services describe sereact --region=us-central1 \
  --format="value(spec.template.spec.containers[0].env[].name,spec.template.spec.containers[0].env[].value)"

Common Issues

1. VM not ready: Qdrant installation takes 2-3 minutes after VM creation
2. Firewall blocking: Check that your IP is in allowed_cidr_blocks
3. Wrong IP type: Ensure you're using external IP for Cloud Run connections

Security Considerations

Production Recommendations

1. Use Static IP: Set use_static_ip = true
2. Restrict Access: Set specific CIDR blocks in allowed_cidr_blocks
3. Use Internal IP: Consider VPC-native networking for internal communication
4. Secure API Key: Store qdrant_api_key in Secret Manager

Development Setup

# terraform.tfvars for development
use_static_ip = false
allowed_cidr_blocks = "0.0.0.0/0"  # Open access (not for production!)

Cost Optimization

Development

• Use use_static_ip = false to avoid static IP charges
• Use smaller VM sizes: machine_type = "e2-micro"

Production

• Use use_static_ip = true for consistent connectivity
• Use appropriate VM sizes based on load

Advanced Configuration

Custom VM Configuration

Edit vm.tf to customize:

• Machine type
• Disk size
• Startup script
• Network configuration

VPC-Native Networking

For internal-only communication:

1. Create VPC connector
2. Configure Cloud Run to use VPC
3. Use internal IP output: cloud_run_qdrant_host_internal

Support

For issues with:

• Terraform: Check terraform plan output
• VM connectivity: Use the helper script get_qdrant_ip.sh
• Cloud Run: Check logs with gcloud run logs tail sereact