Architecture Overview

nself provides a comprehensive, modern backend architecture that combines the best of cloud-native technologies. With v0.3.5 delivering 100% service reliability (17/17 services), smart port allocation, and complete SSL/HTTPS support, nself creates a scalable, production-ready backend infrastructure.

Service Reliability & Port Management v0.3.5

100% Service Success Rate

nself v0.3.5 achieves perfect service reliability with comprehensive fixes:

• Volume Mount Reliability: Fixed critical Docker volume mounting issues that caused service failures
• Dependency Resolution: Improved service startup order and dependency management
• Health Check Enhancement: Better health monitoring and automatic recovery
• Configuration Validation: Pre-startup validation prevents configuration-related failures

Smart Port Allocation Strategy

Intelligent port management for better organization and conflict prevention:

SSL/HTTPS Architecture v0.3.5

Complete HTTPS by Default

All services now run with HTTPS enabled and trusted certificates:

• Dual Domain Support: *.localhost and *.local.nself.org for maximum compatibility
• Automatic Certificate Generation: nself trust installs trusted CA certificates
• Green Lock Compatibility: All major browsers show secure lock icon
• Cross-Platform Support: Works on macOS, Windows, and Linux

Core Architecture Principles

Microservices Architecture

nself is built on microservices principles, where each service has a specific responsibility:

• Single Responsibility: Each service handles one domain or function
• Independent Deployment: Services can be deployed and scaled independently
• Technology Diversity: Use the best technology for each service
• Fault Isolation: Failure in one service doesn't bring down the entire system

Container-First Design

Everything in nself runs in Docker containers:

• Consistent Environments: Same container runs in dev, staging, and production
• Resource Isolation: Each service has its own resource allocation
• Easy Scaling: Scale services by adding more container instances
• Cloud Native: Ready for Kubernetes and cloud deployment

System Components

Core Infrastructure Stack

Data Flow Architecture

Request Flow

1. Client Request: Frontend app or external client makes API call
2. Nginx Proxy: Routes request to appropriate service
3. Authentication: JWT token validation and user context
4. API Processing: GraphQL query or REST endpoint processing
5. Database Query: PostgreSQL data retrieval or modification
6. Response: JSON response back to client

Background Processing Flow

1. Job Creation: API endpoint or webhook creates background job
2. Queue Storage: Job stored in Redis queue
3. Worker Processing: BullMQ worker picks up and processes job
4. External Services: Worker may call external APIs or services
5. Result Storage: Job results stored in database or cache
6. Notifications: Webhooks or real-time updates sent to clients

Service Communication Patterns

Synchronous Communication

• HTTP/REST: Direct service-to-service API calls
• GraphQL: Unified API layer through Hasura
• gRPC: High-performance RPC for Go services

Asynchronous Communication

• Message Queues: BullMQ for job processing
• Event Streams: Redis pub/sub for real-time events
• Webhooks: HTTP callbacks for external integrations

Scalability Architecture

Horizontal Scaling

Scale by adding more instances:

# Scale specific services
NESTJS_API_REPLICAS=3
BULLMQ_EMAIL_REPLICAS=2
PYTHON_ML_API_REPLICAS=4

# Auto-scaling based on metrics
CPU_SCALE_THRESHOLD=80
MEMORY_SCALE_THRESHOLD=85

Vertical Scaling

Scale by increasing resource allocation:

# Resource limits per service
POSTGRES_MEMORY_LIMIT=2048MB
POSTGRES_CPU_LIMIT=2.0

HASURA_MEMORY_LIMIT=1024MB
HASURA_CPU_LIMIT=1.0

Security Architecture

Network Security

• Container Isolation: Each service runs in isolated container
• Internal Networks: Services communicate over private Docker networks
• Reverse Proxy: Nginx handles SSL termination and routing
• Firewall Rules: Only necessary ports exposed externally

Authentication & Authorization

• JWT Tokens: Stateless authentication across services
• Role-Based Access: User roles and permissions
• API Keys: Service-to-service authentication
• Row-Level Security: Database-level access control

Development vs Production Architecture

Development Environment

• All-in-One: Single machine deployment with 100% service reliability
• Hot Reloading: Code changes trigger automatic rebuilds
• HTTPS Development: All services accessible via HTTPS with trusted certificates
• Smart Port Management: Organized port allocation prevents conflicts
• Debug Tools: MailPit (port 3101), database adminer (port 3100), log viewers
• Enhanced Security: SSL by default while maintaining easy debugging

Production Environment

• Distributed: Services can run on multiple machines with smart port allocation
• High Availability: Multiple replicas and enhanced health checks
• 100% Uptime: Proven service reliability with automatic recovery
• SSL/TLS Everywhere: Complete HTTPS with Let's Encrypt or custom certificates
• Monitoring: Comprehensive logging and metrics
• Security Hardening: Enhanced SSL, secrets management, network isolation

Database Architecture

Schema-First Design

• DBML Schema: Human-readable database design
• Migration System: Version-controlled schema changes
• Automatic GraphQL: Hasura generates API from schema
• Type Safety: Shared types across services

Data Access Patterns

• GraphQL First: Primary API through Hasura
• Direct SQL: Custom services can query directly
• Connection Pooling: Efficient database connections
• Read Replicas: Scale read operations

Deployment Architecture

Single Machine Deployment

# Docker Compose on single server
version: '3.8'
services:
  postgres:
    image: postgres:15
    volumes:
      - postgres_data:/var/lib/postgresql/data
  
  hasura:
    image: hasura/graphql-engine:latest
    depends_on:
      - postgres
  
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    depends_on:
      - hasura

Multi-Node Deployment

# Kubernetes deployment example
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nself-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nself-api
  template:
    metadata:
      labels:
        app: nself-api
    spec:
      containers:
      - name: api
        image: my-registry/nself-api:latest
        resources:
          limits:
            memory: "512Mi"
            cpu: "500m"

Monitoring Architecture

Health Monitoring

• Service Health Checks: HTTP endpoints for each service
• Database Monitoring: Connection counts, query performance
• Resource Monitoring: CPU, memory, disk usage
• Application Metrics: Custom business metrics

Logging Architecture

• Centralized Logging: All logs aggregated in one place
• Structured Logs: JSON format for easy parsing
• Log Rotation: Automatic cleanup of old logs
• Search & Analytics: Full-text search capabilities

Disaster Recovery Architecture

Backup Strategy

• Database Backups: Automated PostgreSQL dumps
• File Backups: MinIO storage backup
• Configuration Backups: Environment and config files
• Off-site Storage: Cloud storage for backups

Recovery Procedures

• Point-in-Time Recovery: Restore to specific timestamp
• Rolling Updates: Zero-downtime deployments
• Rollback Capability: Quick revert to previous version
• Multi-Region Support: Geographic distribution

Performance Architecture

Caching Strategy

• Redis Cache: Application-level caching
• GraphQL Caching: Hasura query result caching
• CDN Integration: Static asset caching
• Database Query Cache: PostgreSQL query optimization

Performance Optimization

• Connection Pooling: Efficient database connections
• Lazy Loading: Load data only when needed
• Async Processing: Non-blocking operations
• Resource Limits: Prevent resource exhaustion

Extension Architecture

Plugin System

• Service Templates: Pre-built service configurations
• Custom Services: Add your own microservices
• Middleware: Custom request/response processing
• Hooks: Event-driven extensions

Integration Points

• Webhooks: External service integration
• API Gateways: Custom routing and transformation
• Message Queues: Event-driven architecture
• Custom Functions: Serverless compute

Best Practices

Service Design

• Stateless Services: Store state in databases, not containers
• Idempotent Operations: Safe to retry operations
• Circuit Breakers: Fail fast when dependencies are down
• Graceful Degradation: Continue operating with reduced functionality

Data Management

• Database Per Service: Each service owns its data
• Event Sourcing: Track all changes as events
• CQRS Pattern: Separate read and write models
• Data Consistency: Use transactions and constraints

Next Steps

Now that you understand the architecture:

• PostgreSQL Database - Learn about the database layer
• Hasura GraphQL - Understand the API layer
• Production Deployment - Deploy to production
• Monitoring & Logs - Set up observability

Understanding the architecture helps you make informed decisions about scaling, security, and extending your nself deployment.