website/PROJECT_DEVELOPMENT.md

PROJECT_DEVELOPMENT.md - EZSCALE Site Architecture & Development Plan

1. Project Goal

Replace WHMCS with a custom Laravel 12 application for managing EZSCALE Hosting's VPS, Dedicated Server, Web Hosting, and Game Server business. The new platform must handle customer management, billing/subscriptions, automated provisioning, bandwidth monitoring, and comprehensive admin controls.

2. Infrastructure Overview

Current EZSCALE Stack

• VPS Platform: VirtFusion (KVM/QEMU hypervisor stack)
• Game Servers: Pterodactyl Panel
• Dedicated Servers: SynergyCP
• Web Hosting: Enhance (https://enhance.com/)
• Container Management: Portainer (for BFACP deployment)
• Support System: Standalone ticket system (built-in, replaced SupportPal)
• Network: Juniper switches with VLANs (dedicated customers, corporate, hypervisors)
• Bandwidth Monitoring: ElastiFlow (NetFlow/sFlow collector)

Domain Structure

• ezscale.cloud — Marketing frontend, product catalog, public pages
• account.ezscale.cloud — Customer dashboard, service management, billing
• admin.ezscale.cloud — Admin panel (behind Cloudflare Zero Trust + passkeys)

Hosting

• Application Hosting: EZSCALE's own infrastructure
• Database: MySQL 8.x with full redundancy (multi-region replication, automated backups, DR plan)
• Caching/Queue: Redis
• Email Service: Mailgun or SendGrid

3. Application Architecture

High-Level Architecture

┌──────────────────────────────────────────────────────────────────┐
│                        ezscale.cloud                              │
│          Marketing Site + Product Catalog (Laravel)               │
└──────────────────────────────────────────────────────────────────┘
                                 │
                    ┌────────────┴────────────┐
                    │                         │
┌───────────────────▼─────────┐  ┌───────────▼──────────────────────┐
│   account.ezscale.cloud      │  │    admin.ezscale.cloud           │
│   Customer Dashboard         │  │    Admin Panel                   │
│   (Vue 3 + Inertia.js)       │  │    (Cloudflare Zero Trust)       │
└───────────────┬──────────────┘  └──────────┬───────────────────────┘
                │                            │
┌───────────────┴────────────────────────────┴───────────────────────┐
│                        Laravel 12 Core                              │
│  ┌──────────┐ ┌──────────┐ ┌──────────────┐ ┌───────────────────┐ │
│  │   Auth   │ │ Billing  │ │ Provisioning │ │   Bandwidth       │ │
│  │ Fortify  │ │ Cashier  │ │   Services   │ │   Monitoring      │ │
│  │ Passport │ │ Stripe   │ │  VirtFusion  │ │   ElastiFlow      │ │
│  │          │ │ PayPal   │ │  Pterodactyl │ │   Integration     │ │
│  │          │ │          │ │  SynergyCP   │ │                   │ │
│  │          │ │          │ │  Enhance     │ │                   │ │
│  └──────────┘ └──────────┘ └──────────────┘ └───────────────────┘ │
│  ┌──────────┐ ┌──────────┐ ┌──────────────┐ ┌───────────────────┐ │
│  │ Tickets  │ │Analytics │ │   Customer   │ │   Admin Tools     │ │
│  │ System   │ │Dashboard │ │   API        │ │   Full Control    │ │
│  │Standalone│ │MRR/Churn │ │              │ │                   │ │
│  └──────────┘ └──────────┘ └──────────────┘ └───────────────────┘ │
├─────────────────────────────────────────────────────────────────────┤
│  MySQL 8.x (Multi-region)  │  Redis (Queue/Cache/Session)          │
└─────────────────────────────────────────────────────────────────────┘
         │                                    │
    ┌────┴────────┬──────────┬───────────┬───┴────┬─────────────┐
    │             │          │           │        │             │
VirtFusion  Pterodactyl  SynergyCP  Enhance  ElastiFlow
    API          API         API        API       API

4. Key Design Decisions

Payment & Billing (DECIDED)

• Primary Gateway: Stripe via Laravel Cashier v16 (~80%+ of customers)
• Secondary Gateway: PayPal via srmklive/laravel-paypal (~20% recurring subscriptions)
• Tax: To be decided (TaxJar/Avalara or manual rates)
• Multi-Currency: Full support (EUR, GBP, USD, etc.) via Stripe
• Invoicing: Full-featured PDF generation, custom numbering, multi-currency, email delivery
• Payment Methods: Customers can manage multiple payment methods per account
• Coupons: Full coupon system (percentage, fixed amount, trial extensions, usage limits, expiry)
• Billing Architecture: BillingServiceInterface abstracts Stripe and PayPal for gateway-agnostic code

Frontend & Auth (DECIDED)

• Stack: Vue 3 + Inertia.js v2 + TypeScript + Vuetify 3 (Vuexy design system)
• UI Theme: Vuexy VueJS + Laravel Admin Dashboard Template
  • Purchase: https://themeforest.net/item/vuexy-vuejs-html-laravel-admin-dashboard-template/23328599
  • Demo: https://pixinvent.com/vuexy-vuejs-laravel-admin-template/
  • Features: Pre-built admin components, charts, forms, tables, authentication pages, responsive design
  • Source files will be integrated when development begins
• Auth: Laravel Fortify (headless backend) + Vuexy-styled Vue/Inertia auth pages
• 2FA: Required for admins (passkeys preferred), encouraged for customers (TOTP/Authenticator)
• SSO: Laravel Passport OAuth2 server for future integrations
• API: RESTful API for both customers (service management) and admins

Provisioning (DECIDED - Fully Automated)

All service provisioning is fully automated via API on successful payment:

Service Type	Platform	Automation
VPS	VirtFusion API	Payment succeeds → API creates VPS → credentials emailed
Game Servers	Pterodactyl API	Payment succeeds → API creates server → credentials emailed
Dedicated Servers	SynergyCP API	Payment succeeds → API provisions (if hardware available) → credentials emailed
Web Hosting	Enhance API	Payment succeeds → API creates hosting account → credentials emailed

Note: Dedicated servers may require semi-automation (admin assigns hardware if inventory limited).

Panel Integration (DECIDED)

• Pterodactyl & SynergyCP: Separate login credentials (no SSO)
• Dashboard Integration: Full integration - Laravel dashboard shows:
  • Server status (online/offline)
  • Resource usage (CPU, RAM, disk, bandwidth)
  • Basic controls (restart, power on/off via API)
  • Links to panels for advanced management
• Enhance: Full integration for hosting account status and resource usage

Bandwidth Monitoring (DECIDED)

• Collection: NetFlow/sFlow via ElastiFlow (Elastic-based flow collector)
• Integration: Laravel queries ElastiFlow API/database for bandwidth data
• Display: Real-time bandwidth graphs in customer dashboard
• Billing: Automatic overage billing at end of billing cycle
• Alerts: Multi-threshold warnings (75%, 90%, 100% of quota) via email + Discord webhook
• No Add-ons: Automatic overage billing only (no one-time bandwidth add-ons)

Support Integration (DECIDED)

• System: Standalone ticket system (built-in, no external dependencies)
• Features:
  • Full ticket CRUD with replies for customers and admins
  • Email integration (IMAP polling via webklex/php-imap)
  • Ticket references: [EZSCALE-{id}] format with email threading
  • Departments, priorities, statuses
  • 42 Pest tests for ticket system
• Discord: Admin notifications via webhook (new orders, failures, cancellations, high revenue)

Customer Features (DECIDED)

• Dashboard Layout: Service overview (active services, next invoice, recent tickets, quick actions)
• Self-Service: Full self-service upgrades/downgrades with automatic proration
• Cancellation: Self-service cancellation with optional survey (service remains active until period end)
• Payment Methods: Manage multiple cards/PayPal accounts, set default per service

Admin Panel (DECIDED)

• Access Security: Cloudflare Zero Trust + 2FA (passkeys preferred)
• Control Level: Full control over all customer services
  • Provision, suspend, terminate, modify services
  • View/edit customer accounts
  • Invoice management, payment processing
  • Analytics dashboard (MRR, ARR, churn, revenue trends, popular plans)
• Suspension Policy: Automatic suspension X days past due → terminate Y days later

Analytics & Reporting (DECIDED)

• Comprehensive analytics dashboard:
  • Revenue trends (daily, monthly, yearly)
  • MRR (Monthly Recurring Revenue) and ARR (Annual Recurring Revenue)
  • Churn rate and customer growth
  • Plan popularity and conversion rates
  • Outstanding invoices and overdue accounts

Deployment & CI/CD (DECIDED)

• CI/CD: GitHub Actions pipeline
  • Tests run on push
  • Auto-deploy to staging environment
  • Manual approval for production deployment
• Environments: Full staging environment (staging.account.ezscale.cloud with separate database)

5. Database Schema (Core Tables)

Users & Auth

users
├── id, name, email, email_verified_at, password
├── status (active, suspended, banned)
├── phone, company
├── two_factor_secret, two_factor_recovery_codes
├── passkey_credentials (JSON for WebAuthn)
├── timestamps

user_profiles
├── id, user_id
├── billing_address (line1, line2, city, state, zip, country)
├── shipping_address (line1, line2, city, state, zip, country)
├── tax_id, tax_exempt
├── company_name, company_vat
├── notes (admin notes)
├── timestamps

roles (via spatie/laravel-permission)
├── id, name (admin, customer), guard_name, timestamps

permissions (via spatie/laravel-permission)
├── id, name, guard_name, timestamps

audit_logs
├── id, user_id, admin_id (nullable)
├── action (login, logout, service_provisioned, service_suspended, payment_failed, etc.)
├── resource_type, resource_id
├── ip_address, user_agent
├── changes (JSON - before/after state)
├── timestamps

Billing

plans
├── id, name, slug, description
├── service_type (vps, dedicated, hosting, game_server)
├── price, currency, billing_cycle (monthly, quarterly, annual, hourly)
├── stripe_price_id, paypal_plan_id
├── features (JSON - RAM, CPU, disk, bandwidth, etc.)
├── stock_quantity (nullable - for limited dedicated server inventory)
├── status (active, hidden, archived)
├── sort_order
├── timestamps

subscriptions (extended from Cashier)
├── id, user_id, plan_id
├── gateway (stripe, paypal)
├── gateway_subscription_id
├── gateway_customer_id
├── gateway_price_id
├── status (active, past_due, cancelled, paused, trialing)
├── trial_ends_at, current_period_start, current_period_end
├── cancelled_at, ends_at
├── timestamps

subscription_items (Cashier-managed)
├── id, subscription_id, stripe_id, stripe_product, stripe_price
├── quantity, timestamps

invoices (extended from Cashier)
├── id, subscription_id, user_id
├── gateway (stripe, paypal), gateway_invoice_id
├── number (custom invoice numbering)
├── total, tax, currency, status
├── invoice_pdf (path to generated PDF)
├── due_date, paid_at
├── timestamps

invoice_items
├── id, invoice_id
├── description, amount, quantity
├── timestamps

payment_transactions
├── id, user_id, subscription_id (nullable), invoice_id (nullable)
├── gateway (stripe, paypal), gateway_transaction_id
├── amount, currency, status (succeeded, failed, refunded, pending)
├── payment_method (card, paypal, bank_transfer)
├── description, metadata (JSON)
├── timestamps

coupons
├── id, code, type (percentage, fixed_amount)
├── value, currency (for fixed_amount)
├── applies_to (all, specific_plans - JSON array of plan IDs)
├── max_uses, times_used
├── expires_at
├── timestamps

coupon_redemptions
├── id, coupon_id, user_id, subscription_id
├── discount_amount
├── timestamps

Services & Provisioning

services
├── id, user_id, subscription_id, plan_id
├── service_type (vps, dedicated, hosting, game_server)
├── platform (virtfusion, pterodactyl, synergycp, enhance)
├── platform_service_id (ID in external system)
├── status (pending, active, suspended, terminated)
├── ipv4_address, ipv6_address
├── hostname, domain
├── credentials (encrypted JSON - username, password, access URLs)
├── provisioned_at, suspended_at, terminated_at
├── auto_renew (boolean)
├── timestamps

provisioning_logs
├── id, service_id, user_id
├── action (create, suspend, unsuspend, terminate, upgrade, downgrade)
├── platform, platform_response (JSON)
├── status (pending, success, failed)
├── error_message
├── admin_id (nullable - if manually triggered)
├── timestamps

bandwidth_usage
├── id, service_id
├── period_start, period_end
├── bytes_in, bytes_out, total_bytes
├── quota_bytes, overage_bytes
├── overage_charge
├── source (elastiflow)
├── timestamps

Support (Standalone Ticket System)

support_tickets
├── id, user_id
├── reference (e.g., EZSCALE-001)
├── subject, status (open, closed, pending, in_progress)
├── priority (low, medium, high, urgent)
├── department
├── last_reply_at
├── timestamps

ticket_replies
├── id, ticket_id, user_id
├── body (text)
├── is_staff_reply (boolean)
├── timestamps

6. Key Integrations

6.1 VirtFusion API (VPS Provisioning)

Endpoints needed:

• POST /api/vps/create - Create new VPS instance
• POST /api/vps/{id}/suspend - Suspend VPS
• POST /api/vps/{id}/unsuspend - Unsuspend VPS
• DELETE /api/vps/{id} - Terminate VPS
• GET /api/vps/{id}/status - Get VPS status and resource usage
• POST /api/vps/{id}/reboot - Reboot VPS

Provisioning Service: App\Services\Provisioning\VirtFusionService

6.2 Pterodactyl API (Game Server Provisioning)

Endpoints needed:

• POST /api/application/servers - Create server
• POST /api/application/servers/{id}/suspend - Suspend
• POST /api/application/servers/{id}/unsuspend - Unsuspend
• DELETE /api/application/servers/{id} - Delete
• GET /api/client/servers/{id}/resources - Resource usage

Provisioning Service: App\Services\Provisioning\PterodactylService

6.3 SynergyCP API (Dedicated Server Provisioning)

Endpoints needed:

• POST /api/server/provision - Provision dedicated server
• POST /api/server/{id}/suspend - Suspend
• POST /api/server/{id}/unsuspend - Unsuspend
• POST /api/server/{id}/terminate - Terminate
• GET /api/server/{id} - Get server details and status

Provisioning Service: App\Services\Provisioning\SynergyCPService

6.4 Enhance API (Web Hosting Provisioning)

Endpoints needed:

• POST /api/orgs/{org}/websites - Create hosting account
• PUT /api/orgs/{org}/websites/{id}/suspended - Suspend
• DELETE /api/orgs/{org}/websites/{id} - Delete
• GET /api/orgs/{org}/websites/{id} - Get account status

Provisioning Service: App\Services\Provisioning\EnhanceService

6.5 ElastiFlow API (Bandwidth Monitoring)

Integration approach:

• Query Elasticsearch indices via REST API
• Aggregate bandwidth by service IP address and time period
• Calculate usage per billing cycle
• Store results in bandwidth_usage table
• Scheduled job runs daily to update bandwidth stats

Service: App\Services\Monitoring\BandwidthService

6.6 Standalone Ticket System (Built-in)

No external integration needed. Tickets are managed natively:

• Customer and Admin controllers with full CRUD
• Email integration via IMAP polling (webklex/php-imap)
• Email threading with Message-ID, In-Reply-To, References headers
• Ticket reference format: [EZSCALE-{id}]
• Scheduled: tickets:process-emails runs every 2 minutes

6.7 Email Notifications (Mailgun/SendGrid)

Laravel Notifications for:

• Welcome email (account created)
• Email verification
• Invoice generated (with PDF attachment)
• Payment received (receipt)
• Payment failed (with retry instructions)
• Service provisioned (credentials)
• Service suspended (past due payment)
• Service expiring soon (renewal reminder)
• Bandwidth usage alerts (75%, 90%, 100%)
• Bandwidth overage charge (end of cycle)
• Ticket updates (new reply, status change)

6.8 Discord Webhooks (Admin Alerts)

Notifications sent to Discord:

• New customer signup
• New order / subscription created
• High-value order (e.g., >$500)
• Payment failure
• Service suspension
• Service cancellation
• System errors / provisioning failures
• Security alerts (admin login failures, etc.)

7. Security Architecture

Authentication

• Customer Auth: Email + password (bcrypt), optional 2FA (TOTP), passkey support (WebAuthn)
• Admin Auth: Email + password + required 2FA (passkeys preferred), behind Cloudflare Zero Trust
• Session Management: Redis-backed sessions with secure cookies
• API Auth: Laravel Sanctum tokens for customer/admin APIs

Authorization

• Roles: admin and customer via spatie/laravel-permission
• Policies: Laravel policies for resource-level authorization (can user view this service?)
• Admin Actions: All admin actions logged to audit_logs table
• Admin Impersonation: Admins can impersonate customers (with audit logging)

Data Protection

• Encryption at Rest: Service credentials, API keys, payment tokens encrypted in database
• HTTPS Everywhere: All traffic HTTPS-only, HSTS headers
• CSRF Protection: All forms CSRF-protected
• Rate Limiting: Login, registration, API endpoints rate-limited
• Input Validation: Laravel Form Request classes for all inputs
• SQL Injection Prevention: Eloquent ORM + prepared statements
• XSS Prevention: Blade/Vue escaping, CSP headers

Admin Panel Security

• Cloudflare Zero Trust: Admin panel behind Zero Trust access control
• Passkeys/2FA: Required for all admin accounts
• IP Logging: All admin actions logged with IP address
• Suspicious Activity Alerts: Discord webhook for failed admin logins

8. Development Phases

Phase 1: Foundation & Core Setup

• Initialize Laravel 12 with Vue + Inertia starter kit ✓ (base install in website/)
• Configure domains: ezscale.cloud, account.ezscale.cloud, admin.ezscale.cloud
• Set up MySQL with replication, Redis
• Install dependencies: Cashier, Fortify, Passport, PayPal, Spatie
• Create database schema and migrations
• Set up authentication (Fortify + 2FA + passkeys)
• Configure Cloudflare Zero Trust for admin panel
• Set up CI/CD pipeline (GitHub Actions)

Phase 2: Billing & Subscriptions

• Implement BillingServiceInterface (Stripe + PayPal)
• Build plan catalog (VPS, Dedicated, Hosting, Game Server plans)
• Subscription creation and management
• Invoice generation (PDF, custom numbering, multi-currency)
• Payment method management (multiple cards/PayPal per customer)
• Coupon system (creation, redemption, validation)
• Webhook handlers for Stripe and PayPal
• Automatic overage billing for bandwidth
• Tax calculation (TaxJar/Avalara or manual)

Phase 3: Provisioning Automation

• ProvisioningServiceInterface abstraction
• VirtFusion provisioning service
• Pterodactyl provisioning service
• SynergyCP provisioning service
• Enhance provisioning service
• Event-driven provisioning (listen to payment success events)
• Credential generation and secure storage
• Provisioning failure handling and retry logic

Phase 4: Customer Dashboard (account.ezscale.cloud)

• Service overview dashboard
• Active services list with status and resource usage
• Bandwidth usage graphs (from ElastiFlow)
• Billing history and invoices (PDF download)
• Payment methods management
• Plan upgrades/downgrades (self-service)
• Subscription cancellation (with survey)
• Profile and account settings
• 2FA and passkey setup

Phase 5: Admin Panel (admin.ezscale.cloud)

• Analytics dashboard (MRR, ARR, churn, revenue trends)
• Customer management (view, edit, impersonate, notes)
• Service management (view all services, suspend, terminate, modify)
• Order management (pending orders, approvals)
• Invoice management (view, edit, manual invoicing)
• Coupon management (create, edit, deactivate)
• Plan management (create, edit, pricing changes)
• System configuration (email templates, tax rates, suspension policies)
• Audit log viewer

Phase 6: Bandwidth Monitoring & Billing

• ElastiFlow API integration
• Bandwidth data collection and storage
• Usage calculation per billing cycle
• Bandwidth graphs in customer dashboard
• Multi-threshold alerts (75%, 90%, 100%)
• Automatic overage billing
• Admin bandwidth reports

Phase 7: Support Ticket System ✓

• Standalone ticket system with TicketReply model (no external dependencies)
• Customer and admin Vue pages (5 pages total)
• Email integration via IMAP polling (webklex/php-imap)
• Email threading with ticket references [EZSCALE-{id}]
• 42 Pest tests

Phase 8: Marketing Frontend (ezscale.cloud)

• Product catalog pages (VPS, Dedicated, Hosting, Game Servers)
• Pricing pages with plan comparison
• Signup flow with plan selection
• Blog/news section
• Knowledge base / FAQ
• Legal pages (Terms, Privacy, AUP)

Phase 9: Testing, Migration & Launch

• Comprehensive testing (unit, feature, integration, E2E)
• Security audit and penetration testing
• WHMCS data export and migration scripts
• Customer data migration (users, subscriptions, services, invoices)
• Parallel operation period (run both systems)
• Full cutover to new platform
• Monitoring and alerting setup
• Post-launch support and bug fixes

9. WHMCS Migration Plan

Migration Scope (DECIDED)

• Full historical data migration:
  • All customer accounts and profiles
  • All subscriptions (active and cancelled)
  • Complete invoice history
  • Complete payment history
  • All support tickets (migrate to SupportPal or archive)
  • Service configurations
  • Product/plan mapping

Migration Approach

• Customer Volume: Medium (100-1000 customers)
• Strategy: Automated migration scripts with manual verification
• Staging: Test migration in staging environment first
• Parallel Operation: Run WHMCS and new platform in parallel for 30 days
• Cutover: Maintenance window for final cutover, redirect WHMCS URLs

Migration Steps

1. Export WHMCS database to SQL dump
2. Map WHMCS product IDs to new plan IDs
3. Create migration scripts (Laravel commands):
   • php artisan migrate:whmcs-customers - Import customers and profiles
   • php artisan migrate:whmcs-subscriptions - Import active subscriptions (coordinate with Stripe/PayPal)
   • php artisan migrate:whmcs-invoices - Import invoice history
   • php artisan migrate:whmcs-payments - Import payment history
   • php artisan migrate:whmcs-services - Import service configurations
   • php artisan migrate:whmcs-tickets - Migrate tickets to SupportPal
4. Verify data integrity in staging
5. Email customers about platform migration
6. Set up redirects from old WHMCS URLs
7. Execute final migration during maintenance window
8. Monitor for issues during parallel operation period

10. Open Questions & Decisions Needed

Decided ✓

• Payment gateway: Stripe + PayPal
• Frontend stack: Vue 3 + Inertia.js
• Infrastructure: VirtFusion, Pterodactyl, SynergyCP, Enhance
• Bandwidth monitoring: ElastiFlow (NetFlow/sFlow)
• Support system: Standalone ticket system (built-in)
• Domain structure: ezscale.cloud / account / admin
• Hosting: Own infrastructure with full DB redundancy
• CI/CD: GitHub Actions with staging environment
• Email: Mailgun or SendGrid
• Admin security: Cloudflare Zero Trust + passkeys
• Customer features: Full self-service (upgrades, downgrades, cancellations)
• Provisioning: Fully automated for all service types
• Analytics: Comprehensive dashboard (MRR, ARR, churn, etc.)
• Multi-currency: Yes (Stripe supports this)
• Coupons: Full system with all features
• WHMCS migration: Full historical data import

Still To Decide

• Tax calculation approach: TaxJar/Avalara integration vs manual tax rates?
• Email service final choice: Mailgun or SendGrid?
• Admin panel subdomain: admin.ezscale.cloud or something less obvious for security?
• NetFlow/sFlow deployment: Timeline for switching Juniper to flow exports?
• Customer portal theme/branding DECIDED: Vuexy VueJS + Laravel Admin Dashboard Template

11. Tech Stack Summary

Layer	Technology
Framework	Laravel 12 (PHP 8.3)
Frontend	Vue 3 + Inertia.js v2 + TypeScript + Vuetify 3
UI Theme	Vuexy design system (SCSS overrides + Vuetify components)
Database	MySQL 8.x (multi-region replication)
Cache/Queue	Redis
Payments	Laravel Cashier Stripe v16 + srmklive/laravel-paypal
Auth	Laravel Fortify + Passport (OAuth2)
Roles	spatie/laravel-permission
Email	Mailgun or SendGrid (via Laravel Mail)
2FA	TOTP (Authenticator apps) + WebAuthn (passkeys)
Admin Security	Cloudflare Zero Trust
CI/CD	GitHub Actions
Monitoring	ElastiFlow (bandwidth), Laravel Telescope (debugging)
Provisioning APIs	VirtFusion, Pterodactyl, SynergyCP, Enhance
Support	Standalone ticket system (built-in)
Notifications	Laravel Notifications + Discord webhooks