BACKEND DEVELOPMENTCOMPLETE REFERENCE

Design Principles

Coding Standards

Follow PascalCase for public members and camelCase for private fields. Use clear, descriptive names that express intent.

public class OrderProcessor
{
  private List _orders; // camelCase
  public void ProcessOrder(Order order) { } // PascalCase
  private bool ValidatePayment(Payment payment) { }
}

Use snake_case for variables and functions, UPPER_CASE for constants. 4-space indentation. Max line length 79 chars (PEP 8) or 88 (Black).

def process_order(order: Order) -> bool:
    TIMEOUT_SECONDS = 30
    valid_items = validate_items(order.items)
    return valid_items and order.total > 0

Use snake_case for variables, functions, and modules. PascalCase for types and constants. Follow Rust API guidelines for clarity.

pub struct OrderProcessor {
    active_orders: Vec,
}

pub fn process_order(order: Order) -> Result<(), Error> {
    let ORDER_LIMIT = 1000;
    Ok(())
}

One public class per file. Organize members: public methods, properties, private methods. Use namespaces to organize by feature or domain.

namespace Domain.Orders
{
  public class Order
  {
    public int Id { get; set; }
    public List Items { get; set; }
    
    public decimal GetTotal() => Items.Sum(i => i.Price);
    private void ValidateItems() { }
  }
}

Standard structure: src/package_name for code, tests/ for tests, setup.py/pyproject.toml for config. Use __init__.py for packages.

project/
├── src/myapp/
│   ├── __init__.py
│   ├── models.py
│   └── services.py
├── tests/
│   └── test_models.py
├── pyproject.toml
└── README.md

Use workspaces for multi-crate projects. Separate library crates from binary crates. Clear module hierarchy with mod.rs.

[workspace]
members = ["crates/api", "crates/domain"]

// src/lib.rs
pub mod domain;
pub mod services;

// domain/mod.rs
pub struct Order { }
pub mod validators;

Paradigms

Organize code around objects with state and behavior. Emphasizes encapsulation, inheritance, and polymorphism. Natural for modeling real-world entities.

Treat computation as the evaluation of pure functions. Emphasizes immutability, higher-order functions, and avoiding side effects. Easier reasoning and testing.

Build responsive systems using observable streams and event-driven architecture. Components react to data changes. Good for handling async operations.

Organize code as a sequence of procedures/functions that modify state. Direct control flow. Good for straightforward, step-by-step logic.

Specify what you want rather than how to achieve it. Let the system figure out the steps. Examples: SQL, configuration-based systems.

Concurrent systems based on independent actors communicating via messages. Avoids shared state and race conditions. Excellent for distributed systems.

Architectural Patterns

Concentric layers separating concerns: Entities, Use Cases, Interface Adapters, Frameworks. Dependencies point inward. Highly testable and decoupled.

Ports and Adapters. Core business logic at center with external dependencies as adapters. Enables multiple entry/exit points without changing core logic.

Independently deployable services built around business capabilities. Each service has its own database. Scaled and developed independently. Increased complexity.

Single deployment unit with clear module boundaries. Modules communicate through well-defined interfaces. Balances simplicity of monolith with organization of microservices.

Align software structure with business domain. Use ubiquitous language shared between developers and domain experts. Organize around bounded contexts.

Components communicate through events. Decoupled producers and consumers. Excellent for real-time systems and maintaining eventual consistency across services.

Business capabilities exposed as reusable services. Heavy use of messaging and protocols. Enterprise-level approach, often with ESB and SOAP.

Run code without managing servers. Scale automatically. Pay per execution. Good for event-driven workloads and variable traffic patterns.

Design Patterns

Enterprise Patterns

Abstract data access logic behind a repository interface. Clients interact with repositories instead of directly accessing data. Enables switching persistence mechanisms.

public interface IOrderRepository {
  Task GetByIdAsync(int id);
  Task> GetAllAsync();
  Task SaveAsync(Order order);
}

public class OrderRepository : IOrderRepository {
  private readonly DbContext _db;
  public Task GetByIdAsync(int id) => _db.Orders.FindAsync(id);
}

Maintain a collection of objects affected by a business transaction and coordinates writing changes and handling concurrency. Ensures all changes are atomic.

public interface IUnitOfWork {
  IOrderRepository Orders { get; }
  ICustomerRepository Customers { get; }
  Task SaveChangesAsync();
}

var order = await _uow.Orders.GetByIdAsync(1);
order.Status = OrderStatus.Shipped;
await _uow.SaveChangesAsync();

Encapsulate query logic in reusable specification objects. Separate complex filtering rules from repositories. Makes queries composable and testable.

public class ActiveOrdersSpec : Specification {
  public ActiveOrdersSpec() {
    AddCriteria(o => o.Status == OrderStatus.Active);
    AddInclude(o => o.Items);
  }
}

var orders = await repo.GetAsync(new ActiveOrdersSpec());

Create a layer that translates between external system contracts and your domain model. Isolates domain from changes in external systems.

public class LegacyOrderAdapter {
  public Order AdaptFromLegacy(LegacyOrderDto dto) {
    return new Order {
      Id = dto.OrderId,
      Status = MapLegacyStatus(dto.Status),
      Items = dto.Items.Select(AdaptItem).ToList()
    };
  }
}

Maintain denormalized, query-optimized views of data separate from write models. Each read model is optimized for specific queries. Common in CQRS.

Coordinate long-running, distributed transactions across multiple services. Sagas are event-driven; Process Managers route messages. Handle compensating transactions for failures.

Data Handling

Map objects to database tables. Handle hydration and persistence automatically. Examples: Entity Framework, SQLAlchemy, SeaORM. Trade flexibility for convenience.

Write SQL with less boilerplate through fluent APIs. Dapper, Linq2Db (C#), SQLAlchemy Core (Python), sqlx (Rust). More control than ORMs, less verbose than raw SQL.

var orders = await connection.QueryAsync(
  "SELECT * FROM Orders WHERE CustomerId = @CustomerId",
  new { CustomerId = 123 }
);

Version control your schema changes. Migrations are versioned scripts or code that evolve schema over time. Reversible and trackable. Examples: Flyway, EF Migrations, Alembic.

public class AddOrderTableMigration : Migration {
  public override void Up() {
    CreateTable("Orders")
      .WithColumn("Id").AsInt32().PrimaryKey()
      .WithColumn("CustomerId").AsInt32()
      .WithColumn("Total").AsDecimal();
  }
}

Reduce database hits with in-memory or distributed caches. Cache-aside, write-through, write-behind patterns. Know cache invalidation strategies.

Denormalized views optimized for specific queries. Kept in sync with write model via events. Critical for read-heavy workloads and analytics.

Full-text search and analytics via Elasticsearch, Solr, or similar. Index documents for fast, flexible querying beyond SQL capabilities.

CQRS & Event Sourcing

Client sends a command or query to the API endpoint.

Receives request, validates input, routes to command/query handler.

Cross-cutting concerns: logging, validation, authorization, error handling.

Processes command, validates business rules, emits domain events. Single source of truth.

Persists events transactionally. Ensures no loss. Published asynchronously via message broker.

Subscribe to events, build denormalized read models for fast queries.

Database Patterns

Each microservice owns its database schema. Enables independent scaling and schema evolution. Requires eventual consistency across services.

Multiple services share a database schema. Simpler to query across services but tight coupling. Hard to scale independently.

Serve multiple tenants in a single database. Shared or separate schemas. Requires careful data isolation and querying. Cost-effective but complex.

Replicate data to read-only copies for scaling read workloads. Asynchronous replication. Introduces eventual consistency window.

Partition data across multiple database instances by a shard key (e.g., customer_id). Horizontal scaling for write-heavy workloads.

Separate read and write databases. Writes go to normalized write DB, reads from optimized read DB. Projections keep them in sync.

Capture database changes as a stream of events. Trigger downstream processes, update caches, replicate to other systems.

Mark records as deleted with a timestamp rather than removing them. Preserves history and allows recovery. Add filter to queries.

Database-native versioning. Track row changes over time. Queries can see historical versions. Available in SQL Server, PostgreSQL.

Use version numbers or timestamps instead of locks. Last writer wins after checking version hasn't changed. Good for distributed systems.

API Patterns

Resource-oriented APIs using HTTP verbs (GET, POST, PUT, DELETE). Stateless, cacheable, standardized. Most common pattern. Status codes indicate outcome.

POST /api/orders
GET /api/orders/123
PUT /api/orders/123
DELETE /api/orders/123

Query language for APIs. Clients request exact data needed, avoiding over-fetching. Strong typing, introspection, aggregates data from multiple sources.

query {
  order(id: 123) {
    id
    customer { name email }
    items { sku quantity }
  }
}

High-performance RPC using HTTP/2 and Protocol Buffers. Typed contracts, streaming, multiplexing. Excellent for service-to-service communication.

Real-time bidirectional (WebSocket) or server-to-client (SSE) communication. Avoid polling. Good for notifications and live updates.

Light-weight, convention-free API definition. Map HTTP endpoints directly to handler functions. Less ceremony than controllers. Common in .NET 6+.

Single entry point for clients. Routes requests, handles authentication, rate limiting, response transformation. Decouples clients from backend services.

Messaging & Events

Publishers send messages to a topic; subscribers receive copies. Decouples producers from consumers. Examples: Azure Service Bus Topics, AWS SNS, RabbitMQ Topic Exchange.

FIFO delivery of messages to a queue. Single consumer per message. Guarantees order and delivery. Examples: Azure Queue Storage, RabbitMQ Queues, SQS.

Append-only log of events. Consumers read from any offset. Replay-able history. Examples: Kafka, Pulsar, Azure Event Hubs.

Messages that fail processing multiple times are moved to a DLQ for investigation. Prevents poison pill messages from blocking queue.

Multiple instances consume from same queue in parallel. Each message processed once. Scale processing by adding consumers.

Sender publishes request on a queue, receiver processes and replies on reply queue. Synchronous semantics over async transport.

Large message payload moved to blob storage; message contains reference/claim check. Reduces message size and bus load.

Services communicate directly via events. Each service knows what events trigger its actions. Distributed decision-making. Can get tangled.

Central orchestrator/saga manager directs the flow. Services do not know about each other. Clearer flow but central point of failure.

Concurrency

Non-blocking operations that free up threads while waiting. Efficient use of thread pool. Syntactically simple with async/await keywords. Standard pattern.

public async Task GetOrderAsync(int id) {
  var order = await _db.Orders.FindAsync(id);
  var items = await _db.OrderItems
    .Where(i => i.OrderId == id)
    .ToListAsync();
  return order;
}

Mutual exclusion for accessing shared resources. Mutex (binary), Lock (reentrant), Semaphore (count). Risk of deadlock. Use sparingly in async code.

Thread-safe communication between concurrent tasks/goroutines. Producer-consumer pattern. Typed, built-in cancellation. Rust: mpsc channels, C#: System.Threading.Channels.

Automatically manage thread creation and reuse. Task Parallel Library (C#) coordinates work. Avoids manual thread creation and context switching overhead.

Coalesce multiple identical concurrent requests into a single operation. Return same result to all requesters. Avoid thundering herd.

Compile-time concurrency safety. Ownership rules prevent data races. Move vs borrow semantics. No need for locks on immutable shared data.

Testing

Test individual functions/methods in isolation with mocks. Fast, fine-grained feedback. Cover happy paths and edge cases.

Test components working together (database, external services). Use test doubles or in-memory databases. Slower but more realistic.

Verify API contracts between services without full E2E testing. Consumer-driven contracts. Catch integration issues early.

Generate random inputs and verify invariants hold. Examples: QuickCheck, Hypothesis. Find edge cases you didn't think of.

Capture output (JSON, HTML) and verify against stored snapshots. Good for UI, configuration, complex data structures.

Verify code structure adheres to design rules. ArchUnit (Java), NArchitecture (C#). Enforce layering, dependencies, naming conventions.

Simulate realistic load. Measure throughput, latency, resource usage. Find bottlenecks. Examples: K6, JMeter, Apache Bench.

Inject small code changes and verify tests catch them. Low mutation score indicates insufficient test coverage.

Error Handling

Return Result<T, E> or Either<L, R> instead of throwing exceptions. Explicit error handling. Composable with map/flatMap. Rust Result, Scala Either.

pub fn process_order(order: Order) -> Result {
  let payment = process_payment(&order)?;
  let confirmation = create_confirmation(&order, &payment)?;
  Ok(confirmation)
}

Create domain-specific exception classes inheriting from base. Catch specific exceptions. Avoid catching generic Exception.

public class OrderException : Exception { }
public class InsufficientFundsException : OrderException { }
public class ItemOutOfStockException : OrderException { }

Exponential or fixed backoff for transient failures. Jitter to avoid thundering herd. Max retries to avoid infinite loops. Use Polly (C#), tenacity (Python).

var policy = Policy
  .Handle()
  .WaitAndRetry(new[] {
    TimeSpan.FromSeconds(1),
    TimeSpan.FromSeconds(2),
    TimeSpan.FromSeconds(4)
  });
await policy.ExecuteAsync(() => client.GetAsync(url));

Prevent cascading failures by stopping requests to failing service. Open → Half-Open → Closed states. Polly, Hystrix, or cloud SDK.

Security

Delegate authentication to identity provider. OAuth 2.0 for authorization, OIDC adds identity. Authorization Code flow for web, PKCE for SPAs.

Stateless authentication with JWTs. Short-lived access tokens + long-lived refresh tokens. No session storage needed. Verify signature server-side.

Role-Based (RBAC) or Attribute-Based (ABAC) access control. RBAC: users have roles with permissions. ABAC: fine-grained rules on user/resource attributes.

Simple token-based auth for service-to-service or public APIs. Stateless, easy to revoke. Send in header or query. Less secure than OAuth.

Validate all input at API boundaries. Type, length, format, range. Use libraries like FluentValidation (C#), Pydantic (Python). Prevent injection attacks.

Use parameterized statements to prevent SQL injection. Never concatenate user input into SQL. All major ORMs and drivers support this.

Store secrets (API keys, passwords, connection strings) in secure vaults, not code. Azure Key Vault, AWS Secrets Manager, HashiCorp Vault.

Control which origins can access API. Specify allowed origins, methods, headers. Prevents unauthorized cross-site requests.

Limit requests per IP/user/key to prevent abuse and DoS. Use token bucket or sliding window algorithm. Return 429 Too Many Requests.

Never trust, always verify. Authenticate and authorize every request/connection. Encrypt in transit and at rest. Assume breach mentality.

API operations should be safe to retry. Use idempotency keys to prevent duplicate processing. Critical for payment systems.

Log all security events: authentication, authorization, data access, changes. Immutable logs for compliance and investigation.

Observability

Log as JSON with consistent fields (timestamp, level, trace-id, user-id, message). Query and aggregate logs by field. Tools: ELK, Loki, CloudWatch.

{
  "timestamp": "2024-01-15T10:30:00Z",
  "level": "INFO",
  "trace_id": "abc123",
  "user_id": 456,
  "message": "Order created",
  "order_id": 789
}

Quantitative measurements: request rate, latency, error rate, queue depth. Emit via Prometheus, Datadog, or cloud provider. Alert on thresholds.

Track request flow across services using trace and span IDs. Visualize latency bottlenecks. Tools: Jaeger, Zipkin, Datadog APM.

Expose /health endpoint returning service health status. Check dependencies (database, cache). Used for load balancing and orchestration.

Set up alerts on metrics, logs, and traces. Alert on anomalies and thresholds. Route to on-call via PagerDuty, OpsGenie, etc.

Performance

In-memory (Redis, Memcached), application-level, HTTP caching. Cache-aside pattern. Set appropriate TTLs. Handle invalidation carefully.

Return large result sets in pages. Cursor-based pagination scales better than offset. Include total count carefully (can be expensive).

Group operations to reduce round trips. Bulk insert/update queries. Batch message publishing. Trade latency for throughput.

Add indexes on frequently queried columns. Watch N+1 queries. Analyze query plans. Composite indexes for multi-column filters.

Use async I/O for network calls, file access, database queries. Free up threads. Apply backpressure to avoid overload.

Reuse database/HTTP connections via pools. Reduce connection overhead. Configure pool size based on workload.

Deployment

Run two identical production environments. Switch traffic from blue to green. Zero downtime. Instant rollback by switching back.

Gradually shift traffic to new version. Start with 5%, monitor, increase to 50%, then 100%. Catch issues with partial rollout.

Toggle features on/off at runtime without redeploy. A/B test, gradual rollout, kill switches. LaunchDarkly, Unleash, or custom.

Gradually replace old instances with new version. Requires load balancer and health checks. Brief period with mixed versions.

Build versioned machine images (AMI, Docker image). Deploy by replacing entire instances, never modify in-place. Reproducible and reliable.

Git repo as source of truth for infrastructure and applications. Declarative config, automated sync. Tools: ArgoCD, Flux.

Automated checks before deployment: tests pass, security scan clears, performance benchmarks acceptable. Prevent bad deployments.

Plan schema changes to work with both old and new code. Deploy code before schema or vice versa. Use feature flags to control behavior.

.NET/C# Specific

Built-in DI container in .NET. Register services with IServiceCollection. Constructor injection. Transient, scoped, singleton lifetimes.

services.AddScoped();
services.AddSingleton();
services.AddTransient();

Request flows through middleware components. Each adds behavior (logging, auth, error handling). Order matters. Use app.Use() to add custom middleware.

app.UseAuthentication();
app.UseAuthorization();
app.UseCustomErrorHandling();
app.MapControllers();

Fluent API for defining validation rules. Separate validators from models. Composable, reusable, testable. Integrates with Minimal APIs.

public class CreateOrderValidator : AbstractValidator {
  public CreateOrderValidator() {
    RuleFor(x => x.CustomerId).NotEmpty();
    RuleFor(x => x.Items).NotEmpty().WithMessage("Must have items");
  }
}

Convention-based object mapping. DTO ↔ Domain models. Reduce boilerplate. Mapster is faster than AutoMapper.

IHostedService for long-running tasks. Worker services run continuously. Scoped dependency injection for each iteration.

Organize by feature, not layer. Each feature has command, handler, validator, projections. Reduces cross-feature coupling.

Python Specific

Modern, fast web framework for building APIs. Automatic OpenAPI/Swagger docs. Built on Starlette and Pydantic. Type hints drive validation and docs.

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class Order(BaseModel):
    id: int
    total: float

@app.get("/orders/{order_id}")
async def get_order(order_id: int) -> Order:
    return Order(id=order_id, total=99.99)

Data validation and parsing using Python type hints. Automatic schema generation. Serialization/deserialization. Settings management.

from pydantic import BaseModel, Field, validator

class Order(BaseModel):
    id: int = Field(..., gt=0)
    total: float = Field(..., gt=0)
    
    @validator("total")
    def validate_total(cls, v):
        if v > 1000000:
            raise ValueError("Total too large")
        return v

SQL toolkit and Object-Relational Mapper. Modern API with type hints. SQLAlchemy 2.0+ emphasizes async and explicit SQL.

Use dataclasses/Pydantic for DI. Dependency() in FastAPI for request scope. Or use libraries like python-dependency-injector.

asyncio for concurrency. async/await syntax. Asyncio.gather() for concurrent tasks. Event loops and tasks. Understanding asyncio is critical.

dataclasses for lightweight data structures. Protocols (structural typing) for flexible interfaces. TypedDict for dict typing.

Rust Specific

Async runtime for Rust. Tokio::spawn for tasks, channels for communication, select! for multiplexing. Production-grade for building servers.

use tokio::task;

#[tokio::main]
async fn main() {
    let task1 = task::spawn(async { 1 + 1 });
    let result = task1.await.unwrap();
    println!("Result: {}", result);
}

Modern, ergonomic web framework built on Tokio. Type-safe extractors. Clean routing DSL. Router composition. Built-in middleware.

use axum::{routing::get, Router};

#[tokio::main]
async fn main() {
    let app = Router::new()
        .route("/orders/:id", get(get_order));
    
    axum::Server::bind(&"0.0.0.0:3000".parse()?)
        .serve(app.into_make_service())
        .await?
}

sqlx: compile-time checked SQL queries. SeaORM: async-first ORM. Both leverage Rust type system and async/await.

Result<T, E> type. Custom error enums. thiserror/anyhow crates for ergonomic error handling. Compile-time enforcement of error handling.

pub fn process_order(order: Order) -> Result<(), OrderError> {
    validate_order(&order)?;
    let payment = charge_payment(&order)?;
    Ok(())
}

Use traits for abstractions. Trait objects for dynamic dispatch. Blanket implementations for generics. Natural for DI and mocking.

Serialize/deserialize Rust data structures to/from JSON, TOML, YAML, etc. Derive macros make it easy. Type-safe and performant.

Caching Strategies

Application checks the cache first. On a miss, it queries the database, then writes the result back to the cache. Most common pattern — simple, flexible, and easy to reason about.

Read-through: cache auto-fetches from DB on miss. Write-through: every write goes to cache AND DB synchronously. Provides strong consistency between cache and store.

Writes go to cache immediately, then asynchronously flushed to the database in batches. Great for write-heavy workloads but introduces durability risk if cache crashes before flush.

Refresh-ahead: proactively refresh entries before TTL expiry based on access patterns. Cache warming: pre-populate cache on startup or deploy to avoid cold-start stampede.

HTTP layer: ETag, Last-Modified, Cache-Control headers. CDN layer: geo-distributed edge caching (CloudFront, Cloudflare). Reverse proxy: Varnish, Nginx cache. Reduces origin load dramatically.

Cache-Control: public, max-age=3600, stale-while-revalidate=60
ETag: "abc123"
Vary: Accept-Encoding

TTL: automatic time-based expiration. Eviction policies: LRU (Least Recently Used), LFU (Least Frequently Used), FIFO, MRU. Choose based on access patterns and memory constraints.

Redis Cluster, Hazelcast, Memcached — shared cache across all application nodes. Consistent hashing for key distribution. Replication for HA. Watch for thundering herd on eviction.

// Redis cluster example
var cache = ConnectionMultiplexer.Connect("redis-cluster:6379");
var db = cache.GetDatabase();
await db.StringSetAsync("order:123", json, TimeSpan.FromMinutes(15));

The hardest problem. Strategies: manual purge, event-driven invalidation (publish on write), surrogate-key / tag-based (Fastly, Varnish), versioned keys. Never rely on TTL alone for critical data.

In-process memoization (dictionary, Caffeine, functools.lru_cache). DB query cache: statement-level result caching. Fast but per-instance — not shared across nodes.

from functools import lru_cache

@lru_cache(maxsize=256)
def get_product(product_id: int) -> Product:
    return db.query(Product).get(product_id)

Varnish, Nginx, HAProxy — cache HTTP responses at the proxy layer before hitting your app. VCL rules for fine-grained control. Combine with CDN for multi-tier caching.

Distributed Systems

Strong (linearizability): every read sees the most recent write. Eventual: temporary divergence allowed, converges over time. BASE (Basically Available, Soft-state, Eventually consistent) is the alternative to ACID.

ACID: Atomicity, Consistency, Isolation, Durability — single DB transactions. BASE: Basically Available, Soft-state, Eventually consistent — distributed systems trade-off. Choose based on business requirements.

Redis Redlock, ZooKeeper, etcd — lease-based locks across nodes. Fencing tokens to prevent stale lock holders. Always set TTL to avoid deadlocks. Be cautious — distributed locking is notoriously hard.

Etcd, Consul, Kubernetes leases — one node elected leader for coordination. Raft consensus algorithm underneath. Watch for split-brain. Use lease renewal with heartbeats.

Raft: understandable leader-based consensus. Paxos: theoretical foundation. ZAB (ZooKeeper Atomic Broadcast). All solve the problem of agreeing on a single value across unreliable nodes.

Consul, Eureka, Kubernetes DNS — how services find each other. Client-side vs server-side discovery. Health-check integrated. DNS-based (simple) vs API-based (richer metadata).

Distributed transaction coordinator. Phase 1: prepare (all participants vote). Phase 2: commit/abort. Blocking protocol — coordinator failure blocks all participants. Use Saga pattern instead where possible.

Read Uncommitted → Read Committed → Repeatable Read → Serializable. Higher isolation = fewer anomalies but more contention. Snapshot isolation (MVCC) is the sweet spot for most workloads.

Conflict-free Replicated Data Types — data structures that automatically merge concurrent updates without coordination. G-Counter, PN-Counter, LWW-Register, OR-Set. Used in collaborative editing and offline-first apps.

Causal consistency: respects happens-before relationships — if A caused B, everyone sees A before B. Bounded staleness: reads lag by at most N versions or T seconds. Cosmos DB offers 5 consistency levels.

Resilience Patterns

Set request, connection, and per-operation timeouts. Propagate deadlines across service calls so downstream services know the remaining budget. Without timeouts, one slow service cascades to all callers.

var policy = Policy.TimeoutAsync(
    TimeSpan.FromSeconds(5),
    TimeoutStrategy.Pessimistic);

Isolate resources (thread pools, connection pools, semaphores) per dependency so a failure in one doesn't exhaust resources needed by others. Named after ship bulkheads that contain flooding.

When a dependency fails: return cached data, a default value, or a degraded response. Disable non-critical features under stress. The user gets a partial experience rather than an error.

On SIGTERM: stop accepting new requests, drain in-flight requests, close connections, flush logs. Kubernetes gives 30s by default. Use preStop hooks for zero-downtime deployments.

app.Lifetime.ApplicationStopping.Register(() =>
{
    Log.Information("Shutting down...");
    // Drain in-flight work
});

Intentionally inject failures to discover weaknesses. Tools: Gremlin, Chaos Mesh, Litmus. Start small — latency injection, then pod kills, then network partitions. Run in staging first, then production.

Active-passive: standby takes over on failure. Active-active: load shared, automatic redistribution. Kubernetes reconciliation: desired state vs actual state, auto-restart on probe failure.

Automated backups with defined RPO (Recovery Point Objective). Cross-region replication for DR. Test restores regularly — an untested backup is not a backup.

Kubernetes startup probe: delays liveness/readiness checks for slow-starting services (JVM warm-up, large model loading). Prevents premature pod kills during initialization.

Scalability Patterns

Horizontal (scale out): add more nodes — requires stateless design and load balancing. Vertical (scale up): bigger machine — simpler but has ceiling. Horizontal is the cloud-native default.

Round-robin: simple rotation. Least-connections: route to least-busy. IP-hash: sticky by source. Consistent hashing: minimal redistribution on node change. L4 (TCP) vs L7 (HTTP) load balancing.

Kubernetes HPA (Horizontal Pod Autoscaler): scale on CPU/memory/custom metrics. Cluster autoscaler: add/remove nodes. Schedule-based: pre-scale for known traffic patterns.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
  minReplicas: 2
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        averageUtilization: 70

Global Server Load Balancing: DNS-based geographic routing to nearest datacenter. CDN: edge-cache static assets and API responses at 200+ PoPs worldwide. Together they minimize latency globally.

Gzip: universally supported, good compression. Brotli: 15-25% smaller than gzip for text. Enable at reverse proxy or application level. Use for HTML, JSON, CSS, JS — skip for already-compressed formats.

HTTP/2: multiplexing (multiple streams over one connection), header compression, server push. HTTP/3: built on QUIC (UDP), zero round-trip connection setup, better on lossy networks.

Memoization: cache function results in-process. Object pooling: reuse expensive objects (threads, DB connections, HTTP clients). Both reduce allocation overhead and GC pressure.

Profiling: CPU/memory flame graphs (PySpy, JProfiler, dotTrace). N+1: eager loading, join fetch, DataLoader pattern. EXPLAIN/ANALYZE for query optimization. Always profile before optimizing.

Storage & File Handling

Store files as objects with metadata. Pre-signed URLs for secure direct upload/download. Multipart upload for large files. Lifecycle policies for archival. Event triggers on upload.

var request = new GetPreSignedUrlRequest
{
    BucketName = "my-bucket",
    Key = "uploads/report.pdf",
    Expires = DateTime.UtcNow.AddMinutes(15),
};

Split large files into chunks, upload in parallel, resume on failure. S3 multipart upload, tus protocol. Essential for large files over unreliable connections. Track upload progress.

Process files without loading entirely into memory. Use streams for CSV parsing, image transformation, compression. Prevents OOM on large files. Node.js streams, Python generators, .NET pipelines.

import csv
import io

def process_large_csv(file_stream):
    reader = csv.reader(io.TextIOWrapper(file_stream))
    for row in reader:
        yield transform(row)  # Stream, never buffer all

Thumbnails: Sharp (Node), Pillow (Python), ImageSharp (.NET). PDF extraction, OCR (Tesseract), metadata stripping. Always process async — offload to background jobs or Lambda.

ClamAV: open-source antivirus scanning. Run async — upload → scan queue → move to permanent storage on clean. Quarantine infected files. Never serve unscanned user uploads.

Block storage (EBS, persistent volumes): for database backing. Network file systems (NFS, EFS): shared access across nodes. Content-addressable storage (CAS): deduplication via content hashing.

Protocols & Serialization

HTTP/1.1: persistent connections, chunked encoding. HTTP/2: binary framing, multiplexing, header compression. HTTP/3: QUIC (UDP), zero-RTT handshake. Each version improves latency and throughput.

AMQP (RabbitMQ): reliable enterprise messaging with exchanges, queues, and bindings. MQTT: lightweight IoT pub/sub with QoS levels (0: at most once, 1: at least once, 2: exactly once).

Google's binary serialization format. IDL schema → code generation. 3-10x smaller than JSON. Schema evolution with field numbers. Required for gRPC. Language-neutral.

syntax = "proto3";

message Order {
  int32 id = 1;
  string customer = 2;
  repeated Item items = 3;
  google.protobuf.Timestamp created_at = 4;
}

Avro: schema-embedded binary (Kafka, Hadoop). MessagePack: binary JSON, compact and fast. BSON: MongoDB native format — superset of JSON with types. Choose based on ecosystem and schema needs.

JSON: human-readable, universally supported, schema-optional (JSON Schema). XML: verbose but powerful — XSD validation, XSLT transformation, namespaces. JSON dominates modern APIs; XML persists in enterprise.

Apache Thrift: Facebook's cross-language RPC + serialization framework. FlatBuffers: zero-copy deserialization — read data directly from buffer without unpacking. Used in games and embedded systems.

Extended Architectural Patterns

Entire application deployed as a single unit. Simple to develop, test, deploy. Works well for small teams. Scaling means replicating the whole app. Consider modular monolith before microservices.

Partition the system into self-contained cells — each cell is a complete, independent deployment unit with its own data and compute. Failure in one cell cannot cascade to others. Used at AWS, Azure.

Eliminate the database bottleneck using an in-memory data grid (Hazelcast, Apache Ignite). All processing happens in-memory, data replicated across nodes. Database is async write-behind. Ideal for extreme throughput.

Data flows through sequential processing stages (pipes and filters). Each stage transforms data and passes it to the next. Unix philosophy. Used in ETL, CI/CD, and stream processing. Composable and testable.

Deploy auxiliary functionality (logging, auth, config) as a separate process alongside the main app. Share lifecycle and network. Foundation of service mesh (Envoy, Linkerd proxy). Language-agnostic cross-cutting concerns.

Dedicated backend per client type (web, mobile, IoT). Each BFF aggregates and tailors API responses to its client's needs. Avoids one-size-fits-all APIs. Team ownership aligned to frontend teams.

// Mobile BFF — lightweight payloads
app.get("/api/mobile/dashboard", async (req, res) => {
  const [user, stats] = await Promise.all([
    userService.getSummary(req.userId),
    statsService.getCompact(req.userId),
  ]);
  res.json({ user, stats }); // Tailored for mobile
});

Incrementally migrate from monolith to microservices. Route traffic through a façade: new features go to new services, old features gradually strangled. Zero big-bang rewrite. Named after strangler fig trees.

Extended API Patterns

HTTP callbacks triggered by events. Provider POSTs to subscriber's URL. Include HMAC signature for verification. Implement retry with exponential backoff. Dead-letter failed deliveries. Stripe, GitHub, Slack all use webhooks.

// Verify webhook signature
const signature = req.headers["x-webhook-signature"];
const expected = crypto
  .createHmac("sha256", secret)
  .update(JSON.stringify(req.body))
  .digest("hex");
if (signature !== expected) return res.status(401).end();

XML-based RPC protocol with WS-* standards (WS-Security, WS-ReliableMessaging). WSDL defines contract. Enterprise legacy — still used in banking, healthcare, government. Heavy but feature-rich (transactions, security built-in).

End-to-end typesafe APIs for TypeScript monorepos. No schema definition or code generation — types flow from server to client automatically. Supports subscriptions, middleware, and batching. Perfect for Next.js full-stack apps.

Hypermedia as the Engine of Application State. REST responses include links to related actions/resources. Client discovers capabilities dynamically. True REST maturity level 3. Enables self-describing APIs.

{
  "orderId": 123,
  "status": "pending",
  "_links": {
    "self": { "href": "/orders/123" },
    "cancel": { "href": "/orders/123/cancel", "method": "POST" },
    "payment": { "href": "/orders/123/pay", "method": "POST" }
  }
}

Strategies: URI path (/v2/orders), header (Accept-Version), query param (?version=2), content negotiation (Accept: application/vnd.api.v2+json). URI versioning is most common. Support N-1 versions minimum.

Query params: ?status=active&sort=-created_at. Standardize with RQL, OData, or GraphQL args. Support field selection (?fields=id,name). Prevent injection — whitelist allowed fields. Combine with pagination.

Contract-first REST documentation standard. Define endpoints, schemas, auth in YAML/JSON. Generate client SDKs, server stubs, and interactive docs (Swagger UI, Redoc). Version your spec alongside code.

OpenAPI equivalent for event-driven APIs. Describe message brokers, channels, messages, and schemas. Supports Kafka, AMQP, MQTT, WebSocket. Generate docs and code from spec. Essential for microservices documentation.

Limit concurrent request processing to prevent overload. Queue-based admission control. Semaphore pattern. Distinct from rate limiting (which counts over time). Apply at API gateway or service level.

Aggregate data from multiple microservices into a single response. API Gateway or BFF handles orchestration. Parallel fan-out for independent calls. Handle partial failures with fallbacks.

Batch and cache database queries within a single request to prevent N+1. Collects individual loads, deduplicates, and executes a single batch query. Essential for GraphQL resolvers. Available in all languages.

const userLoader = new DataLoader(async (ids) => {
  const users = await db.users.findMany({
    where: { id: { in: ids } }
  });
  return ids.map(id => users.find(u => u.id === id));
});

Extended Database Patterns

MongoDB, Cosmos DB, Couchbase — store JSON/BSON documents. Flexible schema, embedded subdocuments vs references. Good for content management, catalogs, user profiles. Horizontal scaling via sharding.

Redis, DynamoDB, etcd — simple get/set by key. Sub-millisecond reads. TTL for expiration. Redis adds data structures (lists, sets, sorted sets, streams). DynamoDB adds secondary indexes.

Cassandra, HBase, ScyllaDB — column families with rows and columns. Optimized for write-heavy, time-series, and IoT workloads. Linear horizontal scaling. Tunable consistency. No joins.

Neo4j, Amazon Neptune, ArangoDB — nodes + edges + properties. Traversal queries for relationship-heavy data: social networks, fraud detection, recommendation engines. Cypher / Gremlin query languages.

InfluxDB, TimescaleDB, QuestDB — optimized for timestamped data. Automatic retention policies, downsampling, continuous aggregates. Perfect for metrics, IoT, financial data.

CockroachDB, Google Spanner, TiDB, YugabyteDB — SQL interface with horizontal scaling + full ACID. Distributed transactions across regions. Best of SQL and NoSQL worlds. Higher latency than single-node.

Thin layer over raw SQL with object mapping. No change tracking, no lazy loading — just fast, explicit queries. Dapper (.NET), Knex (Node), SQLx compile-time checked queries (Rust). Choose when ORM overhead is too much.

var orders = await connection.QueryAsync(
    "SELECT * FROM orders WHERE customer_id = @Id",
    new { Id = customerId });

EXPLAIN/ANALYZE to understand execution plans. Covering indexes (include columns needed by query). Join strategies: nested loop, hash join, merge join. Avoid SELECT *. Use CTEs for readability.

EXPLAIN ANALYZE
SELECT o.id, c.name
FROM orders o
JOIN customers c ON o.customer_id = c.id
WHERE o.status = 'pending'
  AND o.created_at > NOW() - INTERVAL '7 days';

Precomputed query results stored as a table. Refresh strategies: on-demand, periodic, incremental. Trade storage for read speed. PostgreSQL REFRESH MATERIALIZED VIEW CONCURRENTLY. Use for dashboards and reports.

Business logic inside the database. Stored procedures: precompiled SQL routines callable from app. Triggers: auto-execute on INSERT/UPDATE/DELETE. Use sparingly — hard to test, version, and debug.

PostgreSQL tsvector/tsquery, Elasticsearch, Apache Lucene. Inverted index, stemming, ranking, fuzzy matching. Use Elasticsearch for advanced faceting, aggregations, and multi-language support.

PostGIS (PostgreSQL), MongoDB GeoJSON, Redis Geo. Spatial indexes (R-tree, GiST). Queries: ST_Distance, ST_Within, ST_Intersects. Used for location-based services, delivery routing, geofencing.

Normalization (1NF→5NF): eliminate redundancy, enforce integrity. Denormalization: pre-join tables for read performance, accept write complexity. OLTP favors normalization; OLAP/read-heavy favors denormalization.

Use multiple database types in one system — each optimized for its access pattern. E.g., PostgreSQL for transactions, Redis for cache, Elasticsearch for search, S3 for files. Adds operational complexity.

Extended Async & Concurrency

Parallelism via partition count. Messages with same key always go to same partition (ordering guarantee). Consumer group members each own partitions. More partitions = more parallelism but higher overhead.

Hangfire (.NET), Celery (Python), Sidekiq (Ruby), BullMQ (Node) — offload heavy/slow tasks from request path. Persistent job storage, retry policies, scheduling, dashboard monitoring.

from celery import Celery

app = Celery("tasks", broker="redis://localhost")

@app.task(bind=True, max_retries=3)
def process_report(self, report_id: int):
    try:
        generate_pdf(report_id)
    except Exception as exc:
        raise self.retry(exc=exc, countdown=60)

Priority queues: high-priority messages processed first (weighted consumers or separate queues). Delay queues: schedule message delivery for future time. Use for scheduled emails, retry delays, SLA-based processing.

At-least-once: broker retries until ACK — consumer must be idempotent. Exactly-once: Transactional Outbox pattern + idempotent consumer with dedup key. True exactly-once is expensive; most systems use at-least-once + idempotency.

Flow control when producers outpace consumers. Strategies: block producer, drop messages, buffer with limit, signal consumer capacity. Reactive Streams spec defines back-pressure protocol. Essential for stream processing.

SELECT FOR UPDATE — acquire row-level lock before reading. Other transactions block until lock released. Prevents lost updates but reduces concurrency. Use for critical financial operations.

BEGIN;
SELECT * FROM accounts
  WHERE id = 123
  FOR UPDATE;  -- Lock this row

UPDATE accounts
  SET balance = balance - 100
  WHERE id = 123;
COMMIT;

Async result placeholders. Future (Java/Rust) / Promise (JS) / Task (.NET) — represent a value that will be available later. Composable: chain, combine, race. Foundation of non-blocking I/O.

Extended Security

Server stores session state (Redis, DB). Client receives HttpOnly, Secure, SameSite cookie with session ID. Stateful — server tracks active sessions. Better for server-rendered apps. Easier revocation than JWT.

XML-based enterprise SSO. Identity Provider (IdP) issues signed XML assertions. Service Provider (SP) validates. Used in corporate environments (Okta, Azure AD). Being replaced by OIDC for modern apps.

Both client and server present certificates. Used for service-to-service auth in zero-trust networks. Service mesh (Istio, Linkerd) automates mTLS. No passwords or tokens — identity is the certificate.

TOTP (Google Authenticator), FIDO2 hardware keys (YubiKey), WebAuthn/Passkeys (biometric). Passkeys are the future — phishing-resistant, no shared secret. Support multiple second factors.

Relationship-Based Access Control. Permissions defined by relationships between users and resources. Google Zanzibar model (SpiceDB, Authzed). ACL: per-resource permission lists. More flexible than RBAC for complex domains.

Open Policy Agent: external policy evaluation using Rego DSL. Cedar (AWS): typed policy language. Decouple authorization logic from application code. Policy-as-code, versioned, testable.

Argon2id (recommended), bcrypt, scrypt, PBKDF2. Never MD5/SHA for passwords. Argon2: memory-hard, resistant to GPU attacks. Use library defaults for cost parameters. Salt automatically included.

from argon2 import PasswordHasher

ph = PasswordHasher()
hash = ph.hash("user_password")
# Verify
ph.verify(hash, "user_password")  # Returns True or raises

At rest: AES-256-GCM, TDE (Transparent Data Encryption), field-level encryption for PII. In transit: TLS 1.3 (mandatory), HSTS headers, certificate pinning for mobile. Encrypt everything — defense in depth.

Content Security Policy (CSP) headers. Output encoding (HTML entity escaping). DOMPurify for sanitizing HTML. Avoid innerHTML. Use framework auto-escaping (React, Angular). Strict CSP eliminates most XSS vectors.

Synchronizer token pattern (hidden form field). SameSite=Strict cookies eliminate most CSRF. Origin/Referer header validation. Double-submit cookie pattern. Modern SPA + SameSite cookies = CSRF largely solved.

Snyk, Trivy, Grype, OWASP Dependency-Check — scan dependencies for known CVEs. Run in CI pipeline, block on critical vulnerabilities. Software Bill of Materials (SBOM) for supply chain transparency.

Extended Observability

ELK stack (Elasticsearch, Logstash, Kibana), EFK (Fluentd), Grafana Loki, Datadog, Splunk. Ship logs from all services to one place. Query across services. Retention policies for cost management.

CPU and memory flame graphs in production. Parca, Pyroscope, Datadog Continuous Profiler. Low overhead (< 2%). Identify hot code paths without reproducing locally. Compare profiles across deploys.

Rate (requests/sec), Errors (failed requests/sec), Duration (latency distribution). The go-to framework for request-driven services (APIs, web apps). Answers: "is my service healthy for users?"

Utilization (% busy), Saturation (queue depth), Errors (error count). For resources: CPU, memory, disk, network. Answers: "is my infrastructure healthy?" Complement RED for full picture.

Google SRE's four signals: Latency, Traffic, Errors, Saturation. Superset of RED + USE. If you can only monitor four things, monitor these. Build alerting around golden signals first.

SLI: measurable indicator (p99 latency, error rate). SLO: target for the SLI (99.9% availability). SLA: contractual agreement with consequences. Error budget = 100% - SLO. When budget exhausted, freeze deploys.

Grafana (open-source), Kibana, Datadog, New Relic. Layer: overview → service → instance. Use golden signals for top-level. Avoid dashboard sprawl — maintain a curated set. Include links to runbooks.

Automated canary requests from external locations. Test critical user flows (login, checkout). Detect outages before users. Tools: Checkly, Datadog Synthetics, Pingdom. Run every 1-5 minutes.

Capture real user performance data from browsers/mobile. Core Web Vitals (LCP, FID, CLS). Correlate frontend perf with backend traces. Tools: Datadog RUM, New Relic Browser, Sentry Performance.

Sentry, Rollbar, Bugsnag — aggregate exceptions across services. Group by stack trace, track regression, assign to owners. Source maps for JS. Release tracking to identify which deploy introduced errors.

New Relic, Dynatrace, Datadog APM — end-to-end transaction tracing, auto-instrumentation, service maps, database query analysis. Correlates traces, metrics, and logs. The single-pane-of-glass for production.

Unique request ID generated at the edge, propagated through all services via headers (X-Request-Id, traceparent). Enables end-to-end request tracing across logs, metrics, and traces. Non-negotiable for microservices.

Head-based: decide at request start (sample 10% of traces). Tail-based: decide after request completes (keep all errors, sample successes). Reduces cost for high-volume systems while preserving signal.

Extended Testing

Full API flows through the entire stack. Supertest (Node), Playwright (browser), RestAssured (Java). Test critical paths: signup → login → purchase → confirm. Run against staging. Keep suite small — E2E is slow.

Stress: push beyond capacity to find breaking point. Soak: run at normal load for hours — find memory leaks, connection exhaustion. Spike: sudden traffic surge — test auto-scaling and circuit breakers.

Chaos: Gremlin, Chaos Mesh, Litmus — inject failures (pod kills, network latency, disk fill). Security: OWASP ZAP (DAST), Snyk (SCA), Semgrep/CodeQL (SAST), pen-testing. Both should run in CI.

Feed random/malformed inputs to find crashes and edge cases. AFL, libFuzzer, cargo-fuzz (Rust). Great for parsers, serializers, and input validation. Can run continuously. Found thousands of CVEs.

Red → Green → Refactor cycle. Write a failing test first, write minimal code to pass, then refactor. Forces design-for-testability. Keeps scope focused. Not just testing — it's a design technique.

Cucumber, SpecFlow — Given/When/Then syntax. Executable specifications that non-technical stakeholders can read. Bridge between business requirements and test code. Use for acceptance criteria.

Feature: Order placement
  Scenario: Successful order
    Given a customer with items in cart
    When they submit the order
    Then order status should be "confirmed"
    And payment should be charged

Mockito (Java), Sinon (JS), unittest.mock (Python), Moq (.NET). Mocks verify interactions; stubs provide canned responses. Use for unit test isolation. Don't mock what you don't own — use adapters.

Factories (Factory Boy, Fishery), fixtures, Object Mother pattern. Generate realistic test data on demand. Avoid shared mutable test state. Use database transactions for test isolation (rollback after each test).

Stub external HTTP services in integration tests. Record real traffic and replay. Simulate errors, latency, timeouts. WireMock (Java/standalone), Mountebank (polyglot), MSW (browser). Essential for testing third-party APIs.

Extended DevOps & Deployment

GitHub Actions, GitLab CI, ArgoCD, Tekton, Jenkins. Stages: lint → test → build → scan → deploy. Trunk-based + feature flags. Keep builds under 10 minutes. Cache dependencies. Parallelize test suites.

Docker, Podman — multi-stage builds for small images, layer caching for fast builds. Distroless/scratch base images for security. Pin versions. Scan images with Trivy. Never run as root in production.

FROM node:20-alpine AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci --production
COPY . .
RUN npm run build

FROM gcr.io/distroless/nodejs20
COPY --from=build /app/dist /app
CMD ["app/server.js"]

EKS, GKE, AKS — container orchestration at scale. Pods, Deployments, Services, Ingress. HPA for auto-scaling. ConfigMaps + Secrets for config. Helm for packaging. Kustomize for environment overlays.

Istio, Linkerd — transparent mTLS, traffic management, observability via sidecar proxies. Retry, circuit breaker, canary routing at infrastructure level. Adds complexity — evaluate need carefully.

Terraform (multi-cloud), Pulumi (real programming languages), CloudFormation/CDK (AWS). Declarative state management. Plan → Apply. State file management. Modules for reuse. Drift detection.

Docker Hub, GitHub Container Registry, Nexus, JFrog Artifactory. Store container images, npm packages, Maven artifacts. Vulnerability scanning on push. Retention policies. Promotion between environments.

Blue-green: instant traffic switch to previous environment. Kubernetes: kubectl rollout undo. Database: backward-compatible migrations (expand-contract). Feature flags: kill switch. Always test rollback.

AWS SAM, Serverless Framework, Vercel, Cloudflare Workers. No server management. Pay-per-invocation. Cold start mitigation: provisioned concurrency, lean runtimes, Snap Start (Java). Best for event-driven workloads.

Init containers: run setup (migrations, config fetch) before main container. Cold-start optimization: provisioned concurrency, lean runtimes, GraalVM native images, connection pooling on startup.

Lambda@Edge, Cloudflare Workers, Deno Deploy — run code at CDN edge locations. Sub-10ms latency. Use for auth, A/B testing, personalization, geolocation routing. Limited runtime and memory.

Dev ≈ Staging ≈ Production. Docker Compose for local dev. Same container images across environments. Infrastructure as Code ensures identical cloud resources. Feature flags instead of environment branches.

Real-Time Techniques

ASP.NET real-time abstraction. Auto-negotiates transport: WebSocket → SSE → Long Polling. Hubs for grouping connections. Redis backplane for multi-server scale. Strongly typed hub contracts.

public class ChatHub : Hub
{
    public async Task SendMessage(string user, string message)
        => await Clients.All.SendAsync("ReceiveMessage", user, message);
}

Client sends request, server holds it open until data is available or timeout. Client immediately reconnects. Legacy real-time fallback. Higher latency and overhead than WebSocket/SSE. Still works everywhere.

Server proactively sends resources before client requests them. Useful for preloading CSS/JS. Largely deprecated in favor of 103 Early Hints and preload link headers. Chrome removed support in 2022.

Stream large HTTP responses progressively without knowing total size upfront. Transfer-Encoding: chunked. Used for streaming APIs, server-sent data, large file downloads. Foundation for HTTP streaming.

Lightweight IoT pub/sub protocol. QoS levels: 0 (fire-and-forget), 1 (at-least-once), 2 (exactly-once). Persistent sessions, last-will messages, retained messages. Brokers: Mosquitto, EMQX, HiveMQ.

Integration & Data Pipelines

ETL: Extract → Transform → Load (Airflow, dbt, Spark). ELT: Load raw data first, transform in warehouse (Snowflake, BigQuery). ELT is modern standard — leverage warehouse compute. dbt for SQL transformations.

Dual processing: batch layer (complete, accurate) + speed layer (fast, approximate). Serving layer merges results. Complex to maintain two codepaths. Being replaced by Kappa for many use cases.

Stream-only processing. Single pipeline processes everything as a stream (Kafka as immutable log). Replay from log for reprocessing. Simpler than Lambda. Works when stream processing is sufficient.

Centralized integration middleware. Message routing, transformation, orchestration. Legacy pattern — replaced by API gateways and event-driven architectures. MuleSoft, IBM Integration Bus. Still in large enterprises.

Track data transformations end-to-end. Great Expectations for data quality. Monte Carlo for data observability. Data lineage: know where data came from and how it was transformed. Essential for compliance.

ClickHouse, Apache Druid, Apache Pinot — sub-second queries on billions of rows. Materialized views for pre-aggregation. Stream ingestion from Kafka. Use for dashboards, metrics, and user-facing analytics.

Bidirectional replication between databases. Both accept writes. Requires conflict resolution (last-write-wins, merge, manual). Complex — prefer single-master with read replicas when possible.

Only transfer changed data since last sync. CDC-based (Debezium), timestamp-based, or version-based. Dramatically reduces data transfer. Used in mobile offline-first apps, caching layers, and replication.

Cross-Cutting Essentials

Shutdown: handle SIGTERM, stop accepting requests, drain in-flight work, close connections, flush logs. Jobs: Quartz (.NET/Java), Hangfire, APScheduler (Python), node-cron. Cron for recurring, delay queues for one-off.

Externalize config from code. Environment variables, Consul KV, Spring Cloud Config, AWS Parameter Store. 12-factor app principles. Config hierarchy: defaults → env file → env vars → CLI args. Never commit secrets.

ICU message format for pluralization and gender. React-intl, i18next, gettext. RTL layout support. Extract strings at build time. Use CLDR for locale data. Store translations in JSON/PO files.

Store timestamps in UTC always. Convert to user timezone for display. Use Temporal API (JS) or NodaTime (.NET). Currency: never use float — Decimal, BigDecimal, or integer cents. Locale-aware formatting.

// Store UTC, display local
const utc = new Date().toISOString();
const local = new Intl.DateTimeFormat("en-US", {
  timeZone: "America/New_York",
  dateStyle: "medium", timeStyle: "short"
}).format(new Date(utc));

Populate database with initial/test data. Separate seed scripts for dev, staging, production. Factories for generating realistic data. Idempotent seeds (check before insert). Reset scripts for testing.

Mask PII in non-prod environments (anonymize names, emails, SSNs). Tools: pg_anonymize, Delphix. Sandboxing: isolate untrusted code execution — Docker, gVisor, Firecracker, WASM. Defense in depth.

Deprecation: Sunset header, API versioning, migration guides, usage tracking before removal. Read-only mode: circuit breaker flag to disable writes during maintenance — serve reads from cache/replicas.

Short-lived feature branches (< 1 day). Merge to main frequently. Feature flags for incomplete work. No long-lived branches. CI runs on every push. Enables continuous deployment.

Right to erasure (soft-delete with hard-delete schedule). Data retention policies. Consent logging. Data portability (export user data). Privacy by design. DPO notification within 72 hours of breach.

Extended Code Quality

Decouple request senders from handlers. Central mediator dispatches commands/queries to their handlers. Foundation for CQRS. MediatR (.NET), mediatr-ts (TypeScript). Pipeline behaviors for cross-cutting concerns.

public record CreateOrderCommand(string Customer) : IRequest;

public class CreateOrderHandler : IRequestHandler
{
    public async Task Handle(CreateOrderCommand cmd, CancellationToken ct)
    {
        var order = new Order(cmd.Customer);
        await _db.SaveChangesAsync(ct);
        return order.Id;
    }
}

Separate API shape from domain model. Map domain entities to DTOs for responses. Prevents leaking internal structure. Use AutoMapper (.NET), class-transformer (TS), or manual mapping. Different DTOs per API version.

SonarQube, ESLint, Checkmarx, Semgrep, CodeQL. Run in CI — gate on quality thresholds. Track code smells, complexity, duplication, security vulnerabilities. Autofix where possible. Enforce team standards.

Extended Scalability

Session affinity: route requests from the same client to the same server. Cookie-based (JSESSIONID) or IP-hash. Required for stateful servers (in-memory sessions). Avoid when possible — prefer stateless + external session store.

Reuse TCP connections across multiple HTTP requests. Avoid handshake overhead. HTTP/1.1 Connection: keep-alive (default). Set appropriate timeout and max-requests. Connection pooling for upstream services.

Defer initialization until first access. ORM lazy loading (load related entities on access). Module lazy loading. Reduces startup time and memory. Beware of N+1 queries with ORM lazy loading.