Modernizing a Cloud-Native Security Platform at Scale

Working on this project involved several non-trivial challenges:

• Legacy Codebase Across Multiple Stacks

  The platform included legacy services written in TypeScript, Python, and Node.js, each with different architectural assumptions and tooling.

• High Entry Cost for Domain Understanding

  Before implementing changes, we had to deeply understand the existing security workflows, data models, and event lifecycles across multiple services.

• Local Integration Testing

  Reproducing realistic end-to-end scenarios locally required significant effort due to the number of interconnected services, external integrations, and asynchronous workflows.

• Infrastructure and Platform Migration

  The transition from GCP to AWS, Spanner to PostgreSQL, and Pub/Sub to Kafka required careful planning to avoid downtime and data inconsistencies.

Here’s how the platform operates when handling real-world security events:

• Event Сollection

  Physical security devices, like card readers at building entrances, cameras in parking lots, or sensors on gates, generate events whenever something happens. For example: "Card #12345 scanned at North Entrance at 8:47 AM."

• Event Streaming

  These events flow into Kafka, which acts like a high-speed pipeline that can handle thousands of events per second. Kafka ensures no events are lost and makes them available to multiple services simultaneously.

• Identity Enrichment

  The identity enrichment service picks up each event and adds context. It looks up who owns Card #12345 and adds information like: "John Smith, Engineering Department, authorized for North Entrance access."

• Rule Processing

  Rule-processing services evaluate each enriched event against configured security rules. For example: Is this person authorized for this entrance? Is this access happening during allowed hours? Is there an unusual pattern (like entering the same building twice without exiting)?

• AI Analytics

  The AI-powered analytics platform looks for patterns across many events. It might detect unusual behavior like when a vehicle repeatedly circling the parking lot, multiple failed access attempts in a short time, or access patterns that don't match someone's normal schedule

• Workflow Orchestration

  When security rules are triggered or anomalies are detected, Temporal manages the response workflow. For example, it can send an alert to the security team, lock down a specific entrance, or require additional verification for the next access attempt. Temporal ensures these multi-step workflows complete reliably, even if they take hours or span multiple shifts.

• Dashboard and Management

  Security operators use the web dashboard to view real-time security events as they happen, see analytics and statistics about access patterns, and manage user permissions and access rights. The dashboard also allows to investigate incidents and review historical data and configure rules and response workflows. All of this happens in real-time, processing events within seconds of when they occur at physical devices.