Beyond HTTP 200: Engineering SentinelAPI, an API Security Monitoring Platform with WSO2 and Machine Learning
From OAuth 2.0 and API throttling to reliable telemetry pipelines, explainable security alerts, and Isolation Forest anomaly detection.
View GitHub Repository ↗Verified Engineering Results
A historical verification snapshot, not live counters or evidence of a confirmed attack.
Rate limiting verified
Actual WSO2 throttling responses observed during controlled testing.
Gateway events persisted
PostgreSQL event count recorded during a verified local test. Historical count, not a live counter.
Linked evidence records
Gateway events linked to a genuinely persisted API error-rate alert.
An HTTP 200 OK tells me that a request succeeded. It does not tell me whether authentication failures are increasing, a client is exhausting its quota, or an unusual pattern is emerging across hundreds of requests.
That gap led me to build SentinelAPI: a local API security monitoring platform that connects WSO2 API Manager to a custom telemetry pipeline, PostgreSQL, explainable detection, Isolation Forest experiments, and a React dashboard.
My central question was practical: How can I turn gateway activity into trustworthy security intelligence without making application requests depend on monitoring?
The gateway enforces policy. SentinelAPI investigates the outcomes. The strongest result was an end-to-end local workflow that persisted actual gateway events and generated an error-rate alert with linked evidence. The screenshots below distinguish those results from dashboard fixtures and offline synthetic evaluation.
1. Why Gateway Enforcement Is Only the Starting Point
WSO2 already handles routing, OAuth authentication, subscriptions, and throttling. I wanted to complement those capabilities rather than recreate them.
Repeated authentication failures, excessive traffic, and elevated error rates each raise questions beyond a single response. A gateway can reject a request correctly while an operator still needs to understand the pattern around it.
Enforcement decides what happens to a request. Monitoring preserves and investigates patterns across requests. Neither a 401, a 404, nor a 429 establishes malicious intent on its own.
2. System Architecture & Key Trade-Offs
I kept collection and detection outside the live gateway request path. WSO2 processes API traffic; the monitoring pipeline observes its logs separately. A failed collector or a slow detection transaction must not become a prerequisite for returning an API response.
Synchronous API request path
- API Client→
- WSO2 API Manager→
- DemoCommerce API
OAuth, routing, and throttling stay at the gateway. Responses return to the client without waiting for detection.
Asynchronous telemetry & detection
- WSO2 API Logs→
- Python Log Bridge→
- Event Collector→
- PostgreSQL
- Monitoring API→
- React Dashboard
The Monitoring API reads persisted records. Neither detector sits in the gateway request path.
| Layer | Technology | Responsibility |
|---|---|---|
| Gateway | WSO2 API Manager 4.5.0 | OAuth, routing, subscriptions, throttling |
| Controlled backend | Python, FastAPI | Predictable API behavior for testing |
| Transport and collection | Python | Bounded log reads, sanitization, normalization |
| Persistence | PostgreSQL 17 | Traffic, rules, alerts, evidence, ML records |
| Schema management | SQLAlchemy, Alembic | Data models and migrations |
| Detection | Python | Asynchronous sliding-window error-rate analysis |
| ML experimentation | Isolation Forest | Offline anomaly scoring and evaluation |
| Monitoring interface | FastAPI, React, TypeScript, Vite | Authenticated data access and investigation |
| Local runtime | Docker, Docker Compose | Reproducible service execution |
The trade-off is that monitoring becomes eventually consistent. Freshness, checkpoints, late events, and recovery need explicit handling. In return, detection remains separate from the gateway's response lifecycle.
3. Building a Controlled WSO2 Integration
I built DemoCommerceAPI in FastAPI to generate predictable responses without involving production traffic or external users. I registered it in WSO2 with version 1.0.0, context /democommerceapi, OAuth 2.0 security, and subscription validation enabled.
The request path was simple: Client → WSO2 Gateway → FastAPI Backend. My first milestone was to verify it through actual responses.
OAuth: locating the failure, not guessing
I created a Developer Portal application, subscribed it to the API, and generated credentials for controlled testing. Without a valid token, WSO2 returned HTTP 401 Unauthorized. With a valid bearer token, the endpoint returned HTTP 200 OK.
During initial testing I repeatedly received 401 responses. The issue was not always gateway configuration: PowerShell was not consistently passing the access token. This WSO2 response helped locate the problem:
{
"code": "900902",
"message": "Missing Credentials"
}I checked credential handling without printing the token. The lesson was to separate client invocation, credential transport, gateway validation, and application authorization before changing configuration.
Rate limiting: configuration versus enforcement
For throttling, I used a separate test application with the 10PerMin application-level policy. Requests initially succeeded, then WSO2 returned HTTP 429 Too Many Requests after the configured quota was exhausted.
The management screen proves configuration, not enforcement. The response codes and persisted gateway records establish what happened during the controlled run. SentinelAPI observes these outcomes; WSO2 remains responsible for protecting the backend.
4. Engineering a Reliable Telemetry Pipeline
WSO2 API logs provided methods, paths, correlation identifiers, event flow, and response status codes. But one invocation can produce REQUEST_IN, REQUEST_OUT, RESPONSE_IN, and RESPONSE_OUT records. Treating every log line as a new request would inflate traffic counts.
I built a bounded host-side Python bridge to parse the source records and transport sanitized observations through a shared Docker spool. The collector normalized completed API events and persisted them in PostgreSQL rather than storing entire raw logs.
| Persisted field | Purpose |
|---|---|
event_id | Unique event identity and replay protection |
timestamp | Event completion time |
http_method | Observed request method |
endpoint | Sanitized route |
status_code | HTTP outcome |
source | Telemetry origin |
api_service_id | Associated registered API |
Checkpointing tracks progress across interruptions. Stable event identifiers and database uniqueness constraints reduce duplicate persistence during replay. Partial records, retries, processing boundaries, and failure recovery all needed deliberate treatment.
I also kept missing information explicit. WSO2 BASIC telemetry does not supply every identity or classification needed for client-specific rules. A source IP is not automatically a verified application identity; inventing that relationship would weaken downstream findings.
5. Rule-Based Detection & Evidence-Backed Alerts
The project defines request-flood, error-rate, endpoint-enumeration, authentication-failure, and unusual-activity strategies. Defining a strategy does not make it eligible for live evaluation.
With the available telemetry, I enabled API-level error-rate detection. The other strategies remain disabled until verified identity and classification data can satisfy their evidence requirements.
Sliding-window evaluation
The enabled worker evaluates a five-minute window with these default conditions:
- At least 20 observed requests.
- An error proportion strictly greater than 50%.
- HTTP status codes of 400 or higher counted as errors.
error_rate = responses_with_status_at_least_400 / total_observed_responses
finding = request_count >= 20 and error_rate > 0.50This is the rule's decision logic, not a captured terminal result. The worker maintains a processing watermark and allows a bounded amount of late-arriving telemetry.
A transaction-scoped PostgreSQL advisory lock prevents competing workers from processing the same workload concurrently. Incident fingerprints deduplicate repeated evaluations. Alert creation, evidence persistence, and checkpoint advancement happen transactionally so that progress cannot advance independently of the corresponding records.
The first persisted finding
Controlled gateway traffic produced this genuine error-rate alert:
| Attribute | Recorded value |
|---|---|
| Title | Observed API error-rate spike |
| Severity | Medium |
| Risk score | 50.00 |
| Status | Open |
| First detected | 24 October 2026, 07:52:41 UTC |
| Linked evidence | 16 gateway records |
The detector generated the alert from stored gateway events; it was not manually inserted for presentation. Its interpretation remains limited: an elevated error rate is a signal for investigation, not proof of an attacker.
Tracing the finding back to events
The security_alerts table stores the finding, alert_evidence links it to observations, and traffic_events retains the underlying events. In this case, supporting records included HTTP 404 responses involving the DemoCommerce orders route.
An investigator can inspect method, sanitized endpoint, response code, source, timestamp, and evidence classification. That makes the finding explainable without attributing the activity to a particular attacker.
6. Isolation Forest: An Experiment, Not an Attack Verdict
Rules are useful when a condition can be stated precisely. I introduced Isolation Forest to explore patterns across traffic-window features that are harder to capture with one threshold.
The algorithm isolates unusual observations through randomized partitioning. I chose it to explore unsupervised anomaly scoring without requiring a large labeled attack corpus for training, and to compare a statistical signal with deterministic rules.
Controlled offline evaluation
I evaluated 180 controlled synthetic traffic windows, covering normal behavior, benign bursts, and unusual scenarios. These are offline experimental results, not production API security accuracy.
The confusion matrix includes both correct and incorrect classifications. It shows that some scenario windows were classified as normal and some normal windows as anomalies; this experiment does not support a claim of perfect separation.
The overlap is part of the result. Legitimate bursts may be statistically unusual, operational failures may resemble abuse, and malicious activity may resemble normal traffic.
I treated model outputs as signals requiring context. The experiment helped me understand trade-offs between precision, recall, and false positives; it did not establish independently verified real-world attack-detection capability.
7. Designing an Honest Monitoring Interface
I built the dashboard with React, TypeScript, and Vite. It separates security overview, API traffic, threat alerts, detection rules, ML monitoring, and API services into focused views.
The FastAPI backend uses authenticated endpoints. Monitoring credentials expose operational data, while security credentials govern alert and detection access. The security API distinguishes reader, analyst, and administrator roles.
When live data is unavailable, the interface should say so. It must not substitute sample numbers and present them as actual traffic. I kept demo mode visibly distinct and added empty and failure states because trustworthy presentation is part of correctness in a security tool.
8. Engineering Challenges, Debugging & Verification
Missing telemetry despite a working gateway
Problem. Authorized requests returned 200 and throttling returned 429, but the collector was unhealthy and PostgreSQL was not receiving new traffic.
Diagnosis. I inspected collector health, Docker mounts, spool contents, and checkpoints. The spool had no event batches and source freshness was false: the host bridge was not delivering fresh input.
Resolution. Starting the existing bridge in continuous processing mode restored the pipeline. The collector reported this health state:
{
"ok": true,
"source_fresh": true
}It subsequently reported Committed 54 gateway events. Direct database verification found 75 persisted gateway events at that point, including 429 outcomes from wso2-gateway with actual request timestamps. These are historical observations from that test, not current counters.
Lesson. A working source does not guarantee a working pipeline. Check what each boundary produced, received, and durably stored.
A successful migration command with an outdated schema
Problem. The database contained initial monitoring tables, but the detection worker expected newer fields and tables.
Diagnosis. New Alembic files existed in the repository, but the migration Docker image predated them. The command succeeded because it had applied every migration the old image knew about.
Resolution. I rebuilt the image and applied the remaining migrations. PostgreSQL reached revision b7d000000001, including detection checkpoints and ML-window structures, while preserving the existing 75 telemetry records.
Lesson. An exit code describes command execution. Verifying the resulting schema establishes whether the intended change actually happened.
Verifying the workflow at several layers
I checked each layer with evidence appropriate to it:
| Layer | What I checked | What that establishes |
|---|---|---|
| Gateway | OAuth responses, routing, throttling | Actual request handling in the local test |
| Telemetry | Logs, spool, freshness, database rows | Collection and durable persistence |
| Detection | Worker behavior, alert and evidence records | An explainable local finding |
| ML | Synthetic confusion matrix and score distributions | Behavior within the offline experiment |
| Interface | Authenticated connections and unavailable-data states | Explicit presentation of data provenance |
A healthy container does not prove fresh ingestion. A frontend check does not verify stored telemetry. An offline ML result does not validate production detection. Checking these boundaries helped me separate what the system demonstrated from what still needed evidence.
9. What I Would Improve Next
Richer telemetry. Trusted application identities, route classifications, and authentication outcomes would allow additional client-specific strategies to be evaluated responsibly.
Broader validation. More representative datasets and operational testing are needed before making claims about real-world ML reliability or enabling the remaining detection categories.
Throughput and resilience. I would benchmark sustained traffic, strengthen pipeline supervision, improve late-event handling, and test historical replay and ingestion-failure alerts. Bounded processing makes limits explicit; it does not remove the need to measure them.
10. What Building SentinelAPI Taught Me
I expected anomaly detection to be the hardest part. Many of the most valuable lessons came from the systems around it: passing credentials correctly, proving throttling through actual responses, recovering ingestion, and finding schema drift behind a successful command.
Building across API management, backend processing, persistence, detection, ML experimentation, and the frontend made me think in connected workflows rather than isolated features.
What I can demonstrate is a local path from gateway activity to durable observations and an explainable detection result. What I cannot infer from that is universal attack coverage or production-validated ML accuracy.
Detecting something unusual is only the beginning. The engineering challenge is preserving the evidence, proving what happened, and communicating the finding within the limits of that evidence.
From gateway enforcement to explainable evidence.
SentinelAPI brings together WSO2 API management and OAuth, verified throttling, telemetry processing and PostgreSQL persistence, sliding-window error-rate detection, evidence-backed alerts, Isolation Forest experimentation, and a full-stack monitoring interface.
A local engineering prototype: its controlled results show a working pipeline, not production-validated attack detection.