Monitoring and Alerting
Monitoring and Alerting
Monitor health, performance, and security of your DuoKey AWS XKS Proxy
Overview
Effective monitoring for AWS XKS includes:
CloudWatch Metrics
Track XKS proxy performance and health
Proxy Logs
Monitor detailed proxy operations
Key Manager Logs
Track key operations at the source
Security Monitoring
Detect anomalies and unauthorized access
AWS CloudWatch Metrics
Key Metrics
| Metric | Description | Thresholds |
|---|---|---|
| XksProxyLatency | Time AWS KMS waits for proxy responses | Normal: <50ms, Warning: 50-100ms, Critical: >100ms |
| XksProxyCredentialAge | Age of proxy authentication credentials | Rotate every 90 days |
| NumberOfRequests | Count of KMS requests | Monitor for anomalies |
| UserErrorCount | Failed requests count | Alert on elevated rates |
# View XksProxyLatency metric
aws cloudwatch get-metric-statistics \
--namespace AWS/KMS \
--metric-name XksProxyLatency \
--dimensions Name=CustomKeyStoreId,Value=cks-1234567890abcdef0 \
--start-time $(date -u -d '1 hour ago' '+%Y-%m-%dT%H:%M:%S') \
--end-time $(date -u '+%Y-%m-%dT%H:%M:%S') \
--period 300 \
--statistics Average,Maximum,Minimum \
--output table# Check credential age
aws cloudwatch get-metric-statistics \
--namespace AWS/KMS \
--metric-name XksProxyCredentialAge \
--dimensions Name=CustomKeyStoreId,Value=cks-1234567890abcdef0 \
--start-time $(date -u -d '24 hours ago' '+%Y-%m-%dT%H:%M:%S') \
--end-time $(date -u '+%Y-%m-%dT%H:%M:%S') \
--period 3600 \
--statistics Maximum \
--output tableCreating CloudWatch Alarms
aws cloudwatch put-metric-alarm \
--alarm-name xks-high-latency \
--alarm-description "Alert when XKS proxy latency exceeds threshold" \
--metric-name XksProxyLatency \
--namespace AWS/KMS \
--statistic Average \
--period 300 \
--evaluation-periods 2 \
--threshold 100 \
--comparison-operator GreaterThanThreshold \
--dimensions Name=CustomKeyStoreId,Value=cks-1234567890abcdef0 \
--alarm-actions arn:aws:sns:us-east-1:123456789012:xks-alertsCloudWatch Dashboard
{
"widgets": [
{
"type": "metric",
"properties": {
"metrics": [
["AWS/KMS", "XksProxyLatency", {"stat": "Average"}],
["...", {"stat": "Maximum"}]
],
"period": 300,
"stat": "Average",
"region": "us-east-1",
"title": "XKS Proxy Latency",
"yAxis": {
"left": {
"min": 0,
"max": 200
}
}
}
},
{
"type": "metric",
"properties": {
"metrics": [
["AWS/KMS", "NumberOfRequests", {"stat": "Sum"}]
],
"period": 300,
"stat": "Sum",
"region": "us-east-1",
"title": "Request Volume"
}
},
{
"type": "metric",
"properties": {
"metrics": [
["AWS/KMS", "UserErrorCount", {"stat": "Sum"}],
[".", "SystemErrorCount", {"stat": "Sum"}]
],
"period": 300,
"stat": "Sum",
"region": "us-east-1",
"title": "Error Rates"
}
}
]
}aws cloudwatch put-dashboard \
--dashboard-name XKS-Monitoring \
--dashboard-body file://xks-dashboard.jsonDuoKey Cockpit Request Logging
The XKS proxy is not a separate service — it is served by the same DuoKey Cockpit process that hosts your other apps. There is no independent proxy binary, configuration file, or dedicated log file to manage; XKS proxy requests are logged alongside the rest of your Cockpit deployment's activity.
Reviewing Proxy Activity
Each incoming AWS KMS request against an XKS endpoint is logged with its operation, response status, and duration. Where you review this activity depends on how your DuoKey Cockpit deployment is operated:
- If DuoKey hosts Cockpit for you, request activity for an endpoint is visible from the endpoint's detail page in DuoKey Cockpit, and DuoKey Support can pull extended logs on request.
- If you self-host DuoKey Cockpit, forward the process's standard output to whatever log aggregation platform you already use for the rest of your Cockpit deployment (for example, a CloudWatch Logs group or your SIEM of choice).
Log Analysis
Once your Cockpit request logs are flowing into a log aggregation platform, you can build the same kind of queries you would for any other structured log stream, for example:
- Count of requests by operation (
GetHealthStatus,GetKeyMetadata,Encrypt,Decrypt) - Average latency per endpoint
- Failed requests over a recent time window
Use your log platform's own query language (CloudWatch Logs Insights, an Elasticsearch/OpenSearch query, etc.) rather than shell scripting against a local file — DuoKey Cockpit does not write to a separate, proxy-specific log file.
Shipping Logs to CloudWatch
If your DuoKey Cockpit deployment runs in a container, configure your container platform's logging driver (for example, the awslogs driver on ECS, or the CloudWatch agent for EC2 or on-prem hosts) to ship its standard output to a CloudWatch Logs group, the same way you would for any other containerized application.
Network Monitoring
#!/bin/bash
# Monitor round-trip time to XKS proxy
PROXY_HOST="xks-proxy.example.com"
LOG_FILE="/var/log/xks-network-monitor.log"
while true; do
TIMESTAMP=$(date -u '+%Y-%m-%dT%H:%M:%S')
RTT=$(curl -o /dev/null -s -w '%{time_total}' https://$PROXY_HOST/health)
RTT_MS=$(echo "$RTT * 1000" | bc)
echo "$TIMESTAMP,$RTT_MS" >> $LOG_FILE
# Alert if RTT exceeds threshold
if (( $(echo "$RTT_MS > 100" | bc -l) )); then
echo "WARNING: High latency detected: ${RTT_MS}ms" | \
aws sns publish \
--topic-arn arn:aws:sns:us-east-1:123456789012:xks-alerts \
--subject "XKS High Latency Alert"
fi
sleep 60
done#!/bin/bash
# Check XKS proxy TLS certificate expiration
PROXY_HOST="xks-proxy.example.com"
CERT_EXPIRY=$(echo | openssl s_client -servername $PROXY_HOST \
-connect $PROXY_HOST:443 2>/dev/null | openssl x509 -noout -enddate | \
cut -d= -f2)
EXPIRY_EPOCH=$(date -d "$CERT_EXPIRY" +%s)
CURRENT_EPOCH=$(date +%s)
DAYS_UNTIL_EXPIRY=$(( ($EXPIRY_EPOCH - $CURRENT_EPOCH) / 86400 ))
echo "Certificate expires in $DAYS_UNTIL_EXPIRY days"
if [ $DAYS_UNTIL_EXPIRY -lt 30 ]; then
echo "WARNING: Certificate expires soon!" | \
aws sns publish \
--topic-arn arn:aws:sns:us-east-1:123456789012:xks-alerts \
--subject "XKS Certificate Expiration Warning"
fiSecurity Monitoring
AWS CloudTrail Integration
# Query CloudTrail for XKS-related events
aws cloudtrail lookup-events \
--lookup-attributes \
AttributeKey=ResourceName,AttributeValue=cks-1234567890abcdef0 \
--start-time $(date -u -d '24 hours ago' '+%Y-%m-%dT%H:%M:%S') \
--query 'Events[*].{
Time:EventTime,
User:Username,
Event:EventName,
Resource:Resources[0].ResourceName
}' \
--output tableDetect Unusual Activity
#!/bin/bash
# Detect unusual patterns using your aggregated Cockpit request logs.
# Replace YOUR_LOG_QUERY with the equivalent query against wherever you've
# shipped your Cockpit request logs (CloudWatch Logs Insights, your SIEM, etc.)
# Spike in failed requests
FAILED_COUNT=$(YOUR_LOG_QUERY --since "5 minutes ago" --status error --count)
if [ $FAILED_COUNT -gt 50 ]; then
echo "ALERT: Unusual spike in failed requests: $FAILED_COUNT in 5 minutes"
fi
# Requests from unexpected IPs
KNOWN_IPS="52.94.133.0/24,52.95.0.0/16"
UNKNOWN_IPS=$(YOUR_LOG_QUERY --field source_ip --distinct | \
grep -v -f <(echo $KNOWN_IPS | tr ',' '\n'))
if [ -n "$UNKNOWN_IPS" ]; then
echo "ALERT: Requests from unknown IPs: $UNKNOWN_IPS"
fiDuoKey Cockpit does not write to a dedicated local file for the XKS proxy — these examples assume you've already forwarded Cockpit's request logs to a log aggregation platform, as described above.
Alerting Strategies
Critical Alerts (Immediate Action)
| Alert | Trigger | Action |
|---|---|---|
| XKS Proxy Unreachable | 3 consecutive health check failures | Page on-call engineer |
| Authentication Failures | >10 auth failures in 5 minutes | Security team notification |
| Extreme Latency | Average >200ms for 10 minutes | Operations team notification |
Warning Alerts (Investigation Needed)
| Alert | Trigger | Action |
|---|---|---|
| Elevated Latency | Average >100ms for 15 minutes | Email operations team |
| Certificate Expiration | <30 days until expiration | Email security team |
| High Error Rate | Error rate >5% for 10 minutes | Email operations team |
Info Alerts (Awareness)
| Alert | Trigger | Action |
|---|---|---|
| Credential Age | Credentials older than 60 days | Email reminder to rotate |
| Usage Spike | Request volume 2x normal baseline | Email operations team |
Monitoring Best Practices
Monitoring Checklist
Compliance Reporting
#!/bin/bash
# Generate monthly XKS compliance report
MONTH=$(date -u -d 'last month' '+%Y-%m')
REPORT_FILE="xks-compliance-report-$MONTH.txt"
echo "DuoKey AWS XKS Compliance Report - $MONTH" > $REPORT_FILE
echo "==========================================" >> $REPORT_FILE
echo "" >> $REPORT_FILE
# Key Operations Summary
echo "Key Operations Summary:" >> $REPORT_FILE
aws cloudtrail lookup-events \
--lookup-attributes AttributeKey=ResourceType,AttributeValue=AWS::KMS::Key \
--start-time $(date -u -d "$MONTH-01" '+%Y-%m-%dT%H:%M:%S') \
--end-time $(date -u -d "$MONTH-01 +1 month" '+%Y-%m-%dT%H:%M:%S') \
--query 'Events[*].EventName' | \
jq -r '.[]' | sort | uniq -c >> $REPORT_FILE