# Server & Infrastructure Monitoring

> Server monitoring alerts with a shell script and cron: disk space, memory usage, CPU load, and SSL certificate expiry, routed to Slack, email, or SMS.

Source: https://docs.alphorn.dev/use-cases/server-monitoring/

Lightweight server monitoring doesn't need a full observability stack. A few scripts and Alphorn can alert you before problems become outages.

## Disk space alerts

```bash title="check-disk.sh"
#!/bin/bash
WEBHOOK="https://app.alphorn.dev/n/V1StGXR8_Z5jdHi6B-myT"
ALPHORN_API_KEY="${ALPHORN_API_KEY:-alp_...}"  # or export it in the environment
THRESHOLD=85
HOSTNAME=$(hostname)

df -h -x squashfs -x tmpfs -x devtmpfs -x overlay --output=pcent,target | tail -n+2 | while read usage mount; do
  PERCENT=${usage%\%}
  if [ "$PERCENT" -ge "$THRESHOLD" ]; then
    PRIORITY=4
    [ "$PERCENT" -ge 95 ] && PRIORITY=5  # critical: page on-call
    curl -fsS -X POST "$WEBHOOK" \
      -H "Authorization: Bearer $ALPHORN_API_KEY" \
      -H "Content-Type: application/json" \
      -d "{
        \"title\": \"Disk space warning: $HOSTNAME\",
        \"message\": \"$mount is at ${PERCENT}% capacity\",
        \"priority\": $PRIORITY,
        \"tags\": [\"server\", \"disk\", \"$HOSTNAME\"]
      }"
  fi
done
```

## Memory usage

This one builds the JSON with `jq`, so install it first.

```bash title="check-memory.sh"
#!/bin/bash
WEBHOOK="https://app.alphorn.dev/n/V1StGXR8_Z5jdHi6B-myT"
ALPHORN_API_KEY="${ALPHORN_API_KEY:-alp_...}"  # or export it in the environment
THRESHOLD=90
HOSTNAME=$(hostname)

MEM_PERCENT=$(free | awk '/^Mem:/ {printf "%.0f", $3/$2 * 100}')

if [ "$MEM_PERCENT" -ge "$THRESHOLD" ]; then
  TOP=$(ps aux --sort=-%mem | head -6 | tail -5 | awk '{printf "%s %s%%\n", $11, $4}')
  # jq escapes the multi-line process list into valid JSON
  jq -n --arg h "$HOSTNAME" --arg p "$MEM_PERCENT" --arg top "$TOP" \
    '{title: "High memory usage: \($h)", message: "Memory at \($p)%. Top processes:\n\($top)", priority: 4, tags: ["server", "memory", $h]}' |
  curl -fsS -X POST "$WEBHOOK" \
    -H "Authorization: Bearer $ALPHORN_API_KEY" \
    -H "Content-Type: application/json" \
    --data-binary @-
fi
```

## CPU load

```bash title="check-cpu.sh"
#!/bin/bash
WEBHOOK="https://app.alphorn.dev/n/V1StGXR8_Z5jdHi6B-myT"
ALPHORN_API_KEY="${ALPHORN_API_KEY:-alp_...}"  # or export it in the environment
HOSTNAME=$(hostname)
CORES=$(nproc)
LOAD=$(awk '{print $1}' /proc/loadavg)

# Alert if the 1-minute load average exceeds the number of CPU cores (needs bc installed)
if [ "$(echo "$LOAD > $CORES" | bc)" -eq 1 ]; then
  curl -fsS -X POST "$WEBHOOK" \
    -H "Authorization: Bearer $ALPHORN_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{
      \"title\": \"High CPU load: $HOSTNAME\",
      \"message\": \"Load average: $LOAD (${CORES} cores)\",
      \"priority\": 4,
      \"tags\": [\"server\", \"cpu\", \"$HOSTNAME\"]
    }"
fi
```

## SSL certificate expiry

```bash title="check-ssl.sh"
#!/bin/bash
WEBHOOK="https://app.alphorn.dev/n/V1StGXR8_Z5jdHi6B-myT"
ALPHORN_API_KEY="${ALPHORN_API_KEY:-alp_...}"  # or export it in the environment
DOMAIN="your-app.com"
DAYS_WARNING=14

EXPIRY=$(echo | openssl s_client -servername "$DOMAIN" -connect "$DOMAIN:443" 2>/dev/null | openssl x509 -noout -enddate 2>/dev/null | cut -d= -f2)

# openssl failed (DNS, connection refused, no certificate): report that instead of "0 days left"
if [ -z "$EXPIRY" ]; then
  curl -fsS -X POST "$WEBHOOK" \
    -H "Authorization: Bearer $ALPHORN_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{
      \"title\": \"SSL check FAILED: $DOMAIN\",
      \"message\": \"Could not read the certificate from $DOMAIN:443\",
      \"priority\": 4,
      \"tags\": [\"server\", \"ssl\", \"$DOMAIN\"]
    }"
  exit 1
fi

EXPIRY_EPOCH=$(date -d "$EXPIRY" +%s)
NOW_EPOCH=$(date +%s)
DAYS_LEFT=$(( (EXPIRY_EPOCH - NOW_EPOCH) / 86400 ))

if [ "$DAYS_LEFT" -le "$DAYS_WARNING" ]; then
  curl -fsS -X POST "$WEBHOOK" \
    -H "Authorization: Bearer $ALPHORN_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{
      \"title\": \"SSL certificate expiring soon\",
      \"message\": \"$DOMAIN expires in $DAYS_LEFT days ($EXPIRY)\",
      \"priority\": 4,
      \"tags\": [\"server\", \"ssl\", \"$DOMAIN\"]
    }"
fi
```

## Cron schedule

Run all checks periodically:

```
*/5 * * * * /path/to/check-disk.sh
*/5 * * * * /path/to/check-memory.sh
*/5 * * * * /path/to/check-cpu.sh
0 8 * * *   /path/to/check-ssl.sh
```

Cron doesn't load your shell profile, so set `ALPHORN_API_KEY` in the scripts or at the top of the crontab (see [Cron Job & Script Monitoring](/use-cases/cron-jobs#reusable-wrapper)).

## Routing examples

| Channel | Filter | Purpose |
|---|---|---|
| Slack (#infra) | Tags has any of `server` | All server alerts |
| PagerDuty | Priority equals 5 and tags has any of `server` | Page on-call for critical issues (disk at 95% or more) |
| Email | Tags has any of `ssl` | Track certificate status |