Glossary
Plain-language definitions for terms used across the dashboards.
- Access log
- File named dispatcher_httpdaccess_*.log. Records every HTTP request that hit a publish pod — bot crawls, human page views, health probes, internal axios calls.
- Dispatcher log
- File named dispatcher_aemdispatcher_*.log. Records the AEM dispatcher's serving decisions: cache hits / misses, response times, cache rule blocks. Used for cache analysis joins.
- Cache layer
- AEM's dispatcher. Sits in front of publish pods and caches rendered pages. When a request hits the dispatcher: it either serves from cache (hit), fetches from publish + caches (miss), passes through (none / non-cacheable), or denies (blocked).
- Bot-only storage
- The tool's core principle. The parser reads every line of an access log but only stores the lines that come from bots. Human and internal traffic is kept as aggregate counts per upload; those raw lines are discarded. A daily log of roughly a million lines typically keeps a few thousand bot rows.
- Traffic types
- After parsing each line: bot (named or unknown_bot in user-agent), internal_health (Envoy/HC sidecar), internal_service (axios from internal services regardless of source IP), internal_other (private-IP non-axios), human (everything else).
- Bot category
- ai (training crawlers — GPTBot, ChatGPT-User, ClaudeBot, etc.), search (Googlebot, Bingbot, Baiduspider), social (facebookbot, Twitterbot), seo (SemrushBot, AhrefsBot, etc.), monitoring (UptimeRobot), unknown (matched bot|crawl|spider catch-all).
- Match rate (cache analysis)
- Percentage of bot events that the dispatcher saw on the same publish pod within ±1 second. High = bots traverse dispatcher. Low = bots reach publish directly (health probes, or CDN absorbs them).
- actionnone
- Dispatcher served the request without a cache decision — typically dynamic content (page fragments, API calls) or URLs explicitly marked non-cacheable. High share = lots of live rendering on bot traffic.
- actionhit / actionmiss / actionpass / actionblocked / actionpurge
- Dispatcher cache outcomes. Hit = served from cache; Miss = fetched from publish, then cached; Pass = bypass cache entirely (e.g. authenticated pages); Blocked = denied by dispatcher filter rules; Purge = cache invalidation request.
- Pod
- An AEM publish pod ID (e.g. cm-pXXXXX-eXXXXXX-aem-publish-<hash>). Multiple pods serve traffic in parallel; same request goes to the same pod's access log + that pod's name appears in the dispatcher's routing field.
- Unparsed samples
- Up to 100 lines per upload that didn't match either Format A or Format B (the two AEM access-log variants). Kept with the upload for format-drift debugging. If unparsed count exceeds ~1% of total, treat as a fire — AEM may have changed format.