Monitoring and logging

Monitoring and logging

Monitor, log, and debug SKAO telescope software.

Access centralised monitoring, logging, and debugging tools across all SKAO datacentres.

SKAO’s centralised solution eliminates multiple dashboards across datacentres, aggregates data, provides monitoring with logging and alerting, and secures communication using zero-trust principles.

Note

Logged data quality determines the logging system’s value. Ensure your application follows the SKA Log Message Format.

⚡ Tutorial

Access pipeline links, dashboards, and logs for the first time.

🔧 How-to Guides

Filter logs in Kibana, query Elasticsearch locally, use Grafana dashboards.

📖 Reference

SKA Log Message Format, log fields, service URLs.

⚙️ How It Works

Architecture overview, Headlamp, Coder, and debugging tools.

Service URLs

Available services vary by environment. Production environments lack some services.

Monitoring and Logging:

Datacentre	Monitoring	Logging
stfc-techops (cicd)	https://monitoring.skao.int	https://k8s.stfc.skao.int/kibana
stfc-dp (cicd)	https://monitoring.skao.int	https://k8s.stfc.skao.int/kibana
aws-*	https://monitoring.skao.int	https://k8s.stfc.skao.int/kibana
mid-itf/low-itf	https://monitoring.skao.int	https://k8s.stfc.skao.int/kibana
mid-aa	https://monitoring.skao.int	https://k8s.mid.internal.skao.int/kibana
low-aa	https://monitoring.skao.int	https://k8s.low.internal.skao.int/kibana

Cluster Access and Debugging:

Datacentre	Headlamp	Coder
stfc-techops (cicd)	https://k8s.stfc.skao.int/headlamp	https://coder.k8s.stfc.skao.int/login
stfc-dp (cicd)	https://sdhp.stfc.skao.int/headlamp	N/A
aws-*	N/A	N/A
mid-itf/low-itf	Local access *	N/A
mid-aa	Local access *	N/A
low-aa	Local access *	N/A

* For local access, follow the Headlamp Locally with InfraHQ guide.

Key tools

• Grafana — Dashboards for infrastructure metrics, CI/CD pipelines, and namespace resources

• Kibana — Log searching and visualisation with Elasticsearch backend

• Headlamp — Real-time Kubernetes cluster access and monitoring

• Coder — Remote debugging environment with IDE access

• Prometheus — Metrics collection with Thanos for high availability