4. Sample dashboards and the metrics they use
VDURA provides five sample dashboards. They are examples for you to import into your own Grafana instance, not part of the VDURA system, and they were exercised against the test realm described in Chapter 1. Their queries are the best evidence of which metrics the VDURA team treats as primary signals, and they double as worked examples for the tables above.
4.1 PanFS Alerts (panfs-alerts)
vmalert view: Firing counts by severity, category and node, a firing over time chart, and a table of the ALERTS series.
| Panel | Type | Query |
|---|---|---|
| Firing (total) | stat | count(ALERTS{alertstate=“firing”}) or vector(0) |
| Critical | stat | count(ALERTS{alertstate=“firing”,severity=“Critical”}) or vector(0) |
| Warning | stat | count(ALERTS{alertstate=“firing”,severity=“Warning”}) or vector(0) |
| Informational | stat | count(ALERTS{alertstate=“firing”,severity=“Informational”}) or vector(0) |
| Firing alerts by severity over time | timeseries | count by (severity) (ALERTS{alertstate=“firing”}) |
| Currently firing | table | ALERTS{alertstate=“firing”} |
| Firing by category | bargauge | count by (category) (ALERTS{alertstate=“firing”}) |
| Firing by node | bargauge | count by (node_name) (ALERTS{alertstate=“firing”}) |
4.2 PanFS API - Audit (panfs-api-audit)
VictoriaLogs only. Reads /var/log/panapi-audit.log with tenant_id and user_id filters: Events over time by result, failed operations, top action x resource pairs, failures by status_code.
| Panel | Type | Query |
|---|---|---|
| Audit events over time (by result) | timeseries | filename:=“/var/log/panapi-audit.log” tenant_id:$tenant_id user_id:$user_id | stats by (_time:$__interval, result) count() events |
| Failed operations (selected range) | stat | filename:=“/var/log/panapi-audit.log” result:=failed tenant_id:$tenant_id user_id:$user_id | stats count() failures |
| Top operations (action x resource) | table | filename:=“/var/log/panapi-audit.log” tenant_id:$tenant_id user_id:$user_id | stats by (action, resource) count() requests | sort by (requests desc) limit 20 |
| Failures by status_code | table | filename:=“/var/log/panapi-audit.log” result:=failed tenant_id:$tenant_id user_id:$user_id | stats by (status_code, action, resource) count() failures | sort by (failures desc) limit 50 |
| Audit log stream | logs | filename:=“/var/log/panapi-audit.log” tenant_id:$tenant_id user_id:$user_id |
4.3 PanFS Logs - Browser (panfs-logs-browser)
VictoriaLogs only. Generic browser over any log file (filename variable) or systemd unit (unit variable) with error/warning/info counts and top collapsed message patterns.
| Panel | Type | Query |
|---|---|---|
| Total lines | stat | filename:$filepath | stats count() total |
| Errors | stat | filename:$filepath (level:~“(?i)error|fatal” OR (NOT level:* AND _msg:~“(?i)error|fail”)) | stats count() errors |
| Warnings | stat | filename:$filepath (level:~“(?i)warn” OR (NOT level:* AND _msg:~“(?i)warn”)) | stats count() warnings |
| Info | stat | filename:$filepath (level:~“(?i)info” OR (NOT level:* AND _msg:~“(?i)info”)) | stats count() info |
| Top 10 patterns (numbers collapsed) | table | filename:$filepath | collapse_nums | stats by (_msg) count() hits | sort by (hits desc) limit 10 |
| Errors by file | table | filename:$filepath (level:~“(?i)error|fatal” OR (NOT level:* AND _msg:~“(?i)error|fail”)) | stats by (filename) count() errors | sort by (errors desc) limit 50 |
| File logs (filename=~$filepath) | logs | filename:$filepath |
| Total lines | stat | unit:$unit | stats count() total |
| Errors | stat | unit:$unit (level:~“(?i)error|fatal” OR (NOT level:* AND _msg:~“(?i)error|fail”)) | stats count() errors |
| Warnings | stat | unit:$unit (level:~“(?i)warn” OR (NOT level:* AND _msg:~“(?i)warn”)) | stats count() warnings |
| Info | stat | unit:$unit (level:~“(?i)info” OR (NOT level:* AND _msg:~“(?i)info”)) | stats count() info |
| Top 10 patterns (numbers collapsed) | table | unit:$unit | collapse_nums | stats by (_msg) count() hits | sort by (hits desc) limit 10 |
| Errors by unit | table | unit:$unit (level:~“(?i)error|fatal” OR (NOT level:* AND _msg:~“(?i)error|fail”)) | stats by (unit) count() errors | sort by (errors desc) limit 50 |
| Journal/unit logs (unit=~$unit) | logs | unit:$unit |
4.4 PanFS Realm Node (panfs-realm-node)
Per node drill down driven by a $Node variable built from label_values(system_uptime, host): CPU per core, top processes by CPU and RSS, memory breakdown, disk I/O and IOPS per device.
| Panel | Type | Query |
|---|---|---|
| Per-host CPU utilization (Total) | gauge | 100 - avg by (host) (cpu_usage_idle{cpu=“cpu-total”, host=~“$Node”}) |
| Per-host CPU utilization (Total) | timeseries | 100 - avg by (host) (cpu_usage_idle{cpu=“cpu-total”, host=~“$Node”}) |
| CPU utilization (Cores) | bargauge | 100 - cpu_usage_idle{cpu!=“cpu-total”, host=“$Node”} |
| Top 10 Processes (CPU Usage) | bargauge | topk(10, procstat_cpu_usage{host=~“$Node”}) |
| Memory Used | gauge | mem_used_percent{host=~“$Node”} |
| Memory Usage | timeseries |
mem_total{host=~“$Node”} mem_used{host=~“$Node”} |
| Memory Usage | timeseries |
mem_wired{host=~“$Node”} mem_active{host=~“$Node”} mem_inactive{host=~“$Node”} mem_laundry{host=~“$Node”} mem_free{host=~“$Node”} |
| Top 10 Processes (Memory Usage) | bargauge | topk(10, procstat_memory_rss{host=~“$Node”}) |
| Disk IO (Read/Write) | timeseries |
diskio_read_bytes{host=~“$Node”, name!=“pass0”} diskio_write_bytes{host=~“$Node”, name!=“pass0”} |
| Disk IO (Rate[$__rate_interval]) | timeseries |
rate(diskio_read_bytes{host=~“$Node”, name!=“pass0”}[$__rate_interval]) rate(diskio_write_bytes{host=~“$Node”, name!=“pass0”}[$__rate_interval]) |
| Disk Throughput | timeseries | sum by (host, node_type) (rate(diskio_read_bytes{host=~“$Node”}[$__rate_interval]) + rate(diskio_write_bytes{host=~“$Node”}[$__rate_interval])) |
| Total IOPS (read + write) per host | timeseries | sum by (node_type) (rate(diskio_reads{host=~“$Node”}[$__rate_interval]) + rate(diskio_writes{host=~“$Node”}[$__rate_interval])) |
4.5 PanFS Realm Overview (panfs-overall-cluster)
Realm wide view: Node inventory (panfs_node), StorageSet and volume tables, firing alerts by severity, CPU and memory utilization, disk and network throughput by node_type, metadata and free capacity from the iopath fs0..2 gauges, DirectFlow bytes in/out, and log volume from VictoriaLogs.
| Panel | Type | Query |
|---|---|---|
| Uptime Cluster Nodes | table | count by (node_type) (system_uptime) |
| Storage | table |
count(group by (name) (panfs_storageset > 0)) count(group by (volume_id) (panfs_volume > 0)) |
| Firing Alerts | bargauge | count by (severity) (ALERTS{alertstate=“firing”}) |
| Versions | table | group by (software_revision) (last_over_time(panfs_node[$__rate_interval])) |
| CPU Utilization | stat | 100 - avg by (host) (cpu_usage_idle{cpu=“cpu-total”}) |
| Memory Utilization | stat | mem_used_percent |
| Cluster Nodes | table | group by (name, ip_address, node_type, node_name, node_state, node_data_serial, node_hardware_serial, node_state_name, node_state_description, enclosure_name, software_revision, cpu_model, memory_bytes, cpu_active_cores, internal_temperature_celsius, cpu_temperature_celsius) (last_over_time(panfs_node[$__rate_interval])) |
| Storage Sets | table | topk(1, last_over_time(panfs_storageset[5m]>0)) by (name) |
| Volumes | table | topk(1, last_over_time(panfs_volume[5m])) by (volume_id) |
| Active alert messages | state-timeline |
max by (message) ( (ALERTS{severity=“Critical”}) * 3 or (ALERTS{severity=“Warning”}) * 2 or (ALERTS{severity=“Informational”}) * 1 ) |
| StorageSets Alerts | table | ALERTS{message=~“.(BladeSet |Storage[ ]?[Ss]et ).”} |
| Volume Alerts | table | ALERTS{message=~“.(Volume ).”} |
| Software Alerts | table | ALERTS{message=~“(Software Process.|The process .restarted)”} |
| Disk Throughput | timeseries | sum by (host, node_type) (rate(diskio_read_bytes[$__rate_interval]) + rate(diskio_write_bytes[$__rate_interval])) |
| Total IOPS (read + write) per host | timeseries | sum by (host, node_type) (rate(diskio_reads[$__rate_interval]) + rate(diskio_writes[$__rate_interval])) |
| CPU Utilization | timeseries | avg by (host) (100 - cpu_usage_idle{cpu=“cpu-total”}) |
| Disk IO Usage | timeseries |
(sum(rate(diskio_read_bytes{node_type=“vpod”}[$__rate_interval])) + sum(rate(diskio_write_bytes{node_type=“vpod”}[$__rate_interval]))) sum(rate(diskio_reads{node_type=“vpod”}[$__rate_interval])) + sum(rate(diskio_writes{node_type=“vpod”}[$__rate_interval])) |
| Throughput Out | timeseries | sum(rate(net_bytes_sent{interface=~“eth0|bond0”}[$__rate_interval]) * 8) by (node_type) |
| Throughput In | timeseries | sum(rate(net_bytes_recv[$__rate_interval]) * 8) by (node_type) |
| Metadata Used | timeseries | sum(magent_local_sw_iopath_fs0_metadata_capacity+magent_local_sw_iopath_fs1_metadata_capacity+magent_local_sw_iopath_fs2_metadata_capacity) |
| Total Capacity (Free) | timeseries | sum(last_over_time(magent_local_sw_iopath_fs0_free_capacity[$__rate_interval])) |
| IOPath Measure (DF Bytes In/Out) | timeseries |
sum(last_over_time(magent_local_sw_iopath_measure_df_bytes_in[$__rate_interval])) sum(last_over_time(magent_local_sw_iopath_measure_df_bytes_out[$__rate_interval])) |
| Duration | timeseries | sum(last_over_time(magent_local_sw_iopath_measure_df_io_duration_usec[$__rate_interval])) |
| Log volume (all logs) | timeseries |
|
| Log rate (total vs error), per second | timeseries |
|