Hi,
I managed to upgrade from 13 to 14 following the upgrade guide, but am missing all predefined dashboard of Grafana. Is there an alternative way to obtain these dashboards? Are the json files somewhere available?
Thanks,
Wouter
Hej,
Have similar observation after upgrading to 14. It seems that /etc/prometheus/prometheus.yml is missing targets for job 'node' and therefor no metrics gets collected. Adding targets manually fixes it: - job_name: 'node' static_configs: - targets: ['beosrv-c:9100'] - targets: - comp-01:9100' - 'comp-02:9100' etc
But I guess this should be generated automatically by qluman?
Cheers, Iggy
On 2/16/26 16:31, Nicolai, Wouter via Qlustar General wrote:
Hi,
I managed to upgrade from 13 to 14 following the upgrade guide, but am missing all predefined dashboard of Grafana. Is there an alternative way to obtain these dashboards? Are the json files somewhere available?
Thanks,
Wouter
Qlustar General mailing list --qlustar-general@qlustar.org To unsubscribe send an email toqlustar-general-leave@qlustar.org
Hi,
the predefined Grafana dashboards are created dynamically when writing the Alerts Config config class. The predefined dashboards are customized to what is monitored for alerts in the cluster. So if slurm isn't used or no slurm node is monitored then the slurm part of the dashboards are excluded. Similar for GPUs or Kubernetes. And if no nodes are monitored for alerts then no predefined dashboards.
At a minimum you should create 2 Alert Groups in Qluman-qt: Head Node and Compute Node. Open the Alert Configs widget via the Manage Configs / Alert Configs menu entry and click the New button to create a new group. Both groups can be left empty.
Next the Alert Groups need to be assigned to the nodes that should be monitored. For the Head Node group open the Enclosure View and select the beosrv-c node (in case of HA select the 2 physical headnodes instead). Then open the context menu and select: Set Config / Alerts / Head Node to assign the config. For the other nodes you probably have a Host Template that is used and is a more convenient way to add the Alerts Config. Open the Config Sets widget via the Manage Sets / Config Sets menu entry. Add the Compute Node alert group to the FE VM Node, Demo VM Node and Standard node Config Sets or whatever sets you have in use.
Last click the Write Files button at the bottom of the Qluman-qt window (or Manage Cluster / Write Files menu entry). Writing the Alerts entry (or just write all) will update the Prometheus config to gather metrics from any node with an Alert Group assigned. It will also update the predefined Gafana dashboards to match the nodes Prometheus now monitors.
You can read more about the alerts in the Qluman guide: https://docs.qlustar.com/Qlustar/14/ClusterOS/qluman-guide/Config-Classes.ht...
Hope that helps.
Regards,
Goswin von Brederlow
Hi,
I did everything as you advised. Head node appeared in grafana dashboard, but compute nodes - no. In prometheus.yml there are no references to compute nodes after "Write Files". Manually adding compute node entries to prometheus.yml works.
Regards Rolandas
On 2/24/26 13:43, Goswin von Brederlow via Qlustar General wrote:
On 2/16/26 16:31, Nicolai, Wouter via Qlustar General wrote:
Hi,
I managed to upgrade from 13 to 14 following the upgrade guide, but am missing all predefined dashboard of Grafana. Is there an alternative way to obtain these dashboards? Are the json files somewhere available?
Thanks,
Wouter
Qlustar General mailing list --qlustar-general@qlustar.org To unsubscribe send an email toqlustar-general-leave@qlustar.org
Hi,
the predefined Grafana dashboards are created dynamically when writing the Alerts Config config class. The predefined dashboards are customized to what is monitored for alerts in the cluster. So if slurm isn't used or no slurm node is monitored then the slurm part of the dashboards are excluded. Similar for GPUs or Kubernetes. And if no nodes are monitored for alerts then no predefined dashboards.
At a minimum you should create 2 Alert Groups in Qluman-qt: Head Node and Compute Node. Open the Alert Configs widget via the Manage Configs / Alert Configs menu entry and click the New button to create a new group. Both groups can be left empty.
Next the Alert Groups need to be assigned to the nodes that should be monitored. For the Head Node group open the Enclosure View and select the beosrv-c node (in case of HA select the 2 physical headnodes instead). Then open the context menu and select: Set Config / Alerts / Head Node to assign the config. For the other nodes you probably have a Host Template that is used and is a more convenient way to add the Alerts Config. Open the Config Sets widget via the Manage Sets / Config Sets menu entry. Add the Compute Node alert group to the FE VM Node, Demo VM Node and Standard node Config Sets or whatever sets you have in use.
Last click the Write Files button at the bottom of the Qluman-qt window (or Manage Cluster / Write Files menu entry). Writing the Alerts entry (or just write all) will update the Prometheus config to gather metrics from any node with an Alert Group assigned. It will also update the predefined Gafana dashboards to match the nodes Prometheus now monitors.
You can read more about the alerts in the Qluman guide: https:// docs.qlustar.com/Qlustar/14/ClusterOS/qluman-guide/Config- Classes.html#sec-Monitoring
Hope that helps.
Regards,
Goswin von Brederlow
Qlustar General mailing list -- qlustar-general@qlustar.org To unsubscribe send an email to qlustar-general-leave@qlustar.org
Please send a screenshot of the 'Properties and Configs' part of the Enclosure View when one of the compute nodes is selected. Additionally a screenshot of the fully unfolded Alerts node in the File Classes tree of the 'Write Files' window.
On 2/25/26 12:24, Rolandas Naujikas via Qlustar General wrote:
Hi,
I did everything as you advised. Head node appeared in grafana dashboard, but compute nodes - no. In prometheus.yml there are no references to compute nodes after "Write Files". Manually adding compute node entries to prometheus.yml works.
Regards Rolandas
On 2/24/26 13:43, Goswin von Brederlow via Qlustar General wrote:
On 2/16/26 16:31, Nicolai, Wouter via Qlustar General wrote:
Hi,
I managed to upgrade from 13 to 14 following the upgrade guide, but am missing all predefined dashboard of Grafana. Is there an alternative way to obtain these dashboards? Are the json files somewhere available?
Thanks,
Wouter
Qlustar General mailing list --qlustar-general@qlustar.org To unsubscribe send an email toqlustar-general-leave@qlustar.org
Hi,
the predefined Grafana dashboards are created dynamically when writing the Alerts Config config class. The predefined dashboards are customized to what is monitored for alerts in the cluster. So if slurm isn't used or no slurm node is monitored then the slurm part of the dashboards are excluded. Similar for GPUs or Kubernetes. And if no nodes are monitored for alerts then no predefined dashboards.
At a minimum you should create 2 Alert Groups in Qluman-qt: Head Node and Compute Node. Open the Alert Configs widget via the Manage Configs / Alert Configs menu entry and click the New button to create a new group. Both groups can be left empty.
Next the Alert Groups need to be assigned to the nodes that should be monitored. For the Head Node group open the Enclosure View and select the beosrv-c node (in case of HA select the 2 physical headnodes instead). Then open the context menu and select: Set Config / Alerts / Head Node to assign the config. For the other nodes you probably have a Host Template that is used and is a more convenient way to add the Alerts Config. Open the Config Sets widget via the Manage Sets / Config Sets menu entry. Add the Compute Node alert group to the FE VM Node, Demo VM Node and Standard node Config Sets or whatever sets you have in use.
Last click the Write Files button at the bottom of the Qluman-qt window (or Manage Cluster / Write Files menu entry). Writing the Alerts entry (or just write all) will update the Prometheus config to gather metrics from any node with an Alert Group assigned. It will also update the predefined Gafana dashboards to match the nodes Prometheus now monitors.
You can read more about the alerts in the Qluman guide: https:// docs.qlustar.com/Qlustar/14/ClusterOS/qluman-guide/Config- Classes.html#sec-Monitoring
Hope that helps.
Regards,
Goswin von Brederlow
Qlustar General mailing list -- qlustar-general@qlustar.org To unsubscribe send an email to qlustar-general-leave@qlustar.org
Qlustar General mailing list -- qlustar-general@qlustar.org To unsubscribe send an email to qlustar-general-leave@qlustar.org
-
Goswin von Brederlow -
ignas.bunikis＠scilifelab.uu.se -
Nicolai, Wouter -
Roland Fehrenbacher -
Rolandas Naujikas