- Qlustar General - Qlustar Mailing Lists

Qlustar HPC Core Stack 24.06
by no-reply-list＠qlustar.org 29 Jun '24

29 Jun '24

The Qlustar team is pleased to announce the availability of the HPC Core Stack 24.06 for Qlustar 13. It was published together with the latest security bug fix update and can be installed by updating the Qlustar 13 chroots on the cluster. The HPC CORE STACK [1] was introduced with the release of Qlustar 13 as a new component definition within Qlustar. It is related to the introduction of Spack. This release includes the following individual major HPC core stack component updates: - Spack version 0.22.0 [2] - Slurm version 23.11.8 [3] - RDMA-Core version 52.0 [4] - HWloc version 2.10.0 [5] - UCX version 1.17.0 [6] The new version of Spack allows easy and direct access to the most recent compilers, MPIs and hundreds of other applications relevant to HPC. Note that the default OpenMPI version is now 5.0.3, a new major version. The latest 4.1.x versions were also tested to still build cleanly with this new Qlustar release. As part of this update, Qlustar AlmaLinux was moved to the 8.10 version which is now the only release supported by upstream. Minor version updates for other Qlustar 13 components are also included, in particular Lustre (2.15.5) and gdrcopy (2.4.1) for use with Nvidia GPUs. [1] https://qlustar.com/about-qlustar/hpc-core-stack [2] https://spack.readthedocs.io/en/latest/ [3] https://www.schedmd.com/slurm/ [4] https://github.com/linux-rdma/rdma-core [5] https://www.open-mpi.org/projects/hwloc/ [6] https://openucx.org/

1 0

Security/Bug fix releases 13.2.0, 12.0.3.6
by no-reply-list＠qlustar.org 29 Jun '24

29 Jun '24

The Qlustar releases 13.2.0 and 12.0.3.6 are ready including a number of security/bug fixes. Please check the following web pages for details about security fixes and special update instructions: https://qlustar.com/qsa/2024/0629241 https://qlustar.com/qsa/2024/0629242 The following non-security related updates/enhancements/bug fixes are included: For Qlustar 13 only: - Qlustar HPC Core Stack 06/24 update (details to be announced separately) - Update to QluMan 13.6.0 + Make qluman-qt work without qlumand to support HA GUI support at any time + Add --power options to qluman-cli + Fix file descriptor leak in qluman-execd + Add right to write NameService config for roles with LDAP User/Group rights + Store QluMan user login certs in a new file to account for incompatible format changes. Path: ~/.config/Qluman/cluster.safe.v2 + qluman-cli: add --create flag for --gencert to generate a one-time login certificate for a new user - Update to AlmaLinux 8.10

1 0

Security/Bug fix releases 13.1.6, 12.0.3.5
by no-reply-list＠qlustar.org 23 Apr '24

23 Apr '24

The Qlustar releases 13.1.6 and 12.0.3.5 are ready including a number of security/bug fixes. Please check the following web pages for details about security fixes and special update instructions: https://qlustar.com/qsa/2024/0422241 https://qlustar.com/qsa/2024/0422242 The following non-security related updates/enhancements/bug fixes are included: For Qlustar 13: - Update to QluMan 13.5.9

1 0

Security/Bug fix releases 13.1.5, 12.0.3.4
by no-reply-list＠qlustar.org 13 Mar '24

13 Mar '24

The Qlustar releases 13.1.5 and 12.0.3.4 are ready including a number of security/bug fixes. Please check the following web pages for details about security fixes and special update instructions: https://qlustar.com/qsa/2024/0313241 https://qlustar.com/qsa/2024/0313242 https://qlustar.com/qsa/2024/0313243 The following non-security related updates/enhancements/bug fixes are included: For Qlustar 13: - Update to QluMan 13.5.5 - Update to Nvidia graphics drivers 550.54.14 - Fix disk-auto-setup for mdRAID/LVM configurations - Fix install of qluman-qt on head-nodes

1 0

qluman-qt dependencies problem
by Hernan Olivera 09 Mar '24

09 Mar '24

Hi I'm trying a fresh install of version 13. When apt install qluman-qt: The following packages have unmet dependencies: qluman-qt-13 : depends: qluman-common-13 (= 13.5.1+13-jammy) but 13.5.3+13-jammy is to be installed E: unabe to correct problems, you have held broken packages. Ideas? thanks -- Hernan Olivera

2 4

Exception trying to save updated Info field for user in LDAP (Qluman/server 13.4.7)
by Rolandas Naujikas 29 Feb '24

29 Feb '24

Exception handling request Traceback (most recent call last): File "/usr/lib/python3/dist-packages/qluman-13/server/admin.py", line 2967, in on_request yield from handler(self, channel, msg) File "/usr/lib/python3/dist-packages/qluman-13/server/admin.py", line 1759, in request_DBUpdateRequest table.update(entries) File "/usr/lib/python3/dist-packages/qluman-13/server/db/Ldap.py", line 106, in update entries = self._do_update(entries) File "/usr/lib/python3/dist-packages/qluman-13/server/db/Ldap.py", line 443, in _do_update self.manage.ol_user_modify(entry.uname, entry) File "/usr/lib/python3/dist-packages/qluman-13/components/ldap/manage.py", line 741, in ol_user_modify attr = USER_DB_TO_LDAP_FIELD_NAME[field] KeyError: 'locked'

3 4

Re: Qluman GUI 13.4.7 / 13.5.1 / 13.5.2 connection issues
by Roland Fehrenbacher 26 Feb '24

26 Feb '24

On 2/26/24 08:23, Nicolai, Wouter wrote: > Yes, using 13.5.3 I`m able to connect to the cluster directly. > Thanks for resolving this issue. OK, very good. > For your information, I do get the following logging with an error: > > Qt: Session management error: None of the authentication protocols specified are supported > 2024-02-26 08:14:45,105 ERROR qlunet.Node Node[b'admin@clustername']: ZMQ error on reconnect, trying again in 10s > Traceback (most recent call last): > File "/usr/lib/python3/dist-packages/qlunet/Node.py", line 1054, in slave_reset_keys > raise last_err > File "/usr/lib/python3/dist-packages/qlunet/Node.py", line 1044, in slave_reset_keys > self._monitor = self._socket.get_monitor_socket() > File "/usr/lib/python3/dist-packages/zmq/sugar/socket.py", line 1008, in get_monitor_socket > self.monitor(addr, events) > File "zmq/backend/cython/socket.pyx", line 697, in zmq.backend.cython.socket.Socket.monitor > File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc > zmq.error.ZMQError: Address already in use This is a long-standing issue with the ZMQ library. Fortunately it is harmless, only leads to a connection delay in rare situations (e.g. when a user connects for the first time). Nevertheless, we'll make a new attempt to fix this.

1 0

Qluman GUI 13.4.7 / 13.5.1 / 13.5.2 connection issues
by Nicolai, Wouter 23 Feb '24

23 Feb '24

Hi all, When I try to connect to our head node using qluman I'm getting the following error while using using "ssh -X root@servername qluman-qt" to connect: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/qluman-13/qluman-qt.py", line 74, in <module> main() File "/usr/lib/python3/dist-packages/qluman-13/qluman-qt.py", line 70, in main w = MainWindow.make(background=not args.nobackground, app=app) File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 204, in make w.conf(cluster_model=model, background=background, app=app) File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 213, in conf self.restoreState() File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 320, in restoreState state = QtCore.Qt.WindowState(int(state)) TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType' Also tried to connect using apptainer with qluman-13.5.2-singularity.sqsh (or qluman-13.4.7-singularity.sqsh) and receive something similar: QStandardPaths: error creating runtime directory '/run/user/1000' (Read-only file system) QStandardPaths: error creating runtime directory '/run/user/1000' (Read-only file system) Qt: Session management error: None of the authentication protocols specified are supported Traceback (most recent call last): File "/usr/lib/python3/dist-packages/qluman-13/qluman-qt.py", line 74, in <module> main() File "/usr/lib/python3/dist-packages/qluman-13/qluman-qt.py", line 70, in main w = MainWindow.make(background=not args.nobackground, app=app) File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 204, in make w.conf(cluster_model=model, background=background, app=app) File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 213, in conf self.restoreState() File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 320, in restoreState state = QtCore.Qt.WindowState(int(state)) TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType' Qlustar server version is 13.5.1 (fresh install of last month). We also have an old headnode running qluman 13.2.1. I am able to connect to the new head node after connecting to the qluman server of our old headnode (running qluman server 13.2.1) using "ssh -X root@OLDservername qluman-qt" and connecting the new headnode using the Qluman GUI. While this is a nice workaround, I would like to decommission the old headnode in the near future. Am I doing something wrong while trying to connect directly to our new head node using either ssh -X or apptainer? Head node is fully up to date, ran "apt- update" & "apt dist-upgrade" Thanks, Wouter

2 2

Security/Bug fix releases 13.1.4, 12.0.3.3
by no-reply-list＠qlustar.org 30 Jan '24

30 Jan '24

The Qlustar releases 13.1.4 and 12.0.3.3 are ready including a number of security/bug fixes. Please check the following web pages for details about security fixes and special update instructions: https://qlustar.com/qsa/2024/0130241 https://qlustar.com/qsa/2024/0130242 The following non-security related updates/enhancements/bug fixes are included: For Qlustar 13: - Update to QluMan 13.5.0 - Update to Lustre 2.15.4 - Update to Nvidia graphics drivers 535.154.05 - Switch to use pam_slurm_adopt for restricting ssh access to slurm nodes

1 0

Infiniband - headnode does not have ib0 interface / ip adress on infiniband interface
by wo.nicolai＠vuykrotterdam.com 30 Jan '24

30 Jan '24

Hi all, I've started with a new cluster using Qlustar 13.1 (fresh install). Hardware setup contains (currently) 3 HP DL380 machines, all hardware boxes have an infiniband interface and are connected to the same infiniband switch. 1 headnode (with openSM during installation), FE-login as VM, 2 compute nodes. I noticed that the on the head node the infiniband interface is down. Upon investigating I discovered the following: In the qlustar management interface, the network config for the headnode shows ib0 in the commandline interface on the headnode using "ip address" I do not see any ib0 interface. I do see ibo49d1 (o not zero) "9: ibo49d1: <BROADCAST,MULTICAST> mtu 2044 qdisc fq_codel state DOWN group default qlen 256 link/infiniband 80:00:02:25:fe:80:00:00:00:00:00:00:04:09:73:ff:ff:e4:f4:22 brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff altname ibp4s0d1" in /etc/network/interfaces interface ib0 is mentioned When I update the network config for the headnode in the qlustar management interface from ib0 to ibo49d1 and reboot after, the interface remains down without any ip address. I also noticed that for such a change in the qlustar management interface, I do not need to write files? in /etc/network/interface ib0 remains mentioned, while I expected that this would be updated to ibo49d1. Manually changing ib0 into ibo49d1 in /etc/network/interfaces did not work either. The infiniband port is active: ibv_devinfo -d mlx4_0 hca_id: mlx4_0 transport: InfiniBand (0) fw_ver: 2.42.5700 node_guid: 0409:73ff:ffe4:f420 sys_image_guid: 0409:73ff:ffe4:f423 vendor_id: 0x02c9 vendor_part_id: 4103 hw_ver: 0x0 board_id: HP_1380110017 phys_port_cnt: 2 port: 1 state: PORT_DOWN (1) max_mtu: 4096 (5) active_mtu: 1024 (3) sm_lid: 0 port_lid: 0 port_lmc: 0x00 link_layer: Ethernet port: 2 state: PORT_ACTIVE (4) max_mtu: 4096 (5) active_mtu: 4096 (5) sm_lid: 1 port_lid: 1 port_lmc: 0x00 link_layer: InfiniBand Using ibnetdiscover on the headnode, both compute nodes and the switch is listed. Obviously I'm missing something. How do I get IPoIB up and running on the headnode? The above is for the head node only. the compute nodes do have an interface ib0 and an ip adress for this interface.

3 3

2025

2024

2023

2022

2021

2020

2019

2018

Qlustar General