- Qlustar General - Qlustar Mailing Lists

Qluman GUI 13.4.7 / 13.5.1 / 13.5.2 connection issues
by Nicolai, Wouter 23 Feb '24

23 Feb '24

Hi all, When I try to connect to our head node using qluman I'm getting the following error while using using "ssh -X root@servername qluman-qt" to connect: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/qluman-13/qluman-qt.py", line 74, in <module> main() File "/usr/lib/python3/dist-packages/qluman-13/qluman-qt.py", line 70, in main w = MainWindow.make(background=not args.nobackground, app=app) File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 204, in make w.conf(cluster_model=model, background=background, app=app) File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 213, in conf self.restoreState() File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 320, in restoreState state = QtCore.Qt.WindowState(int(state)) TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType' Also tried to connect using apptainer with qluman-13.5.2-singularity.sqsh (or qluman-13.4.7-singularity.sqsh) and receive something similar: QStandardPaths: error creating runtime directory '/run/user/1000' (Read-only file system) QStandardPaths: error creating runtime directory '/run/user/1000' (Read-only file system) Qt: Session management error: None of the authentication protocols specified are supported Traceback (most recent call last): File "/usr/lib/python3/dist-packages/qluman-13/qluman-qt.py", line 74, in <module> main() File "/usr/lib/python3/dist-packages/qluman-13/qluman-qt.py", line 70, in main w = MainWindow.make(background=not args.nobackground, app=app) File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 204, in make w.conf(cluster_model=model, background=background, app=app) File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 213, in conf self.restoreState() File "/usr/lib/python3/dist-packages/qluman-13/client/qt/MainWindow.py", line 320, in restoreState state = QtCore.Qt.WindowState(int(state)) TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType' Qlustar server version is 13.5.1 (fresh install of last month). We also have an old headnode running qluman 13.2.1. I am able to connect to the new head node after connecting to the qluman server of our old headnode (running qluman server 13.2.1) using "ssh -X root@OLDservername qluman-qt" and connecting the new headnode using the Qluman GUI. While this is a nice workaround, I would like to decommission the old headnode in the near future. Am I doing something wrong while trying to connect directly to our new head node using either ssh -X or apptainer? Head node is fully up to date, ran "apt- update" & "apt dist-upgrade" Thanks, Wouter

2 2

Security/Bug fix releases 13.1.4, 12.0.3.3
by no-reply-list＠qlustar.org 30 Jan '24

30 Jan '24

The Qlustar releases 13.1.4 and 12.0.3.3 are ready including a number of security/bug fixes. Please check the following web pages for details about security fixes and special update instructions: https://qlustar.com/qsa/2024/0130241 https://qlustar.com/qsa/2024/0130242 The following non-security related updates/enhancements/bug fixes are included: For Qlustar 13: - Update to QluMan 13.5.0 - Update to Lustre 2.15.4 - Update to Nvidia graphics drivers 535.154.05 - Switch to use pam_slurm_adopt for restricting ssh access to slurm nodes

1 0

Infiniband - headnode does not have ib0 interface / ip adress on infiniband interface
by wo.nicolai＠vuykrotterdam.com 30 Jan '24

30 Jan '24

Hi all, I've started with a new cluster using Qlustar 13.1 (fresh install). Hardware setup contains (currently) 3 HP DL380 machines, all hardware boxes have an infiniband interface and are connected to the same infiniband switch. 1 headnode (with openSM during installation), FE-login as VM, 2 compute nodes. I noticed that the on the head node the infiniband interface is down. Upon investigating I discovered the following: In the qlustar management interface, the network config for the headnode shows ib0 in the commandline interface on the headnode using "ip address" I do not see any ib0 interface. I do see ibo49d1 (o not zero) "9: ibo49d1: <BROADCAST,MULTICAST> mtu 2044 qdisc fq_codel state DOWN group default qlen 256 link/infiniband 80:00:02:25:fe:80:00:00:00:00:00:00:04:09:73:ff:ff:e4:f4:22 brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff altname ibp4s0d1" in /etc/network/interfaces interface ib0 is mentioned When I update the network config for the headnode in the qlustar management interface from ib0 to ibo49d1 and reboot after, the interface remains down without any ip address. I also noticed that for such a change in the qlustar management interface, I do not need to write files? in /etc/network/interface ib0 remains mentioned, while I expected that this would be updated to ibo49d1. Manually changing ib0 into ibo49d1 in /etc/network/interfaces did not work either. The infiniband port is active: ibv_devinfo -d mlx4_0 hca_id: mlx4_0 transport: InfiniBand (0) fw_ver: 2.42.5700 node_guid: 0409:73ff:ffe4:f420 sys_image_guid: 0409:73ff:ffe4:f423 vendor_id: 0x02c9 vendor_part_id: 4103 hw_ver: 0x0 board_id: HP_1380110017 phys_port_cnt: 2 port: 1 state: PORT_DOWN (1) max_mtu: 4096 (5) active_mtu: 1024 (3) sm_lid: 0 port_lid: 0 port_lmc: 0x00 link_layer: Ethernet port: 2 state: PORT_ACTIVE (4) max_mtu: 4096 (5) active_mtu: 4096 (5) sm_lid: 1 port_lid: 1 port_lmc: 0x00 link_layer: InfiniBand Using ibnetdiscover on the headnode, both compute nodes and the switch is listed. Obviously I'm missing something. How do I get IPoIB up and running on the headnode? The above is for the head node only. the compute nodes do have an interface ib0 and an ip adress for this interface.

3 3

node install II (forget the previous one)
by Christophe Guilbert 08 Jan '24

08 Jan '24

Scratch my previous email since I moved forward. Here my new problem. I managed to create beo-206 , I can see it in "Enclosure View" Than I select it in "Enclosure View" and open "Manage Hosts" -> "Hardware Wizard" -> "Selected" I have the following message: "Missing Hardware info: beo-206" I created specific "host template", "hardware property set" for that node (2 x 12 core, 2 thread per core), I tried all kind of properties combinations. no luck , the same message of missing hardware is coming back. I did took example on beosrv-c and set up the properties as following: "beo-206" -> "Select host template" -> "<None>" "beo-206" -> "Set Config" -> "Network Config" -> "Standard-Node" it does not work either. same error. On the node side , PXE is looping trying to get a DHCP address. rebooting the node did not changed the outcome. What I am missing here, how to complete the "Hardware Wizard" ? I tried all kind of properties for beo-206, which ones would you recommend me to try ? Thanks Chris

2 1

node install
by Christophe Guilbert 03 Jan '24

03 Jan '24

Hi guys and happy new year. I am trying to move forward with my first node install So far, my node was set to beo-206, I can see it in Enclosure View and host overview. I am now stuck , What next ? Was trying to play with "hardware Wizard" but beo-206 is not even listed there. Was playing with "manage sets" , create a new "Host Templates" but could not find any logical way to complete the node configuration BTW what is SM twin^2 1GB ? My node is 2 x 24 cores, how do I create a template ? how Do i automatically set up the hardware type for slurm or any template, how to create an image ? .... I would assume that my next step would be to list and register the node with the correct hardware, how to do that ? I would like to create a template for this node as I have about 20 of the same time. Need help , thanks Chris

1 0

Security/Bug fix releases 13.1.3, 12.0.3.2
by no-reply-list＠qlustar.org 15 Dec '23

15 Dec '23

The Qlustar releases 13.1.3 and 12.0.3.2 are ready including a number of security/bug fixes. Please check the following web pages for details about security fixes and special update instructions: https://qlustar.com/qsa/2023/1215231 https://qlustar.com/qsa/2023/1215232 The following non-security related updates/enhancements/bug fixes are included: For Qlustar 13 and 12.0: - Update to QluMan 13.4.7 - Update to Slurm 23.02.7

1 0

Security/Bug fix releases 13.1.2, 12.0.3.1
by no-reply-list＠qlustar.org 11 Dec '23

11 Dec '23

The Qlustar releases 13.1.2 and 12.0.3.1 are ready including a number of security/bug fixes. Please check the following web pages for details about security fixes and special update instructions: https://qlustar.com/qsa/2023/1209231 https://qlustar.com/qsa/2023/1209232 https://qlustar.com/qsa/2023/1209233 The following non-security related updates/enhancements/bug fixes are included: For Qlustar 13 and 12.0: - Update to QluMan 13.4.6 - Update to Slurm 23.0.6 For Qlustar 13 only: - Update to AlmaLinux 8.9 - Update to ZFS 2.1.14 (fixes a critical bug preventing data corruption) - Update to Nvidia graphics drivers 535.113.01 - Add support for current openfoam (.com and .org) versions in spack

1 0

host add example
by Christophe Guilbert 06 Dec '23

06 Dec '23

Hi, I was wondering if anyone wrote an easy step by step tutorial to add nodes from start to finish , from booting PXE node to the integration of this node in slurm. The documentation is very dense and I am not sure about the sequence of events Booting the nodes with PXE and catching the MAC/IP is obvious but what next is not (to me at least), how to create a node template which will be use to create an image and slurm entry? . I know that all those points are discuss in the doc but its hard to glue them together. The tutorial does not have to be big , just pointing the sequence of events and menus used so we have something to start with before getting into more details and fancy . ;) Thanks for any tips Chris

2 1

qluster new user question
by Joseph John 21 Nov '23

21 Nov '23

Dear All, Good afternoon I am Joseph John, I would like to try out Qluster for HPC setup, what I have is 4 old powerfull machines, each one with 250 GB OS disk 14 TB disk extra space and 48 CPUs , 250 GB disk space Now I am planning to make use of each 14 TB disk space together using glusterfs and each machine will boot from its own hard disk (250 GB) I have My doubt is is it necessary in qluster that, we boot from PXE for nodes , can I configure the nodes using the local available harddisk other than PXE boot Like to request for the URL of the documentation on how to setup the QLustar without using PXE boot Thanks Joseph John

2 1

installation problems
by Christophe Guilbert 16 Nov '23

16 Nov '23

Hi , I am completely new with qlustar. I would need some help to start having the cluster running as I don't really understand the doc. I did install qlustar (4 times) with no problem , Since I did not find out how to run the virtual FE node (and the demo mode VM by the way ). The last install was without create a virtual FE node. Following the section "1.5.5. Spack Package Manager" , it assume that I installed the FE node which I did not . There' s no alternative. I tried in vain to switch from root to user softadm in the main terminal , the user softadm is not even in the /etc/passwd. Under root I try my luck with the spack command leading to : spack compiler find -bash: spack: command not found How to do the spack section without creating the virtual FE node ? Also I don't mind reinstalling qlustar with the FE VM node but I would appreciate how to run it. Now here is my second question , how to run the demo VM ? Here is my frustration sequences of event in case someone want to rewrite that part of the doc ;-) demo-system -a start Starting VMs in a tmux session ... Created tmux session demo-system. => Attach with 'console-demo-vms' So far so good so I ran console-demo-vms Something happened and got a nice command line in the bottom of the terminal, but it looks like I am still on the front end , not in the demo VM , not only the prompt shows the front end hostname but htop command shows the whole ram of 32GB and CPU #, according to |/etc/qlustar/vm-configs/demo.conf| |it should be : | CN_MEM=1285 HA_MEM=1024 I am really confuse now, console-demo-vms did not show the demo VM , Any reason , what am I doing wrong here ? Now from the head node under root : 0 root@jcluster ~ # dsh -a uptime beo-201: ssh: connect to host beo-201 port 22: No route to host beo-202: ssh: connect to host beo-202 port 22: No route to host beo-203: ssh: connect to host beo-203 port 22: No route to host beo-204: ssh: connect to host beo-204 port 22: No route to host beo-201 are supposed to be the demo VMs, I have many more questions but that's it for now but planing forward, I did managed to run QluMan_gui, I played over night (until 5am) trying to add a working user and add a node . etc .. The doc for me for unusable, there's no tutorial online I could find (other than the first install which is very simple). I would be really appreciate if someone could describe the full procedure to add a node from catching the MAC address of the new node, building an image for that guy and include it in slurm. something to start with . Thanks for your help . I really would love to have qclusar working ASAP for my very urgent project research involving GPU Thanks Chris

2 8

2025

2024

2023

2022

2021

2020

2019

2018

Qlustar General