"F" == marra@irc cnr it marra@irc.cnr.it writes:
Hi Franco,
it looks as if the IB network might not be exported in /etc/exports. Also please check whether you have set NEED_RDMA="yes" in /etc/default/nfs-kernel-server. This is needed to enable NFSoRDMA.
If you don't want to use NFSoRDMA, you can remove the IB network from your Filesystem Exports resources in QluMan alltogether or just uncheck "Allow RDMA" in the FS mounts definitions. In the latter case, the IB network can be used for NFS but only with TCP/IP rather than RDMA.
Best,
Roland
F> Dear Roland, firstly I like to thank you so much for your kind F> and detailed answer, that allowed me to understand more details F> about this very nice Qlustar distribution and to investigate F> better my issue. I hope I will not bore you too much with my F> reply.
F> When I try to login as a simple user from the frontend to a F> compute node (standard-node) I get a password request and the nfs F> directories are not mounted. These are the relevant lines of the F> output of the journalctl -xe command:
F> Jul 25 12:16:34 HP4 systemd[1]: data-home.automount: Got F> automount request for /data/home, triggered by 38058 (sshd) Jul F> 25 12:16:34 HP4 systemd[1]: Mounting Mount point /data/home... F> Jul 25 12:16:34 HP4 kernel: RPC: Registered rdma transport F> module. Jul 25 12:16:34 HP4 kernel: RPC: Registered rdma F> backchannel transport module. Jul 25 12:16:34 HP4 mount[38060]: F> mount.nfs: access denied by server while mounting F> beosrv-ib:/srv/data/home Jul 25 12:16:34 HP4 systemd[1]: F> data-home.mount: Mount process exited, code= exited status=32 Jul F> 25 12:16:34 HP4 systemd[1]: data-home.mount: Failed with result F> 'exit-code'. Jul 25 12:16:34 HP4 systemd[1]: Failed to mount F> Mount point /data/home. Jul 25 12:16:34 HP4 systemd[1]: F> data-home.automount: Got automount request for /data/home, F> triggered by 38058 (sshd) Jul 25 12:16:34 HP4 systemd[1]: F> Mounting Mount point /data/home... Jul 25 12:16:34 HP4 F> mount[38065]: mount.nfs: access denied by server while mounting F> beosrv-ib:/srv/data/home Jul 25 12:16:34 HP4 systemd[1]: F> data-home.mount: Mount process exited, code= exited status=32 Jul F> 25 12:16:34 HP4 systemd[1]: data-home.mount: Failed with result F> 'exit-code'. Jul 25 12:16:34 HP4 systemd[1]: Failed to mount F> Mount point /data/home. Jul 25 12:16:34 HP4 systemd[1]: F> data-home.automount: Got automount request for /data/home, F> triggered by 38058 (sshd) Jul 25 12:16:34 HP4 systemd[1]: F> Mounting Mount point /data/home... Jul 25 12:16:34 HP4 F> mount[38067]: mount.nfs: access denied by server while mounting F> beosrv-ib:/srv/data/home Jul 25 12:16:34 HP4 systemd[1]: F> data-home.mount: Mount process exited, code= exited status=32 Jul F> 25 12:16:34 HP4 systemd[1]: data-home.mount: Failed with result F> 'exit-code'. Jul 25 12:16:34 HP4 systemd[1]: Failed to mount F> Mount point /data/home. Jul 25 12:16:34 HP4 sshd[38058]: Rhosts F> authentication refused for marra: no home directory F> /data/home/marra
F> From this I understand that the nfs request is for the Infiniband F> interface of the headnode. However, I have a virtual front end F> that miss an IB interface, and the Filesystem exports config F> looks like:
F> Name: Home Server: beosrv-c Export Path: /srv/data/home Network F> priorities: Boot IB
F> I do not know how the priority is exactly managed, but the mount F> command on the VM-FE show me:
F> beosrv-c:/srv/data/home on /data/home type nfs ...
F> so I am sure the home directory is mounted on the ethernet F> network. Is it normal to mix mounting options for FE and standard F> nodes?
F> The network FS Mounts dialog for the same directory in Qluman-qt F> looks like:
F> Resource: Home Export Path: /srtv/data/home [blank] [ ] Override F> Network: (grayed Boot) [X] Allow RDMA
F> The preview config of the node HP4 shows me some alerts (red dots F> or green/red dots) for the following points:
F> (RED/GREEN) /etc -> (RED) Network -> (RED) interfaces.d/qluman F> (I suppose this is not relevant to my issue) F> ################################################################## F> #------------- File is auto-generated by Qluman! F> #-------------# ------------- Manual changes will be F> #overwritten! -------------# F> #----------------------------------------------------------------#
F> auto BOOT iface BOOT inet dhcp metric 10
F> auto ib0 iface ib0 inet static address 192.168.53.104 netmask F> 24 pre-up /lib/qlustar/ib-initialize
F> (RED/GREEN) /etc -> (RED/GREEN) qlustar -> (RED) Disk config F> # ZFS config for single disk (/dev/sda): F> # Zpool name: SYS 8GB zvol for swap (not activated) F> #Filesystems: /var (max 2GB) + /scratch - both compressed
F> [BASE] ZPOOLS = SYS ZFS = var, scratch F> #ARC_LIMIT = 1024 ZVOLS = swap
F> [SYS] vdevs = V-SYS
F> [V-SYS] devs = /dev/sda type =
F> [swap] zpool = SYS size = 8G
F> [var] zpool = SYS quota = 20G reservation = 20G compress = lz4
F> [scratch] zpool = SYS compress = lz4
F> (RED/GREEN) sysconfig -> (RED/GREEN) network-scripts -> (RED) F> ifcfg-BOOT
F> ################################################################## F> #------------- File is auto-generated by Qluman! F> #-------------# ------------- Manual changes will be F> #overwritten! -------------# F> #----------------------------------------------------------------# F> DEVICE=BOOT BOOTPROTO=dhcp ONBOOT=yes TYPE=Ethernet F> HWADDR=a0:d3:c1:fd:9c:a8
F> Maybe a solution could be to delete the IB network from the F> config of the Filesystem exports so to be sure to be consistent F> with the network protocol both for the VM-FE and the nodes.
F> If you have time to give me your hints, I will really appreciate F> your help.
F> Thank you and best regards,
F> Franco