"J" == Jarett DeAngelis jarett@bioteam.net writes:
J> I mean it just isn't an option. The attached is what I get if I J> turn UEFI options off entirely. It doesn't recognize the drive as J> bootable. If I turn UEFI options on, the USB drive is there, but J> of course doesn't work.
Have you turned 'secure boot' off? If not please try that.
Secure Boot is turned off, yes.
Again, the puzzling thing is that I can easily boot off other ISOs written to USB with no issues.
J
On Wed, Dec 18, 2019, 3:06 AM Roland Fehrenbacher rf@q-leap.de wrote:
"J" == Jarett DeAngelis jarett@bioteam.net writes:
J> I mean it just isn't an option. The attached is what I get if I J> turn UEFI options off entirely. It doesn't recognize the drive as J> bootable. If I turn UEFI options on, the USB drive is there, but J> of course doesn't work.Have you turned 'secure boot' off? If not please try that. _______________________________________________ Qlustar-General mailing list -- qlustar-general@qlustar.org To unsubscribe send an email to qlustar-general-leave@qlustar.org
"J" == Jarett DeAngelis jarett@bioteam.net writes:
J> Secure Boot is turned off, yes. Again, the puzzling thing is J> that I can easily boot off other ISOs written to USB with no J> issues.
Sure that's interesting but unfortunately doesn't help. Many head-nodes were set up with this image, so it's something peculiar about your stick or the HP workstations. Do you have another type of machine to try this on?
Yes, I've also tried it on a OneMix 3S Yoga laptop. It doesn't appear bootable on that machine either. It has a pretty generic Aptio BIOS so I don't think anything "strange" would going on with its boot process. I've tried imaging both a 16GB USB2 stick and a 128GB USB3 stick with the dd method on the website and neither are bootable. J On Fri, 2019-12-20 at 18:43 +0100, Roland Fehrenbacher wrote:
"J" == Jarett DeAngelis jarett@bioteam.net writes:
J> Secure Boot is turned off, yes. Again, the puzzling thingis J> that I can easily boot off other ISOs written to USB with no J> issues. Sure that's interesting but unfortunately doesn't help. Many head- nodeswere set up with this image, so it's something peculiar about your stickor the HP workstations. Do you have another type of machine to try this on?_______________________________________________Qlustar-General mailing list -- qlustar-general@qlustar.org To unsubscribe send an email to qlustar-general-leave@qlustar.org
"J" == jarett jarett@bioteam.net writes:
J> Yes, I've also tried it on a OneMix 3S Yoga laptop. It doesn't J> appear bootable on that machine either. It has a pretty generic J> Aptio BIOS so I don't think anything "strange" would going on J> with its boot process. I've tried imaging both a 16GB USB2 stick J> and a 128GB USB3 stick with the dd method on the website and J> neither are bootable.
Hmm, I'm really running out of ideas now. Just downloaded the installer once more myself, burned the ISO on a stick, plugged it into my laptop and booted just fine from it. So the ISO is definitely OK. Hope you'll find the culprit.
J> On Fri, 2019-12-20 at 18:43 +0100, Roland Fehrenbacher wrote:
J> "J" == Jarett DeAngelis <
J> jarett@bioteam.net
>> writes:
J> Secure Boot is turned off, yes. Again, the puzzling thing is
J> that I can easily boot off other ISOs written to USB with no
J> issues.
J> Sure that's interesting but unfortunately doesn't help. Many J> head-nodes
J> were set up with this image, so it's something peculiar about J> your stick
J> or the HP workstations. Do you have another type of machine J> to try this on?
Hi all (especially Roland who has been so helpful),
I’ve resumed trying to troubleshoot this. Are we absolutely, positively sure the ISO on the website is set up correctly? Since this last email I have tried dd imaging it (after checking sha256sum) into several devices and booting on several machines. It won’t boot via legacy/BIOS on any of these machines. If I use the “alternative” method available from the Windows “Rufus” utility, it can get part of the way through booting before complaining of a missing EXFAT kernel module.
I have also tried burning the image to CD-R. When I booted from it I saw a number of errors related to bad sectors on the CD, but it eventually continued through the installation. After it was finished, rebooting to the installed hard drive didn’t work; it says it isn’t bootable, although the expected partition map seems to be there.
I’m going to try to burn it again in case I had a bad CD-R. I’ve tried to do it via my Mac, though, and it says the ISO image is invalid. This raises a red flag for me. Are there alternate installation methods I can try?
Thanks, Jarett
On Dec 21, 2019, at 8:26 AM, Roland Fehrenbacher rf@q-leap.de wrote:
"J" == jarett jarett@bioteam.net writes:
J> Yes, I've also tried it on a OneMix 3S Yoga laptop. It doesn't J> appear bootable on that machine either. It has a pretty generic J> Aptio BIOS so I don't think anything "strange" would going on J> with its boot process. I've tried imaging both a 16GB USB2 stick J> and a 128GB USB3 stick with the dd method on the website and J> neither are bootable.
Hmm, I'm really running out of ideas now. Just downloaded the installer once more myself, burned the ISO on a stick, plugged it into my laptop and booted just fine from it. So the ISO is definitely OK. Hope you'll find the culprit.
J> On Fri, 2019-12-20 at 18:43 +0100, Roland Fehrenbacher wrote:
J> "J" == Jarett DeAngelis <
J> jarett@bioteam.net
writes:
J> Secure Boot is turned off, yes. Again, the puzzling thing is
J> that I can easily boot off other ISOs written to USB with no
J> issues.
J> Sure that's interesting but unfortunately doesn't help. Many J> head-nodes
J> were set up with this image, so it's something peculiar about J> your stick
J> or the HP workstations. Do you have another type of machine J> to try this on?
Qlustar-General mailing list -- qlustar-general@qlustar.org To unsubscribe send an email to qlustar-general-leave@qlustar.org
"J" == Jarett DeAngelis jarett@bioteam.net writes:
Hi Jarett,
welcome back. I kind of hoped that this strange problem would dissolve by itself, but it seems to be stubborn :)
J> Hi all (especially Roland who has been so helpful), I’ve resumed J> trying to troubleshoot this. Are we absolutely, positively sure J> the ISO on the website is set up correctly?
Yes. It must be a special issue of your hardware. Too many successful installs, and so far, your case is the only one I'm aware of with this kind of problem.
J> Since this last email I have tried dd imaging it (after checking J> sha256sum) into several devices and booting on several J> machines. It won’t boot via legacy/BIOS on any of these machines.
That's really weird. What type of machines were they? Do you have any Supermicro server by any chance where you can try it? That's the most common platform Qlustar is used on.
J> If I use the “alternative” method available from the Windows J> “Rufus” utility, it can get part of the way through booting J> before complaining of a missing EXFAT kernel module.
I can't comment about Rufus.
J> I have also tried burning the image to CD-R. When I booted from J> it I saw a number of errors related to bad sectors on the CD, but J> it eventually continued through the installation. After it was J> finished, rebooting to the installed hard drive didn’t work; it J> says it isn’t bootable, although the expected partition map seems J> to be there.
I don't know what harm the bad sectors could have done, but if the installation went through and the BIOS is setup correctly for legacy, it should probably boot. Burning another DVD might be safer though. Have you checked whether your machines have the latest BIOS? There might be a flaw in their legacy boot mode ...
J> I’m going to try to burn it again in case I had a bad CD-R. I’ve J> tried to do it via my Mac, though, and it says the ISO image is J> invalid. This raises a red flag for me. Are there alternate J> installation methods I can try?
On some servers, there is the possibility to boot a remote ISO via IPMI, don't know whether you're machine supports that.
One other thing you could try is to install Ubuntu on your headnode and then Qlustar within a VM through kvm on top of it. While you will loose some performance (not too much these days), for evaluation purposes this might be sufficient.
Best,
Roland
I am having nearly the exact same issue.
I have many HP Z420 systems I have tried this with. They can all install ANY other OS I have tried via USB in Legacy mode (Non-UEFI). I have built the USB with Rufus, Win32diskImager, dd mode inside of CentOS as well as through Windows, UniversalUSB installer, Unetbootin, Xboot, Etcher, and several other not listed here. It's not the USB drive, as I've used SEVERAL different sticks. I have re-used these sticks after trying Qlustar, to install other OS's, and they still work perfectly fine, and can load the vfat module when installing. Currently, I've built a Rocks7 cluster from these various USB sticks with NO issues. Hell, I've even tried to restore the thumb as a HDD from the qlustar image inside of CentOS to no luck.
The error is simple: run/initramfs/live unknown filesystem type vfat
I've changed the format of the USB to fat16, fat32, NTFS. I've used MBR and GPT partitioning to no avail. I even tried to format the USB as ext2 and ext3, made it bootable, active partition, and then write the iso to the thumbdrive, and still doesn't work (various other reasons as well have occured through these other methods too).
I have searched the internet for many hours to see if I can find a way to fix this. I've tried to exit to rescue disk, and see if i can mount -f or modprobe, but the vfat, or ntfs modules are not loaded to the kernel to allow for this. There has been a suggestion that the order of startup within this kernel is wrong and it's trying to mount the USB drive before it's had a chance to load the module(s) for the USB.
I've also tried this on various sized thumbs... 4G up to 64G... not it.
I've also tried to build a CD/DVD to boot from, and I run into the exact same issue: unknown filesystem type vfat.
It cannot be just you, Jarett. I have no idea where to go from here, as I've tried everything I can find to fix this, other than building this headnode with Ubuntu, then build a new kernel mkinitnfo.... and see if I can get it to hopefully include the vfat module so I can install qlustar.
I might consider trying the Ubuntu build, then maybe, possibly, create a VM inside of Ubuntu, and pray that it can try to install qlustar there, but, it seems unlikely to work, imo.
Any ideas would be great, if someone else knows what might be the issue. I have 1 more non-HP system I can try to install this on, but regardless of the make/model of system, SO FAR, has been unsuccessful.
"J" == jeremy jeremy@kthoris.com writes:
Hi Jeremy,
J> I am having nearly the exact same issue.
not quite. Jarett's problem is that his machines won't even start booting from the flash drive.
J> I have many HP Z420 systems I have tried this with. They can all J> install ANY other OS I have tried via USB in Legacy mode J> (Non-UEFI). I have built the USB with Rufus, Win32diskImager, dd J> mode inside of CentOS as well as through Windows, UniversalUSB J> installer, Unetbootin, Xboot, Etcher, and several other not J> listed here. It's not the USB drive, as I've used SEVERAL J> different sticks. I have re-used these sticks after trying J> Qlustar, to install other OS's, and they still work perfectly J> fine, and can load the vfat module when installing. Currently, J> I've built a Rocks7 cluster from these various USB sticks with NO J> issues. Hell, I've even tried to restore the thumb as a HDD from J> the qlustar image inside of CentOS to no luck.
J> The error is simple: run/initramfs/live unknown filesystem type J> vfat
Sounds as if you're using an old version of the installer. Can you please double-check that you have 11.0.0-3? In any case, the vfat error message is misleading (it's known to appear with any installer < 11.0.x), but is not fatal by itself. It's just the last message one sees in those cases that we encountered sometimes with older installers. The vfat module is definitely on the 11.0.0-3 installer stick.
J> I've changed the format of the USB to fat16, fat32, NTFS. I've J> used MBR and GPT partitioning to no avail. I even tried to J> format the USB as ext2 and ext3, made it bootable, active J> partition, and then write the iso to the thumbdrive, and still J> doesn't work (various other reasons as well have occured through J> these other methods too).
J> I have searched the internet for many hours to see if I can find J> a way to fix this. I've tried to exit to rescue disk, and see if J> i can mount -f or modprobe, but the vfat, or ntfs modules are not J> loaded to the kernel to allow for this. There has been a J> suggestion that the order of startup within this kernel is wrong J> and it's trying to mount the USB drive before it's had a chance J> to load the module(s) for the USB.
J> I've also tried this on various sized thumbs... 4G up to J> 64G... not it.
J> I've also tried to build a CD/DVD to boot from, and I run into J> the exact same issue: unknown filesystem type vfat.
Yes, booting the same thing from a different media won't help.
J> It cannot be just you, Jarett.
Other cases of not being able to boot at all are not known so far, but again your problem is different.
J> I have no idea where to go from here, as I've tried everything I J> can find to fix this, other than building this headnode with J> Ubuntu, then build a new kernel mkinitnfo.... and see if I can J> get it to hopefully include the vfat module so I can install J> qlustar.
J> I might consider trying the Ubuntu build, then maybe, possibly, J> create a VM inside of Ubuntu, and pray that it can try to install J> qlustar there, but, it seems unlikely to work, imo.
J> Any ideas would be great, if someone else knows what might be the J> issue. I have 1 more non-HP system I can try to install this on, J> but regardless of the make/model of system, SO FAR, has been J> unsuccessful.
See above.
Best,
Roland
J> I have many HP Z420 systems I have tried this with. They can all J> install ANY other OS I have tried via USB in Legacy mode J> (Non-UEFI). I have built the USB with Rufus, Win32diskImager, dd J> mode inside of CentOS as well as through Windows, UniversalUSB J> installer, Unetbootin, Xboot, Etcher, and several other not J> listed here. It's not the USB drive, as I've used SEVERAL J> different sticks. I have re-used these sticks after trying J> Qlustar, to install other OS's, and they still work perfectly J> fine, and can load the vfat module when installing. Currently, J> I've built a Rocks7 cluster from these various USB sticks with NO J> issues. Hell, I've even tried to restore the thumb as a HDD from J> the qlustar image inside of CentOS to no luck.
J> The error is simple: run/initramfs/live unknown filesystem type J> vfat
J> I've also tried to build a CD/DVD to boot from, and I run into J> the exact same issue: unknown filesystem type vfat.
Yes, booting the same thing from a different media won't help.
So, I wanted to update everyone on this. The current state of affairs on my three Skylake HP desktops that I’ve been trying to use as cluster nodes is as follows: Trying to boot from a dd-imaged USB in legacy mode: drive isn’t even recognized as bootable; skips right over it in the Boot Menu list Trying to boot from a dd-imaged USB in UEFI mode: drive recognized but returns you to the Boot Menu as soon as you try to boot from it Trying to boot from a Rufus-imaged USB (with “ISO mode”) in legacy mode: drive isn’t recognized Trying to boot from a Rufus-imaged USB (with “ISO mode”) in UEFI mode: gets to a certain point in the installer, then does what Jeremy is reporting with the VFAT driver error
At the end of my rope, I finally tried booting from a CD-R. The first time I tried burning it, the disc was rife with errors and the system wouldn’t boot properly after installation. The second time seemed to work and I was able to complete head node installation, but I couldn’t get compute nodes to bootstrap correctly. I may try this again tonight, because I found a way to boot from USB:
I tried installing Bright not long after this (which you all can try now if you have 8 compute nodes or fewer: https://community.brightcomputing.com/feed/easy8-users-getting-started https://community.brightcomputing.com/feed/easy8-users-getting-started). I started out with exactly the same problem as the Qlustar installer. I beat my head against the problem some more and was finally able to get all the way through the installation process by using the YUMI utility ( https://www.pendrivelinux.com/yumi-multiboot-usb-creator/ https://www.pendrivelinux.com/yumi-multiboot-usb-creator/ ) and telling it to place the boot image in “partition 4” on the USB as listed in the imaging options. Why this is a thing that matters, I have no idea. Maybe someone else has a clue as to why starting the installer from inside the YUMI boot environment and choosing this extra partition option seems to work where no other option does.
So, Jeremy, I think our problems are related, but not identical. I would be willing to bet, though, that something about what HP does with desktop (or workstation) BIOSes causes this problem. I’ve scoured the internet for search terms like “HP cannot boot Linux” but while a number of people seem to have this issue or something related to it, no one seems to have solutions. https://h30434.www3.hp.com/t5/Notebook-Boot-and-Lockup/Cannot-boot-from-usb-... https://h30434.www3.hp.com/t5/Notebook-Boot-and-Lockup/Cannot-boot-from-usb-flash-drive-to-install-linux/td-p/6251826
Really, really strange stuff. But — give YUMI a try and please report back whether it worked for you.
Thanks, Jarett
Another update here: I remember now what my original problem was with Qlustar after installation: it wouldn't boot. No matter what combination of BIOS tweaks I tried, I got "no bootable disk detected" from my system. Just as a lark, I tried booting from the YUMI key I used to install Qlustar this go-around and told it "boot from the first hard disk," which is actually the default option. Worked like a charm. What this means is that I can only boot the system with this YUMI USB inserted. I tried having it reinstall grub on the SSD. Nope. Still only boots with the YUMI key as a "first stage."
So, Roland and Jeremy -- something Qlustar does with its bootloader, I think, is incompatible with something weird in the HP (not HPE) BIOSes. It isn't just HP: I got the same error trying to boot the Qlustar installer from my OneMix 3S laptop, which has a pretty minimal Aptio BIOS.
Anyway, another datapoint for us to mull over. I will start another thread about my latest round of Qlustar questions ;)
"J" == Jarett DeAngelis jarett@bioteam.net writes:
J> At the end of my rope, I finally tried booting from a CD-R. The J> first time I tried burning it, the disc was rife with errors and J> the system wouldn’t boot properly after installation. The second J> time seemed to work and I was able to complete head node J> installation,
That's already good news.
J> but I couldn’t get compute nodes to bootstrap correctly. I may J> try this again tonight, because I found a way to boot from USB:
J> I beat my head against the problem some more and was finally able J> to get all the way through the installation process by using the J> YUMI utility ( J> https://www.pendrivelinux.com/yumi-multiboot-usb-creator/ ) and J> telling it to place the boot image in “partition 4” on the USB as J> listed in the imaging options. Why this is a thing that matters, J> I have no idea. Maybe someone else has a clue as to why starting J> the installer from inside the YUMI boot environment and choosing J> this extra partition option seems to work where no other option J> does.
J> So, Jeremy, I think our problems are related, but not J> identical. I would be willing to bet, though, that something J> about what HP does with desktop (or workstation) BIOSes causes J> this problem. I’ve scoured the internet for search terms like “HP J> cannot boot Linux” but while a number of people seem to have this J> issue or something related to it, no one seems to have solutions. J> https://h30434.www3.hp.com/t5/Notebook-Boot-and-Lockup/Cannot-boot-from-usb-...
It seems rather clear to me now that 'UEFI Compatibility Mode' booting from USB or hard disk is really broken on these HP machines and apparently also on others. As to YUMI: I assume YUMI is doing some kind of chain loading, so that the sticks support UEFI booting but are also capable of starting non UEFI OS media in a second stage.
Full UEFI support for Qlustar (installer and netboot) is in the pipeline for 2020 and this should solve all these and related issues.
Best,
Roland
-
Jarett DeAngelis -
jarett＠bioteam.net
-
jeremy＠kthoris.com -
Roland Fehrenbacher