Dear Qlustar Team,
I apologise if this isn't the appropriate channel for such report. If there's a better place to direct this question, please let me know.
I believe I've encountered a bug with the recent update of Spack in the Qlustar HPC Core Stack 24.06.
After upgrading my Qlustar system (both chroot and head-node, using 'apt update' followed by 'apt upgrade'), I started receiving the following error when attempting to install packages with Spack:
==> Error: Package 'spack.pkg.builtin.zlib-api' not found. Use 'spack create' to create a new package.
Did you mean one of the following packages? libkcapi grib-api libcap
Upon investigation, I found a relevant change in Spack itself: https://github.com/spack/spack/commit/e51748ee8f89e0d3db4e426e3d04157129a456...
This change introduces a new virtual 'zlib-api', which replaces 'zlib' in most packages to allow users to switch to zlib-ng by default for better performance.
However, I noticed that the /etc/spack/defaults/package.json file was missing the definition of this virtual library. I added the line "zlib-api": ["zlib", "zlib-ng+compat"] to the file, but I'm still encountering the same error.
What am I missing? How can I resolve this issue?
For reproducibility purposes: - Spack version: 0.22.0 (previous version before the update 0.20) - Command I'm running: spack install r
Furthermore, I believe this would affect most of the installation packages in Spack, since zlib is used in alot of them.
Thank you for your assistance.
Best regards, Tiago Almeida
Dear Tiago,
we cannot reproduce this, get no error about spack.pkg.builtin.zlib-api. However, the build or R fails because of an error in the gettext dependency:
7588 In file included from term-styled-ostream.oo.c:26: >> 7589 ./libcroco/cr-sel-eng.h:43:10: fatal error: libxml/tree.h: No such file or directory 7590 43 | #include <libxml/tree.h> 7591 | ^~~~~~~~~~~~~~~ 7592 compilation terminated. >> 7593 make[4]: *** [Makefile:6192: config.h] Error 1 7594 make[4]: Leaving directory '/tmp/softadm/spack-stage/spack-stage-gett ext-0.22.5-lsirwkacds7qvk6gdtsjf4mn4heft4ji/spack-src/libtextstyle/lib'
This is something that should be fixed upstream. You might succeed by building an older version.
Best,
Roland
On 7/18/24 12:35, tiagomeloalmeida--- via Qlustar General wrote:
Dear Qlustar Team,
I apologise if this isn't the appropriate channel for such report. If there's a better place to direct this question, please let me know.
I believe I've encountered a bug with the recent update of Spack in the Qlustar HPC Core Stack 24.06.
After upgrading my Qlustar system (both chroot and head-node, using 'apt update' followed by 'apt upgrade'), I started receiving the following error when attempting to install packages with Spack:
==> Error: Package 'spack.pkg.builtin.zlib-api' not found. Use 'spack create' to create a new package.
Did you mean one of the following packages? libkcapi grib-api libcap
Upon investigation, I found a relevant change in Spack itself: https://github.com/spack/spack/commit/e51748ee8f89e0d3db4e426e3d04157129a456...
This change introduces a new virtual 'zlib-api', which replaces 'zlib' in most packages to allow users to switch to zlib-ng by default for better performance.
However, I noticed that the /etc/spack/defaults/package.json file was missing the definition of this virtual library. I added the line "zlib-api": ["zlib", "zlib-ng+compat"] to the file, but I'm still encountering the same error.
What am I missing? How can I resolve this issue?
For reproducibility purposes:
- Spack version: 0.22.0 (previous version before the update 0.20)
- Command I'm running: spack install r
Furthermore, I believe this would affect most of the installation packages in Spack, since zlib is used in alot of them.
Thank you for your assistance.
Best regards, Tiago Almeida _
Dear Roland,
Thanks for your reply,
If you are unable to reproduce the issue, could you please provide me with some information to help me understand the source of the problem?
Firstly, could you outline your steps for upgrading the system packages? I'm concerned that I might have done something incorrectly or that I'm missing a step.
Additionally, could you please verify if your /etc/spack/defaults/packages.yaml file on the login node has the definition for the "zlib-api" virtual library? It should look something like this: "zlib-api": ["zlib", "zlib-ng+compat"] or "zlib-api": ["zlib-ng+compat", "zlib"]
Furthermore, what is the output of your "spack providers" command on the login node when run as softadm? Mine is the following:
$ spack providers Virtual packages: D daal flame glu iconv jpeg libllvm luajit mpe onedal pil scalapack tbb xxd awk elf fuse go-or-gccgo-bootstrap ipp lapack libosmesa mariadb-client mpi opencl pkgconfig sycl unwind yacc blas fftw-api gl golang java libglx lua-lang mkl mysql-client pbs rpc szip uuid ziglang
Although I added "zlib-api": ["zlib-ng+compat", "zlib"] to the /etc/spack/defaults/packages.yaml file, the zlib-api does not appear in the virtual packages list in my output.
Also, could you please show me the beginning of your /var/lib/spack/repos/builtin/packages/r/package.py file, also on the login node? For instance, this is mine:
--------------------------------- $ cat /var/lib/spack/repos/builtin/packages/r/package.py # Copyright 2013-2024 Lawrence Livermore National Security, LLC and other # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT)
import os
from spack.package import *
class R(AutotoolsPackage): """R is 'GNU S', a freely available language and environment for statistical computing and graphics which provides a wide variety of statistical and graphical techniques: linear and nonlinear modelling, statistical tests, time series analysis, classification, clustering, etc. Please consult the R project homepage for further information."""
homepage = "https://www.r-project.org" url = "https://cloud.r-project.org/src/base/R-4/R-4.4.0.tar.gz"
extendable = True
license("GPL-2.0-or-later")
version("4.4.0", sha256="ace4125f9b976d2c53bcc5fca30c75e30d4edc401584859cbadb080e72b5f030") version("4.3.3", sha256="80851231393b85bf3877ee9e39b282e750ed864c5ec60cbd68e6e139f0520330") version("4.3.2", sha256="b3f5760ac2eee8026a3f0eefcb25b47723d978038eee8e844762094c860c452a") version("4.3.1", sha256="8dd0bf24f1023c6f618c3b317383d291b4a494f40d73b983ac22ffea99e4ba99") version("4.3.0", sha256="45dcc48b6cf27d361020f77fde1a39209e997b81402b3663ca1c010056a6a609") version("4.2.3", sha256="55e4a9a6d43be314e2c03d0266a6fa5444afdce50b303bfc3b82b3979516e074") version("4.2.2", sha256="0ff62b42ec51afa5713caee7c4fde7a0c45940ba39bef8c5c9487fef0c953df5") version("4.2.1", sha256="4d52db486d27848e54613d4ee977ad952ec08ce17807e1b525b10cd4436c643f") version("4.2.0", sha256="38eab7719b7ad095388f06aa090c5a2b202791945de60d3e2bb0eab1f5097488") version("4.1.3", sha256="15ff5b333c61094060b2a52e9c1d8ec55cc42dd029e39ca22abdaa909526fed6") version("4.1.2", sha256="2036225e9f7207d4ce097e54972aecdaa8b40d7d9911cd26491fac5a0fab38af") version("4.1.1", sha256="515e03265752257d0b7036f380f82e42b46ed8473f54f25c7b67ed25bbbdd364") version("4.1.0", sha256="e8e68959d7282ca147360fc9644ada9bd161bab781bab14d33b8999a95182781") version("4.0.5", sha256="0a3ee079aa772e131fe5435311ab627fcbccb5a50cabc54292e6f62046f1ffef") version("4.0.4", sha256="523f27d69744a08c8f0bd5e1e6c3d89a4db29ed983388ba70963a3cd3a4a802e") version("4.0.3", sha256="09983a8a78d5fb6bc45d27b1c55f9ba5265f78fa54a55c13ae691f87c5bb9e0d") version("4.0.2", sha256="d3bceab364da0876625e4097808b42512395fdf41292f4915ab1fd257c1bbe75") version("4.0.1", sha256="95fe24a4d8d8f8f888460c8f5fe4311cec656e7a1722d233218bc03861bc6f32") version("4.0.0", sha256="06beb0291b569978484eb0dcb5d2339665ec745737bdfb4e873e7a5a75492940") version("3.6.3", sha256="89302990d8e8add536e12125ec591d6951022cf8475861b3690bc8bf1cefaa8f") version("3.6.2", sha256="bd65a45cddfb88f37370fbcee4ac8dd3f1aebeebe47c2f968fd9770ba2bbc954") version("3.6.1", sha256="5baa9ebd3e71acecdcc3da31d9042fb174d55a42829f8315f2457080978b1389") version("3.6.0", sha256="36fcac3e452666158e62459c6fc810adc247c7109ed71c5b6c3ad5fc2bf57509") version("3.5.3", sha256="2bfa37b7bd709f003d6b8a172ddfb6d03ddd2d672d6096439523039f7a8e678c") version("3.5.2", sha256="e53d8c3cf20f2b8d7a9c1631b6f6a22874506fb392034758b3bb341c586c5b62") version("3.5.1", sha256="0463bff5eea0f3d93fa071f79c18d0993878fd4f2e18ae6cf22c1639d11457ed") version("3.5.0", sha256="fd1725535e21797d3d9fea8963d99be0ba4c3aecadcf081b43e261458b416870") version("3.4.4", sha256="b3e97d2fab7256d1c655c4075934725ba1cd7cb9237240a11bb22ccdad960337") version("3.4.3", sha256="7a3cb831de5b4151e1f890113ed207527b7d4b16df9ec6b35e0964170007f426") version("3.4.2", sha256="971e30c2436cf645f58552905105d75788bd9733bddbcb7c4fbff4c1a6d80c64") version("3.4.1", sha256="02b1135d15ea969a3582caeb95594a05e830a6debcdb5b85ed2d5836a6a3fc78") version("3.4.0", sha256="288e9ed42457c47720780433b3d5c3c20983048b789291cc6a7baa11f9428b91") version("3.3.3", sha256="5ab768053a275084618fb669b4fbaadcc39158998a87e8465323829590bcfc6c") version("3.3.2", sha256="d294ad21e9f574fb4828ebb3a94b8cb34f4f304a41687a994be00dd41a4e514c") version("3.3.1", sha256="3dc59ae5831f5380f83c169bac2103ad052efe0ecec4ffa74bde4d85a0fda9e2") version("3.3.0", sha256="9256b154b1a5993d844bee7b1955cd49c99ad72cef03cce3cd1bdca1310311e4") version("3.2.5", sha256="60745672dce5ddc201806fa59f6d4e0ba6554d8ed78d0f9f0d79a629978f80b5") version("3.2.3", sha256="b93b7d878138279234160f007cb9b7f81b8a72c012a15566e9ec5395cfd9b6c1") version("3.2.2", sha256="9c9152e74134b68b0f3a1c7083764adc1cb56fd8336bec003fd0ca550cd2461d") version("3.2.1", sha256="d59dbc3f04f4604a5cf0fb210b8ea703ef2438b3ee65fd5ab536ec5234f4c982") version("3.2.0", sha256="f5ae953f18ba6f3d55b46556bbbf73441350f9fd22625402b723a2b81ff64f35") version("3.1.3", sha256="07e98323935baa38079204bfb9414a029704bb9c0ca5ab317020ae521a377312") version("3.1.2", sha256="bcd150afcae0e02f6efb5f35a6ab72432be82e849ec52ce0bb89d8c342a8fa7a")
variant("X", default=False, description="Enable X11 support (TCLTK, PNG, JPEG, TIFF, CAIRO)") variant("memory_profiling", default=False, description="Enable memory profiling") variant("rmath", default=False, description="Build standalone Rmath library")
depends_on("blas") depends_on("lapack")
depends_on("bzip2") depends_on("curl+libidn2") # R didn't anticipate the celebratory non-breaking major version bump of curl 8. depends_on("curl@:7", when="@:4.2") depends_on("icu4c") depends_on("java") depends_on("libtirpc") depends_on("ncurses") depends_on("pcre", when="@:3.6.3") depends_on("pcre2", when="@4:") depends_on("readline") depends_on("xz") depends_on("which", type=("build", "run")) depends_on("zlib-api") depends_on("zlib@1.2.5:", when="^[virtuals=zlib-api] zlib") depends_on("texinfo", type="build") --------------------- CUT the output
As you can see, my most recent version is 4.4.0 and it depends_on("zlib-api"), which for some reason is not being recognized in my system.
Best regards, Tiago Almeida
Hi again,
I wanted to provide a quick update: I've figured out how to fix the problem.
Here's what I did: spack clean -a spack providers
I believe what happened was that my Spack installation was using a build configuration cached in "/usr/share/spack/root/opt/.user/cache/cache/providers/builtin-index.json". As a result, any changes I made to my packages.yaml file weren't being applied. The first command cleans this cache along with temporary build files and downloads. The second command then rebuilds the builtin-index.json using my current version of packages.yaml.
I'm still unsure why my upgrade didn't include the zlib-api in packages.yaml, but that's not crucial for now.
Best regards, Tiago Almeida
-
Roland Fehrenbacher -
tiagomeloalmeida＠ua.pt