Sitelet https://github.com/nodejs/node/issues/48008
Skip to content

Included zlib is built with optional instructions on aarch64  #48008

Closed as not planned
Closed as not planned
Included zlib is built with optional instructions on aarch64 #48008

Description

@JulianGro
JulianGro
opened on May 14, 2023

Version

18.16.0 and 18.15.0

Platform

Linux c9869ce0a7f0 5.15.76-v8+ #1597 SMP PREEMPT Fri Nov 4 12:16:41 GMT 2022 aarch64 aarch64 aarch64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

No response

How often does it reproduce? Is there a required condition?

No response

What is the expected behavior? Why is that the expected behavior?

It should check the host CPU for the available instructions and/or there should be a command line switch for building without aes instructions.

What do you see instead?

The included Zlib is hardcoded to be built with -march=armv8-a+aes+crc on AArch64. Common AArch64 devices such as the Raspberry Pi 4 don't support aes instructions.

Additional information

Changing this in line 69 (v18.16.0) of deps/zlib/zlib.gyp to

                  'cflags': [ '-march=armv8-a+crc' ],

throws Assembler errors:

  cc -o /node/out/Release/obj.target/zlib_adler32_simd/deps/zlib/adler32_simd.o ../deps/zlib/adler32_simd.c '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_GLIBCXX_USE_CXX11_ABI=1' '-DNODE_OPENSSL_CONF_NAME=nodejs_conf' '-DNODE_OPENSSL_HAS_QUIC' '-DICU_NO_USER_DATA_OVERRIDE' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_THREADS' '-DADLER32_SIMD_NEON' -I../deps/zlib  -msign-return-address=all -pthread -Wall -Wextra -Wno-unused-parameter -fPIC -O3 -fno-omit-frame-pointer  -MMD -MF /node/out/Release/.deps//node/out/Release/obj.target/zlib_adler32_simd/deps/zlib/adler32_simd.o.d.raw   -c
cc: warning: switch '-msign-return-address=all' is no longer supported
  cc -o /node/out/Release/obj.target/zlib_arm_crc32/deps/zlib/crc32_simd.o ../deps/zlib/crc32_simd.c '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_GLIBCXX_USE_CXX11_ABI=1' '-DNODE_OPENSSL_CONF_NAME=nodejs_conf' '-DNODE_OPENSSL_HAS_QUIC' '-DICU_NO_USER_DATA_OVERRIDE' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_THREADS' '-DCRC32_ARMV8_CRC32' '-DARMV8_OS_LINUX' -I../deps/zlib  -msign-return-address=all -pthread -Wall -Wextra -Wno-unused-parameter -fPIC -march=armv8-a+crc -O3 -fno-omit-frame-pointer  -MMD -MF /node/out/Release/.deps//node/out/Release/obj.target/zlib_arm_crc32/deps/zlib/crc32_simd.o.d.raw   -c
cc: warning: switch '-msign-return-address=all' is no longer supported
  g++ -o /node/out/Release/obj.target/gtest/deps/googletest/src/gtest-assertion-result.o ../deps/googletest/src/gtest-assertion-result.cc '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_GLIBCXX_USE_CXX11_ABI=1' '-DNODE_OPENSSL_CONF_NAME=nodejs_conf' '-DNODE_OPENSSL_HAS_QUIC' '-DICU_NO_USER_DATA_OVERRIDE' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_THREADS' '-DGTEST_HAS_POSIX_RE=0' '-DGTEST_LANG_CXX11=1' -I../deps/googletest -I../deps/googletest/include  -msign-return-address=all -pthread -Wall -Wextra -Wno-unused-parameter -fPIC -O3 -fno-omit-frame-pointer -fno-rtti -fno-exceptions -std=gnu++17 -MMD -MF /node/out/Release/.deps//node/out/Release/obj.target/gtest/deps/googletest/src/gtest-assertion-result.o.d.raw   -c
g++: warning: switch '-msign-return-address=all' is no longer supported
  g++ -o /node/out/Release/obj.target/gtest/deps/googletest/src/gtest-death-test.o ../deps/googletest/src/gtest-death-test.cc '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_GLIBCXX_USE_CXX11_ABI=1' '-DNODE_OPENSSL_CONF_NAME=nodejs_conf' '-DNODE_OPENSSL_HAS_QUIC' '-DICU_NO_USER_DATA_OVERRIDE' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_THREADS' '-DGTEST_HAS_POSIX_RE=0' '-DGTEST_LANG_CXX11=1' -I../deps/googletest -I../deps/googletest/include  -msign-return-address=all -pthread -Wall -Wextra -Wno-unused-parameter -fPIC -O3 -fno-omit-frame-pointer -fno-rtti -fno-exceptions -std=gnu++17 -MMD -MF /node/out/Release/.deps//node/out/Release/obj.target/gtest/deps/googletest/src/gtest-death-test.o.d.raw   -c
g++: warning: switch '-msign-return-address=all' is no longer supported
/tmp/ccNlzIJg.s: Assembler messages:
/tmp/ccNlzIJg.s:152: Error: selected processor does not support `pmull v20.1q,v1.1d,v0.1d'
/tmp/ccNlzIJg.s:156: Error: selected processor does not support `pmull v19.1q,v4.1d,v0.1d'
/tmp/ccNlzIJg.s:160: Error: selected processor does not support `pmull v18.1q,v3.1d,v0.1d'
/tmp/ccNlzIJg.s:164: Error: selected processor does not support `pmull v17.1q,v2.1d,v0.1d'
/tmp/ccNlzIJg.s:172: Error: selected processor does not support `pmull2 v1.1q,v1.2d,v0.2d'
/tmp/ccNlzIJg.s:176: Error: selected processor does not support `pmull2 v4.1q,v4.2d,v0.2d'
/tmp/ccNlzIJg.s:180: Error: selected processor does not support `pmull2 v3.1q,v3.2d,v0.2d'
/tmp/ccNlzIJg.s:184: Error: selected processor does not support `pmull2 v2.1q,v2.2d,v0.2d'
/tmp/ccNlzIJg.s:210: Error: selected processor does not support `pmull v0.1q,v1.1d,v5.1d'
/tmp/ccNlzIJg.s:214: Error: selected processor does not support `pmull2 v1.1q,v1.2d,v5.2d'
/tmp/ccNlzIJg.s:222: Error: selected processor does not support `pmull v1.1q,v0.1d,v5.1d'
/tmp/ccNlzIJg.s:226: Error: selected processor does not support `pmull2 v0.1q,v0.2d,v5.2d'
/tmp/ccNlzIJg.s:234: Error: selected processor does not support `pmull v0.1q,v1.1d,v5.1d'
/tmp/ccNlzIJg.s:238: Error: selected processor does not support `pmull2 v1.1q,v1.2d,v5.2d'
/tmp/ccNlzIJg.s:249: Error: selected processor does not support `pmull v2.1q,v0.1d,v5.1d'
/tmp/ccNlzIJg.s:253: Error: selected processor does not support `pmull2 v0.1q,v0.2d,v5.2d'
/tmp/ccNlzIJg.s:265: Error: selected processor does not support `pmull v2.1q,v0.1d,v5.1d'
/tmp/ccNlzIJg.s:269: Error: selected processor does not support `pmull2 v0.1q,v0.2d,v5.2d'
/tmp/ccNlzIJg.s:281: Error: selected processor does not support `pmull v2.1q,v0.1d,v5.1d'
/tmp/ccNlzIJg.s:285: Error: selected processor does not support `pmull2 v0.1q,v0.2d,v5.2d'
/tmp/ccNlzIJg.s:295: Error: selected processor does not support `pmull v5.1q,v0.1d,v5.1d'
/tmp/ccNlzIJg.s:309: Error: selected processor does not support `pmull v5.1q,v5.1d,v2.1d'
/tmp/ccNlzIJg.s:316: Error: selected processor does not support `pmull v2.1q,v2.1d,v4.1d'
/tmp/ccNlzIJg.s:327: Error: selected processor does not support `pmull v2.1q,v2.1d,v4.1d'
/tmp/ccNlzIJg.s:334: Error: selected processor does not support `pmull v1.1q,v1.1d,v3.1d'
make[1]: *** [deps/zlib/zlib_arm_crc32.target.mk:95: /node/out/Release/obj.target/zlib_arm_crc32/deps/zlib/crc32_simd.o] Error 1

This happens on both, GCC 11.3.0 and on GCC 7.5.0

Activity

  1. bnoordhuis commented on May 15, 2023

    @bnoordhuis
    bnoordhuis
    on May 15, 2023
    Member

    It should check the host CPU for the available instructions

    It does; see deps/zlib/cpu_features.c, the arm_cpu_enable_crc32 variable in particular.

    Apropos gcc: it doesn't support setting target machine attributes on a per-function basis1, only per compilation unit. If that's the problem you're hitting, try building with clang instead.

    1 Newer gcc versions possibly do. You can try submitting a fix to the upstream project; it's Chromium's zlib fork.

  2. JulianGro commented on May 15, 2023

    @JulianGro
    JulianGro
    on May 15, 2023
    Author

    On an old GCC version, compilation failes with

      cc -o /node/out/Release/obj.target/zlib_arm_crc32/deps/zlib/crc32_simd.o ../deps/zlib/crc32_simd.c '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_GLIBCXX_USE_CXX11_ABI=1' '-DNODE_OPENSSL_CONF_NAME=nodejs_conf' '-DNODE_OPENSSL_HAS_QUIC' '-DICU_NO_USER_DATA_OVERRIDE' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_THREADS' '-DCRC32_ARMV8_CRC32' '-DARMV8_OS_LINUX' -I../deps/zlib  -msign-return-address=all -pthread -Wall -Wextra -Wno-unused-parameter -fPIC -march=armv8-a+aes+crc -O3 -fno-omit-frame-pointer  -MMD -MF /node/out/Release/.deps//node/out/Release/obj.target/zlib_arm_crc32/deps/zlib/crc32_simd.o.d.raw   -c
cc1: error: invalid feature modifier in '-march=armv8-a+aes+crc'

    on a Raspberry Pi 4, which suggests that it doesn't suggest instructions in practice.

    The documentation lists gcc and g++ >= 10.1 or newer as build dependency, so it shouldn't depend on any features that gcc doesn't provide.

  3. bnoordhuis commented on May 15, 2023

    @bnoordhuis
    bnoordhuis
    on May 15, 2023
    Member

    Yes, and it's building fine on our CI. The problem is likely local to your toolchain.

  4. bnoordhuis commented on May 16, 2023

    @bnoordhuis
    bnoordhuis
    on May 16, 2023
    Member

    I'll close this out but let me know if there's still something to do or discuss.

  5. bnoordhuis
    closed this as not plannedon May 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions