Sitelet https://github.com/python/cpython/issues/94208
Skip to content

test_ssl fails on recent FreeBSD (OpenSSL 1.1.1p), minimum_version=TLSv1_2 #94208

Closed
Closed
test_ssl fails on recent FreeBSD (OpenSSL 1.1.1p), minimum_version=TLSv1_2#94208
Labels
3.10 (EOL)end of life3.11only security fixes3.12only security fixesOS-freebsdtype-bugAn unexpected behavior, bug, or error

Description

@vstinner
vstinner
opened on Jun 24, 2022

test_ssl started to fail today on AMD64 FreeBSD Shared 3.x:
https://buildbot.python.org/all/#/builders/483/builds/2643

It seems like ssl.SSLContext.minimum_version is now TLSVersion.TLSv1_2.

0:42:57 load avg: 0.32 Re-running test_ssl in verbose mode (matching: test_openssl111_deprecations, test__create_stdlib_context, test_constructor, test_protocol_tlsv1_2, test_min_max_version)
test_openssl111_deprecations (test.test_ssl.BasicSocketTests.test_openssl111_deprecations) ... 
  test_openssl111_deprecations (test.test_ssl.BasicSocketTests.test_openssl111_deprecations) (protocol=<_SSLMethod.PROTOCOL_TLSv1: 3>) ... ERROR
test__create_stdlib_context (test.test_ssl.ContextTests.test__create_stdlib_context) ... ERROR
test_constructor (test.test_ssl.ContextTests.test_constructor) ... ERROR
test_min_max_version (test.test_ssl.ContextTests.test_min_max_version) ... FAIL
test_protocol_tlsv1_2 (test.test_ssl.ThreadedTests.test_protocol_tlsv1_2)
Connecting to a TLSv1.2 server with various client options. ... ERROR

Stdout:

 PROTOCOL_TLSv1_2->PROTOCOL_TLSv1_2 CERT_NONE
 {PROTOCOL_TLS->PROTOCOL_TLSv1_2} CERT_NONE
 PROTOCOL_TLSv1_2->PROTOCOL_TLS CERT_NONE
 {PROTOCOL_TLSv1->PROTOCOL_TLSv1_2} CERT_NONE

======================================================================
ERROR: test_openssl111_deprecations (test.test_ssl.BasicSocketTests.test_openssl111_deprecations) (protocol=<_SSLMethod.PROTOCOL_TLSv1: 3>)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 621, in test_openssl111_deprecations
    ssl.SSLContext(protocol)
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/ssl.py", line 500, in __new__
    self = _SSLContext.__new__(cls, protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid or unsupported protocol version 3

======================================================================
ERROR: test__create_stdlib_context (test.test_ssl.ContextTests.test__create_stdlib_context)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 1720, in test__create_stdlib_context
    ctx = ssl._create_stdlib_context(ssl.PROTOCOL_TLSv1)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/ssl.py", line 810, in _create_unverified_context
    context = SSLContext(protocol)
              ^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/ssl.py", line 500, in __new__
    self = _SSLContext.__new__(cls, protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid or unsupported protocol version 3

======================================================================
ERROR: test_constructor (test.test_ssl.ContextTests.test_constructor)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 1143, in test_constructor
    ctx = ssl.SSLContext(protocol)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/ssl.py", line 500, in __new__
    self = _SSLContext.__new__(cls, protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid or unsupported protocol version 3

======================================================================
ERROR: test_protocol_tlsv1_2 (test.test_ssl.ThreadedTests.test_protocol_tlsv1_2)
Connecting to a TLSv1.2 server with various client options.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 257, in wrapper
    return func(*args, **kw)
           ^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 3457, in test_protocol_tlsv1_2
    try_protocol_combo(ssl.PROTOCOL_TLSv1_2, ssl.PROTOCOL_TLSv1, False)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 2870, in try_protocol_combo
    client_context = ssl.SSLContext(client_protocol)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/ssl.py", line 500, in __new__
    self = _SSLContext.__new__(cls, protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid or unsupported protocol version 3

Stdout:

 PROTOCOL_TLSv1_2->PROTOCOL_TLSv1_2 CERT_NONE
 {PROTOCOL_TLS->PROTOCOL_TLSv1_2} CERT_NONE
 PROTOCOL_TLSv1_2->PROTOCOL_TLS CERT_NONE
 {PROTOCOL_TLSv1->PROTOCOL_TLSv1_2} CERT_NONE

======================================================================
FAIL: test_min_max_version (test.test_ssl.ContextTests.test_min_max_version)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/support/warnings_helper.py", line 57, in wrapper
    return test(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 1287, in test_min_max_version
    self.assertIn(
    ^^^^^^^^^^^^^^
AssertionError: <TLSVersion.TLSv1_1: 770> not found in {<TLSVersion.SSLv3: 768>, <TLSVersion.TLSv1: 769>}

----------------------------------------------------------------------
Ran 5 tests in 0.113s

FAILED (failures=1, errors=4)
test test_ssl failed

test.pythoninfo:

ssl.HAS_SNI: True
ssl.OPENSSL_VERSION: OpenSSL 1.1.1p  21 Jun 2022
ssl.OPENSSL_VERSION_INFO: (1, 1, 1, 16, 15)
ssl.OP_ALL: 0x80000054
ssl.OP_NO_TLSv1_1: 0x10000000
ssl.SSLContext.maximum_version: -1
ssl.SSLContext.minimum_version: 771
ssl.SSLContext.options: 2186412116
ssl.SSLContext.protocol: 16
ssl.SSLContext.verify_mode: 2

ssl.default_https_context.maximum_version: -1
ssl.default_https_context.minimum_version: 771
ssl.default_https_context.options: 2186412116
ssl.default_https_context.protocol: 16
ssl.default_https_context.verify_mode: 2

ssl.stdlib_context.maximum_version: -1
ssl.stdlib_context.minimum_version: 771
ssl.stdlib_context.options: 2186412116
ssl.stdlib_context.protocol: 16
ssl.stdlib_context.verify_mode: 0

Activity

  1. vstinner
    added
    type-bugAn unexpected behavior, bug, or error
    on Jun 24, 2022

  2. vstinner commented on Jun 24, 2022

    @vstinner
    vstinner
    on Jun 24, 2022
    MemberAuthor

    cc @koobs

  3. kumaraditya303
    added
    OS-freebsd
    on Jun 24, 2022

  4. koobs commented on Jun 26, 2022

    @koobs
    koobs
    on Jun 26, 2022

    The defaults of the FreeBSD OpenSSL port/package may have changed (or I inadvertently changed them).

    Python tests should either either check for TLS version support in the underlying SSL library before running the tests, or skipping (not FAIL'ing) due to non-presence of the required version for the test to pass, as they cant pass a test for protocol version support that doesn't exist.

    Any OpenSSL library for any OS under test/CI for CPython may change support for any protocol version or feature at any time, so its worth leveling up the SSL tests in that regard.

  5. tiran
    added
    release-blocker
    on Jun 27, 2022

  6. tiran commented on Jun 27, 2022

    @tiran
    tiran
    on Jun 27, 2022
    Member

    @pablogsal The failing tests on FreeBSD are blocking the release pipeline, https://buildbot.python.org/all/#/release_status I'm working on a fix.

  7. miss-islington
    added 4 commits that reference this issue on Jun 28, 2022

    pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…

    842bf58

    pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…

    09ac5bd

    gh-94208: Add more TLS version/protocol checks for FreeBSD (GH-94347)

    5e08eec

    gh-94208: Add more TLS version/protocol checks for FreeBSD (GH-94347)

    cf64db6
  8. tiran
    added
    3.11only security fixes
    and removed
    release-blocker
    on Jun 28, 2022

  9. 8 remaining items

  10. vstinner commented on Jun 28, 2022

    @vstinner
    vstinner
    on Jun 28, 2022
    MemberAuthor

    My guess is it blindly relies on stable buildbots instead of the tier labels.

    Yep, it's a simple boolean test: STABLE vs "not STABLE" workers.

  11. koobs commented on Jun 30, 2022

    @koobs
    koobs
    on Jun 30, 2022

    I'd like to take the opportunity to put out a explicit call for a second core maintainer to promote FreeBSD to Tier 2 on the following basis:

    • I have provided proactive and consistent support for multiple supported FreeBSD branches for several years, and am committed to improving and extending testing in the long term.
    • Our FreeBSD test coverage has a demonstrated history of uniquely identifying issues and improving the quality of Python upstream
    • I can (and have) provided SSH access to these buildbot workers to support and ease issue isolation and reproduction.
    • We (FreeBSD) have a demonstrated proactive and positive relationship with upstream developers and record for supporting Python developers in issue resolution and resolving issues in FreeBSD where necessary
    • Ecosystem support for FreeBSD in our Ports/Packages system is considered Tier 1 downstream, and we proactively produce and support all supported Python branch versions, including in-development versions for early user testing
  12. gvanrossum
    added a commit that references this issue on Jun 30, 2022

    pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…

    fee1b9f
  13. ambv
    added 9 commits that reference this issue on Jul 27, 2022

    pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…

    a7e54e0

    [3.9] pythongh-94208: Add more TLS version/protocol checks for FreeBSD (

    06ef0ae

    pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…

    77cdade

    [3.8] pythongh-94208: Add more TLS version/protocol checks for FreeBSD (

    428ed95

    pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…

    91970d1

    [3.7] pythongh-94208: Add more TLS version/protocol checks for FreeBSD (

    0c778f1

    [3.9] gh-94208: Add more TLS version/protocol checks for FreeBSD (GH-…

    017080f

    [3.8] gh-94208: Add more TLS version/protocol checks for FreeBSD (GH-…

    48b323e

    [3.7] gh-94208: Add more TLS version/protocol checks for FreeBSD (GH-…

    c199831
  14. ambv
    added 2 commits that reference this issue on Oct 7, 2022

    pythongh-94208: Add even more TLS version/protocol checks for FreeBSD

    eeb9507

    [3.9] gh-94208: Add even more TLS version/protocol checks for FreeBSD (…

    da1fe38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.10 (EOL)end of life3.11only security fixes3.12only security fixesOS-freebsdtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions