681c46853d
Since the addition of macos14 M1 runners in our CI jobs we've been
seeing periodic random failures in the test_threads CI job.
Specifically we've seen instances in which the shared pointer in the
test (which points to a monotonically incrementing uint64_t went
backwards.
From taking a look at the disassembled code in the failing case, we see
that __atomic_load_n when emitted in clang 15 looks like this
0000000100120488 <_ossl_rcu_uptr_deref>:
100120488: f8bfc000 ldapr x0, [x0]
10012048c: d65f03c0 ret
Notably, when compiling with gcc on the same system we get this output
instead:
0000000100120488 <_ossl_rcu_uptr_deref>:
100120488: f8bfc000 ldar x0, [x0]
10012048c: d65f03c0 ret
Checking the arm docs for the difference between ldar and ldapr:
https://developer.arm.com/documentation/ddi0602/2023-09/Base-Instructions/LDAPR--Load-Acquire-RCpc-Register-
https://developer.arm.com/documentation/dui0802/b/A64-Data-Transfer-Instructions/LDAR
It seems that the ldar instruction provides a global cpu fence, not
completing until all writes in a given cpus writeback queue have
completed
Conversely, the ldapr instruction attmpts to achieve performance
improvements by honoring the Local Ordering register available in the
system coprocessor, only flushing writes in the same address region as
other cpus on the system.
I believe that on M1 virtualized cpus the ldapr is not properly ordering
writes, leading to an out of order read, despite the needed fencing.
I've opened an issue with apple on this here:
https://developer.apple.com/forums/thread/749530
I believe that it is not safe to issue an ldapr instruction unless the
programmer knows that the Local order registers are properly configured
for use on the system.
So to fix it I'm proposing with this patch that we, in the event that:
1) __APPLE__ is defined
AND
2) __clang__ is defined
AND
3) __aarch64__ is defined
during the build, that we override the ATOMIC_LOAD_N macro in the rcu
code such that it uses a custom function with inline assembly to emit
the ldar instruction rather than the ldapr instruction. The above
conditions should get us to where this is only used on more recent MAC
cpus, and only in the case where the affected clang compiler emits the
offending instruction.
I've run this patch 10 times in our CI and failed to reproduce the
issue, whereas previously I could trigger it within 5 runs routinely.
Reviewed-by: Tom Cosgrove <tom.cosgrove@arm.com>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/23974)
(cherry picked from commit
|
||
|---|---|---|
| .ctags.d | ||
| .github | ||
| apps | ||
| cloudflare-quiche@7ab6a55cfe | ||
| Configurations | ||
| crypto | ||
| demos | ||
| dev | ||
| doc | ||
| engines | ||
| exporters | ||
| external/perl | ||
| fuzz | ||
| gost-engine@ede3886cc5 | ||
| include | ||
| krb5@aa9b4a2a64 | ||
| ms | ||
| oqs-provider@8c746d7e29 | ||
| os-dep | ||
| providers | ||
| pyca-cryptography@7e33b0e773 | ||
| python-ecdsa@4de8d5bf89 | ||
| ssl | ||
| test | ||
| tlsfuzzer@dbd56c1490 | ||
| tlslite-ng@771e9f59d6 | ||
| tools | ||
| util | ||
| VMS | ||
| wycheproof@2196000605 | ||
| .git-blame-ignore-revs | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| ACKNOWLEDGEMENTS.md | ||
| AUTHORS.md | ||
| build.info | ||
| CHANGES.md | ||
| CODE-OF-CONDUCT.md | ||
| config | ||
| config.com | ||
| configdata.pm.in | ||
| Configure | ||
| CONTRIBUTING.md | ||
| HACKING.md | ||
| INSTALL.md | ||
| LICENSE.txt | ||
| NEWS.md | ||
| NOTES-ANDROID.md | ||
| NOTES-DJGPP.md | ||
| NOTES-NONSTOP.md | ||
| NOTES-PERL.md | ||
| NOTES-UNIX.md | ||
| NOTES-VALGRIND.md | ||
| NOTES-VMS.md | ||
| NOTES-WINDOWS.md | ||
| README-ENGINES.md | ||
| README-FIPS.md | ||
| README-PROVIDERS.md | ||
| README-QUIC.md | ||
| README.md | ||
| SUPPORT.md | ||
| VERSION.dat | ||
Welcome to the OpenSSL Project
OpenSSL is a robust, commercial-grade, full-featured Open Source Toolkit for the TLS (formerly SSL), DTLS and QUIC (currently client side only) protocols.
The protocol implementations are based on a full-strength general purpose cryptographic library, which can also be used stand-alone. Also included is a cryptographic module validated to conform with FIPS standards.
OpenSSL is descended from the SSLeay library developed by Eric A. Young and Tim J. Hudson.
The official Home Page of the OpenSSL Project is www.openssl.org.
Table of Contents
Overview
The OpenSSL toolkit includes:
-
libssl an implementation of all TLS protocol versions up to TLSv1.3 (RFC 8446), DTLS protocol versions up to DTLSv1.2 (RFC 6347) and the QUIC (currently client side only) version 1 protocol (RFC 9000).
-
libcrypto a full-strength general purpose cryptographic library. It constitutes the basis of the TLS implementation, but can also be used independently.
-
openssl the OpenSSL command line tool, a swiss army knife for cryptographic tasks, testing and analyzing. It can be used for
- creation of key parameters
- creation of X.509 certificates, CSRs and CRLs
- calculation of message digests
- encryption and decryption
- SSL/TLS/DTLS and client and server tests
- QUIC client tests
- handling of S/MIME signed or encrypted mail
- and more...
Download
For Production Use
Source code tarballs of the official releases can be downloaded from www.openssl.org/source. The OpenSSL project does not distribute the toolkit in binary form.
However, for a large variety of operating systems precompiled versions of the OpenSSL toolkit are available. In particular, on Linux and other Unix operating systems, it is normally recommended to link against the precompiled shared libraries provided by the distributor or vendor.
We also maintain a list of third parties that produce OpenSSL binaries for various Operating Systems (including Windows) on the Binaries page on our wiki.
For Testing and Development
Although testing and development could in theory also be done using the source tarballs, having a local copy of the git repository with the entire project history gives you much more insight into the code base.
The official OpenSSL Git Repository is located at git.openssl.org. There is a GitHub mirror of the repository at github.com/openssl/openssl, which is updated automatically from the former on every commit.
A local copy of the Git Repository can be obtained by cloning it from the original OpenSSL repository using
git clone git://git.openssl.org/openssl.git
or from the GitHub mirror using
git clone https://github.com/openssl/openssl.git
If you intend to contribute to OpenSSL, either to fix bugs or contribute new features, you need to fork the OpenSSL repository openssl/openssl on GitHub and clone your public fork instead.
git clone https://github.com/yourname/openssl.git
This is necessary because all development of OpenSSL nowadays is done via GitHub pull requests. For more details, see Contributing.
Build and Install
After obtaining the Source, have a look at the INSTALL file for detailed instructions about building and installing OpenSSL. For some platforms, the installation instructions are amended by a platform specific document.
- Notes for UNIX-like platforms
- Notes for Android platforms
- Notes for Windows platforms
- Notes for the DOS platform with DJGPP
- Notes for the OpenVMS platform
- Notes on Perl
- Notes on Valgrind
Specific notes on upgrading to OpenSSL 3.x from previous versions can be found in the ossl-guide-migration(7ossl) manual page.
Documentation
README Files
There are some README.md files in the top level of the source distribution containing additional information on specific topics.
- Information about the OpenSSL QUIC protocol implementation
- Information about the OpenSSL Provider architecture
- Information about using the OpenSSL FIPS validated module
- Information about the legacy OpenSSL Engine architecture
The OpenSSL Guide
There are some tutorial and introductory pages on some important OpenSSL topics within the OpenSSL Guide.
Manual Pages
The manual pages for the master branch and all current stable releases are available online.
Demos
The are numerous source code demos for using various OpenSSL capabilities in the demos subfolder.
Wiki
There is a Wiki at wiki.openssl.org which is currently not very active. It contains a lot of useful information, not all of which is up-to-date.
License
OpenSSL is licensed under the Apache License 2.0, which means that you are free to get and use it for commercial and non-commercial purposes as long as you fulfill its conditions.
See the LICENSE.txt file for more details.
Support
There are various ways to get in touch. The correct channel depends on your requirement. See the SUPPORT file for more details.
Contributing
If you are interested and willing to contribute to the OpenSSL project, please take a look at the CONTRIBUTING file.
Legalities
A number of nations restrict the use or export of cryptography. If you are potentially subject to such restrictions, you should seek legal advice before attempting to develop or distribute cryptographic code.
Copyright
Copyright (c) 1998-2024 The OpenSSL Project Authors
Copyright (c) 1995-1998 Eric A. Young, Tim J. Hudson
All rights reserved.