user0/rust
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #4141 from rust-lang/rustup-2025-01-19

Browse source 
Automatic Rustup
This commit is contained in:
Ralf Jung 2025-01-19 17:32:20 +00:00 • committed by GitHub
commit 49375c48f7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
481 changed files with 8216 additions and 5440 deletions
Download patch file Download diff file Expand all files Collapse all files

68
.github/workflows/ghcr.yml vendored Normal file
View file

@ -0,0 +1,68 @@
# Mirror DockerHub images used by the Rust project to ghcr.io.
# Images are available at https://github.com/orgs/rust-lang/packages.
#
# In some CI jobs, we pull images from ghcr.io instead of Docker Hub because
# Docker Hub has a rate limit, while ghcr.io doesn't.
# Those images are pushed to ghcr.io by this job.
#
# Note that authenticating to DockerHub or other registries isn't possible
# for PR jobs, because forks can't access secrets.
# That's why we use ghcr.io: it has no rate limit and it doesn't require authentication.
name: GHCR image mirroring
on:
workflow_dispatch:
schedule:
# Run daily at midnight UTC
- cron: '0 0 * * *'
jobs:
mirror:
name: DockerHub mirror
runs-on: ubuntu-24.04
if: github.repository == 'rust-lang/rust'
permissions:
# Needed to write to the ghcr.io registry
packages: write
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Log in to registry
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.repository_owner }} --password-stdin
# Download crane in the current directory.
# We use crane because it copies the docker image for all the architectures available in
# DockerHub for the image.
# Learn more about crane at
# https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md
- name: Download crane
run: |
curl -sL "https://github.com/google/go-containerregistry/releases/download/${VERSION}/go-containerregistry_${OS}_${ARCH}.tar.gz" | tar -xzf -
env:
VERSION: v0.20.2
OS: Linux
ARCH: x86_64
- name: Mirror DockerHub
run: |
# List of DockerHub images to mirror to ghcr.io
images=(
# Mirrored because used by the mingw-check-tidy, which doesn't cache Docker images
"ubuntu:22.04"
# Mirrored because used by all linux CI jobs, including mingw-check-tidy
"moby/buildkit:buildx-stable-1"
)
# Mirror each image from DockerHub to ghcr.io
for img in "${images[@]}"; do
echo "Mirroring ${img}..."
# Remove namespace from the image if any.
# E.g. "moby/buildkit:buildx-stable-1" becomes "buildkit:buildx-stable-1"
dest_image=$(echo "${img}" | cut -d'/' -f2-)
./crane copy \
"docker.io/${img}" \
"ghcr.io/${{ github.repository_owner }}/${dest_image}"
done

392
COPYRIGHT
View file

@ -3,6 +3,7 @@ Short version for non-lawyers:
The Rust Project is dual-licensed under Apache 2.0 and MIT
terms.
It is Copyright (c) The Rust Project Contributors.
Longer version:
@ -11,374 +12,23 @@ copyright assignment is required to contribute to the Rust project.
Some files include explicit copyright notices and/or license notices.
For full authorship information, see the version control history or
https://thanks.rust-lang.org
Except as otherwise noted (below and/or in individual files), Rust is
licensed under the Apache License, Version 2.0 <LICENSE-APACHE> or
<http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
<LICENSE-MIT> or <http://opensource.org/licenses/MIT>, at your option.
The Rust Project includes packages written by third parties.
The following third party packages are included, and carry
their own copyright notices and license terms:
* LLVM, located in src/llvm-project, is licensed under the following
terms.
==============================================================================
The LLVM Project is under the Apache License v2.0 with LLVM Exceptions:
==============================================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---- LLVM Exceptions to the Apache 2.0 License ----
As an exception, if, as a result of your compiling your source code, portions
of this Software are embedded into an Object form of such source code, you
may redistribute such embedded portions in such Object form without complying
with the conditions of Sections 4(a), 4(b) and 4(d) of the License.
In addition, if you combine or link compiled forms of this Software with
software that is licensed under the GPLv2 ("Combined Software") and if a
court of competent jurisdiction determines that the patent provision (Section
3), the indemnity provision (Section 9) or other Section of the License
conflicts with the conditions of the GPLv2, you may retroactively and
prospectively choose to deem waived or otherwise exclude such Section(s) of
the License, but only in their entirety and only with respect to the Combined
Software.
==============================================================================
Software from third parties included in the LLVM Project:
==============================================================================
The LLVM Project contains third party software which is under different license
terms. All such code will be identified clearly using at least one of two
mechanisms:
1) It will be in a separate directory tree with its own `LICENSE.txt` or
`LICENSE` file at the top containing the specific license and restrictions
which apply to that software, or
2) It will contain specific license and restriction terms at the top of every
file.
==============================================================================
Legacy LLVM License (https://llvm.org/docs/DeveloperPolicy.html#legacy):
==============================================================================
University of Illinois/NCSA
Open Source License
Copyright (c) 2003-2019 University of Illinois at Urbana-Champaign.
All rights reserved.
Developed by:
LLVM Team
University of Illinois at Urbana-Champaign
http://llvm.org
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal with
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimers.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimers in the
documentation and/or other materials provided with the distribution.
* Neither the names of the LLVM Team, University of Illinois at
Urbana-Champaign, nor the names of its contributors may be used to
endorse or promote products derived from this Software without specific
prior written permission.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
SOFTWARE.
* Portions of the FFI code for interacting with the native ABI
is derived from the Clay programming language, which carries
the following license.
Copyright (C) 2008-2010 Tachyon Technologies.
All rights reserved.
Redistribution and use in source and binary forms, with
or without modification, are permitted provided that the
following conditions are met:
1. Redistributions of source code must retain the above
copyright notice, this list of conditions and the
following disclaimer.
2. Redistributions in binary form must reproduce the
above copyright notice, this list of conditions and
the following disclaimer in the documentation and/or
other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
DEVELOPERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
OF SUCH DAMAGE.
* Portions of internationalization code use code or data from Unicode, which
carry the following license:
UNICODE LICENSE V3
COPYRIGHT AND PERMISSION NOTICE
Copyright © 1991-2024 Unicode, Inc.
NOTICE TO USER: Carefully read the following legal agreement. BY
DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING DATA FILES, AND/OR
SOFTWARE, YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE
TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE, DO NOT
DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE THE DATA FILES OR SOFTWARE.
Permission is hereby granted, free of charge, to any person obtaining a
copy of data files and any associated documentation (the "Data Files") or
software and any associated documentation (the "Software") to deal in the
Data Files or Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, and/or sell
copies of the Data Files or Software, and to permit persons to whom the
Data Files or Software are furnished to do so, provided that either (a)
this copyright and permission notice appear with all copies of the Data
Files or Software, or (b) this copyright and permission notice appear in
associated Documentation.
THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF
THIRD PARTY RIGHTS.
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE
BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES,
OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA
FILES OR SOFTWARE.
Except as contained in this notice, the name of a copyright holder shall
not be used in advertising or otherwise to promote the sale, use or other
dealings in these Data Files or Software without prior written
authorization of the copyright holder.
<https://thanks.rust-lang.org>
Except as otherwise noted, Rust is licensed under the Apache License, Version
2.0 <LICENSE-APACHE> or <http://www.apache.org/licenses/LICENSE-2.0> or the MIT
license <LICENSE-MIT> or <http://opensource.org/licenses/MIT>, at your option.
We track licenses for third-party materials in two ways:
* We use [REUSE](https://reuse.software) to track license information for
in-tree source files - both those authored by the Rust project and those
authored by third parties. See `REUSE.toml`, and our cached output of the
`reuse` tool which is committed to `license-metadata.json`.
* We use `cargo` to track license information for out-of-tree dependencies.
These two sources of information are collected by the tool `generate-copyright`
into a file called `COPYRIGHT.html`, which is shipped with each binary release
of Rust. Please refer to that file for detailed information as to the components of
any given Rust release. We also produce a `COPYRIGHT-library.html` file which only
covers the subset of source code used in the Rust Standard Library, as opposed
to the toolchain as a whole.

211
Cargo.lock
View file

@ -254,9 +254,9 @@ dependencies = [
[[package]]
name = "bitflags"
version = "2.6.0"
version = "2.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b048fb63fd8b5923fc5aa7b340d8e156aec7ec02f0c78fa8a6ddc2613f6f71de"
checksum = "1be3f42a67d6d345ecd59f675f3f012d6974981560836e938c22b424b85ce1be"
[[package]]
name = "blake3"
@ -407,7 +407,7 @@ dependencies = [
"semver",
"serde",
"serde_json",
"thiserror 2.0.9",
"thiserror 2.0.11",
]
[[package]]
@ -481,9 +481,9 @@ dependencies = [
[[package]]
name = "clap"
version = "4.5.23"
version = "4.5.26"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3135e7ec2ef7b10c6ed8950f0f792ed96ee093fa088608f1c76e569722700c84"
checksum = "a8eb5e908ef3a6efbe1ed62520fb7287959888c88485abe072543190ecc66783"
dependencies = [
"clap_builder",
"clap_derive",
@ -501,9 +501,9 @@ dependencies = [
[[package]]
name = "clap_builder"
version = "4.5.23"
version = "4.5.26"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "30582fc632330df2bd26877bde0c1f4470d57c582bbc070376afcd04d8cb4838"
checksum = "96b01801b5fc6a0a232407abc821660c9c6d25a1cafc0d4f85f29fb8d9afc121"
dependencies = [
"anstream",
"anstyle",
@ -514,23 +514,23 @@ dependencies = [
[[package]]
name = "clap_complete"
version = "4.5.40"
version = "4.5.42"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac2e663e3e3bed2d32d065a8404024dad306e699a04263ec59919529f803aee9"
checksum = "33a7e468e750fa4b6be660e8b5651ad47372e8fb114030b594c2d75d48c5ffd0"
dependencies = [
"clap",
]
[[package]]
name = "clap_derive"
version = "4.5.18"
version = "4.5.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ac6a0c7b1a9e9a5186361f67dfa1b88213572f427fb9ab038efb2bd8c582dab"
checksum = "54b755194d6389280185988721fffba69495eed5ee9feeee9a599b53db80318c"
dependencies = [
"heck 0.5.0",
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -561,7 +561,7 @@ dependencies = [
"rustc_tools_util",
"serde",
"serde_json",
"syn 2.0.94",
"syn 2.0.96",
"tempfile",
"termize",
"tokio",
@ -671,7 +671,7 @@ dependencies = [
"nom",
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -896,7 +896,7 @@ dependencies = [
"proc-macro2",
"quote",
"strsim",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -907,7 +907,7 @@ checksum = "d336a2a514f6ccccaa3e09b02d41d35330c07ddf03a62165fcec10bb561c7806"
dependencies = [
"darling_core",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -944,7 +944,7 @@ checksum = "62d671cc41a825ebabc75757b62d3d168c577f9149b2d49ece1dad1f72119d25"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -965,7 +965,7 @@ dependencies = [
"darling",
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -975,7 +975,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab63b0e2bf4d5928aff72e83a7dace85d7bba5fe12dcc3c5a572d78caffd3f3c"
dependencies = [
"derive_builder_core",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -987,7 +987,7 @@ dependencies = [
"darling",
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -1065,7 +1065,7 @@ checksum = "97369cbbc041bc366949bc74d34658d6cda5621039731c6310521892a3a20ae0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -1362,7 +1362,7 @@ checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -1502,17 +1502,18 @@ dependencies = [
[[package]]
name = "handlebars"
version = "6.2.0"
version = "6.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fd4ccde012831f9a071a637b0d4e31df31c0f6c525784b35ae76a9ac6bc1e315"
checksum = "3d6b224b95c1e668ac0270325ad563b2eef1469fbbb8959bc7c692c844b813d9"
dependencies = [
"derive_builder",
"log",
"num-order",
"pest",
"pest_derive",
"serde",
"serde_json",
"thiserror 1.0.69",
"thiserror 2.0.11",
]
[[package]]
@ -1596,7 +1597,7 @@ dependencies = [
"markup5ever",
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -1785,7 +1786,7 @@ checksum = "1ec89e9337638ecdc08744df490b221a7399bf8d164eb52a665454e60e075ad6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -2085,9 +2086,9 @@ dependencies = [
[[package]]
name = "libz-sys"
version = "1.1.20"
version = "1.1.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2d16453e800a8cf6dd2fc3eb4bc99b786a9b90c663b8559a5b1a041bf89e472"
checksum = "df9b68e50e6e0b26f672573834882eb57759f6db9b3be2ea3c35c91188bb4eaa"
dependencies = [
"cc",
"libc",
@ -2114,9 +2115,9 @@ dependencies = [
[[package]]
name = "linux-raw-sys"
version = "0.4.14"
version = "0.4.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78b3ae25bc7c8c38cec158d1f2757ee79e9b3740fbc7ccf0e59e4b08d793fa89"
checksum = "d26c52dbd32dccf2d10cac7725f8eae5296885fb5703b261f7d0a0739ec807ab"
[[package]]
name = "litemap"
@ -2717,7 +2718,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b7cafe60d6cf8e62e1b9b2ea516a089c008945bb5a275416789e7db0bc199dc"
dependencies = [
"memchr",
"thiserror 2.0.9",
"thiserror 2.0.11",
"ucd-trie",
]
@ -2741,7 +2742,7 @@ dependencies = [
"pest_meta",
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -2757,21 +2758,21 @@ dependencies = [
[[package]]
name = "phf"
version = "0.11.2"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ade2d8b8f33c7333b51bcf0428d37e217e9f32192ae4772156f65063b8ce03dc"
checksum = "1fd6780a80ae0c52cc120a26a1a42c1ae51b247a253e4e06113d23d2c2edd078"
dependencies = [
"phf_shared 0.11.2",
"phf_shared 0.11.3",
]
[[package]]
name = "phf_codegen"
version = "0.11.2"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8d39688d359e6b34654d328e262234662d16cc0f60ec8dcbe5e718709342a5a"
checksum = "aef8048c789fa5e851558d709946d6d79a8ff88c0440c587967f8e94bfb1216a"
dependencies = [
"phf_generator 0.11.2",
"phf_shared 0.11.2",
"phf_generator 0.11.3",
"phf_shared 0.11.3",
]
[[package]]
@ -2786,11 +2787,11 @@ dependencies = [
[[package]]
name = "phf_generator"
version = "0.11.2"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "48e4cc64c2ad9ebe670cb8fd69dd50ae301650392e81c05f9bfcb2d5bdbc24b0"
checksum = "3c80231409c20246a13fddb31776fb942c38553c51e871f8cbd687a4cfb5843d"
dependencies = [
"phf_shared 0.11.2",
"phf_shared 0.11.3",
"rand",
]
@ -2800,23 +2801,23 @@ version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6796ad771acdc0123d2a88dc428b5e38ef24456743ddb1744ed628f9815c096"
dependencies = [
"siphasher",
"siphasher 0.3.11",
]
[[package]]
name = "phf_shared"
version = "0.11.2"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "90fcb95eef784c2ac79119d1dd819e162b5da872ce6f3c3abe1e8ca1c082f72b"
checksum = "67eabc2ef2a60eb7faa00097bd1ffdb5bd28e62bf39990626a582201b7a754e5"
dependencies = [
"siphasher",
"siphasher 1.0.1",
]
[[package]]
name = "pin-project-lite"
version = "0.2.15"
version = "0.2.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "915a1e146535de9163f3987b8944ed8cf49a18bb0056bcebcdcece385cece4ff"
checksum = "3b3cff922bd51709b605d9ead9aa71031d81447142d828eb4a6eba76fe619f9b"
[[package]]
name = "pin-utils"
@ -2886,9 +2887,9 @@ checksum = "dc375e1527247fe1a97d8b7156678dfe7c1af2fc075c9a4db3690ecd2a148068"
[[package]]
name = "proc-macro2"
version = "1.0.92"
version = "1.0.93"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37d3544b3f2748c54e147655edb5025752e2303145b5aefb3c3ea2c78b973bb0"
checksum = "60946a68e5f9d28b0dc1c21bb8a97ee7d018a8b322fa57838ba31cc878e22d99"
dependencies = [
"unicode-ident",
]
@ -3151,7 +3152,7 @@ dependencies = [
"rinja_parser",
"rustc-hash 2.1.0",
"serde",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -3792,7 +3793,7 @@ dependencies = [
"fluent-syntax",
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
"unic-langid",
]
@ -3927,7 +3928,7 @@ version = "0.0.0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -4075,7 +4076,7 @@ version = "0.0.0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
"synstructure",
]
@ -4258,6 +4259,7 @@ dependencies = [
"rustc_serialize",
"rustc_type_ir",
"rustc_type_ir_macros",
"smallvec",
"tracing",
]
@ -4663,7 +4665,7 @@ version = "0.0.0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
"synstructure",
]
@ -4752,7 +4754,7 @@ dependencies = [
"proc-macro2",
"quote",
"serde",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -4786,9 +4788,9 @@ dependencies = [
[[package]]
name = "rustix"
version = "0.38.42"
version = "0.38.43"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f93dc38ecbab2eb790ff964bb77fa94faf256fd3e73285fd7ba0903b76bedb85"
checksum = "a78891ee6bf2340288408954ac787aa063d8e8817e9f53abb37c695c6d834ef6"
dependencies = [
"bitflags",
"errno",
@ -4889,14 +4891,14 @@ checksum = "5a9bf7cf98d04a2b28aead066b7496853d4779c9cc183c440dbac457641e19a0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
name = "serde_json"
version = "1.0.134"
version = "1.0.135"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d00f4175c42ee48b15416f6193a959ba3a0d67fc699a0db9ad12df9f83991c7d"
checksum = "2b0d7ba2887406110130a978386c4e1befb98c674b4fba677954e4db976630d9"
dependencies = [
"indexmap",
"itoa",
@ -4969,6 +4971,12 @@ version = "0.3.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "38b58827f4464d87d377d175e90bf58eb00fd8716ff0a62f80356b5e61555d0d"
[[package]]
name = "siphasher"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "56199f7ddabf13fe5074ce809e7d3f42b42ae711800501b5b16ea82ad029c39d"
[[package]]
name = "slab"
version = "0.4.9"
@ -5147,9 +5155,9 @@ dependencies = [
[[package]]
name = "syn"
version = "2.0.94"
version = "2.0.96"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "987bc0be1cdea8b10216bd06e2ca407d40b9543468fafd3ddfb02f36e77f71f3"
checksum = "d5d0adab1ae378d7f53bdebc67a39f1f151407ef230f0ce2883572f5d8985c80"
dependencies = [
"proc-macro2",
"quote",
@ -5164,7 +5172,7 @@ checksum = "c8af7666ab7b6390ab78131fb5b0fce11d6b7a6951602017c35fa82800708971"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -5292,11 +5300,11 @@ dependencies = [
[[package]]
name = "thiserror"
version = "2.0.9"
version = "2.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f072643fd0190df67a8bab670c20ef5d8737177d6ac6b2e9a236cb096206b2cc"
checksum = "d452f284b73e6d76dd36758a0c8684b1d5be31f92b89d07fd5822175732206fc"
dependencies = [
"thiserror-impl 2.0.9",
"thiserror-impl 2.0.11",
]
[[package]]
@ -5307,18 +5315,18 @@ checksum = "4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
name = "thiserror-impl"
version = "2.0.9"
version = "2.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b50fa271071aae2e6ee85f842e2e28ba8cd2c5fb67f11fcb1fd70b276f9e7d4"
checksum = "26afc1baea8a989337eeb52b6e72a039780ce45c3edfcc9c5b9d112feeb173c2"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -5441,9 +5449,9 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20"
[[package]]
name = "tokio"
version = "1.42.0"
version = "1.43.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5cec9b21b0450273377fc97bd4c33a8acffc8c996c987a7c5b319a0083707551"
checksum = "3d61fa4ffa3de412bfea335c6ecff681de2b609ba3c77ef3e00e521813a9ed9e"
dependencies = [
"backtrace",
"bytes",
@ -5519,7 +5527,7 @@ checksum = "395ae124c09f9e6918a2310af6038fba074bcf474ac352496d5910dd59a2226d"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -5690,7 +5698,7 @@ checksum = "1ed7f4237ba393424195053097c1516bd4590dc82b84f2f97c5c69e12704555b"
dependencies = [
"proc-macro-hack",
"quote",
"syn 2.0.94",
"syn 2.0.96",
"unic-langid-impl",
]
@ -5828,9 +5836,9 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uuid"
version = "1.11.0"
version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8c5f0a0af699448548ad1a2fbf920fb4bee257eae39953ba95cb84891a0446a"
checksum = "b913a3b5fe84142e269d63cc62b64319ccaf89b748fc31fe025177f767a756c4"
dependencies = [
"getrandom",
]
@ -5896,7 +5904,7 @@ dependencies = [
"log",
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
"wasm-bindgen-shared",
]
@ -5918,7 +5926,7 @@ checksum = "30d7a95b763d3c45903ed6c81f156801839e5ee968bb07e534c44df0fcd330c2"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
"wasm-bindgen-backend",
"wasm-bindgen-shared",
]
@ -5968,12 +5976,12 @@ dependencies = [
[[package]]
name = "wasm-encoder"
version = "0.222.0"
version = "0.223.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3432682105d7e994565ef928ccf5856cf6af4ba3dddebedb737f61caed70f956"
checksum = "7e636076193fa68103e937ac951b5f2f587624097017d764b8984d9c0f149464"
dependencies = [
"leb128",
"wasmparser 0.222.0",
"wasmparser 0.223.0",
]
[[package]]
@ -6011,6 +6019,15 @@ name = "wasmparser"
version = "0.222.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4adf50fde1b1a49c1add6a80d47aea500c88db70551805853aa8b88f3ea27ab5"
dependencies = [
"bitflags",
]
[[package]]
name = "wasmparser"
version = "0.223.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d5a99faceb1a5a84dd6084ec4bfa4b2ab153b5793b43fd8f58b89232634afc35"
dependencies = [
"bitflags",
"indexmap",
@ -6019,22 +6036,22 @@ dependencies = [
[[package]]
name = "wast"
version = "222.0.0"
version = "223.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ce7191f4b7da0dd300cc32476abae6457154e4625d9b1bc26890828a9a26f6e"
checksum = "d59b2ba8a2ff9f06194b7be9524f92e45e70149f4dacc0d0c7ad92b59ac875e4"
dependencies = [
"bumpalo",
"leb128",
"memchr",
"unicode-width 0.2.0",
"wasm-encoder 0.222.0",
"wasm-encoder 0.223.0",
]
[[package]]
name = "wat"
version = "1.222.0"
version = "1.223.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fde61b4b52f9a84ae31b5e8902a2cd3162ea45d8bf564c729c3288fe52f4334"
checksum = "662786915c427e4918ff01eabb3c4756d4d947cd8f635761526b4cc9da2eaaad"
dependencies = [
"wast",
]
@ -6100,7 +6117,7 @@ dependencies = [
"rayon",
"serde",
"serde_json",
"syn 2.0.94",
"syn 2.0.96",
"windows-metadata",
]
@ -6133,7 +6150,7 @@ checksum = "9107ddc059d5b6fbfbffdfa7a7fe3e22a226def0b2608f72e9d552763d3e1ad7"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -6144,7 +6161,7 @@ checksum = "29bee4b38ea3cde66011baa44dba677c432a78593e202392d1e9070cf2a7fca7"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -6376,9 +6393,9 @@ checksum = "1e9df38ee2d2c3c5948ea468a8406ff0db0b29ae1ffde1bcf20ef305bcc95c51"
[[package]]
name = "xattr"
version = "1.3.1"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8da84f1a25939b27f6820d92aed108f83ff920fdf11a7b19366c27c4cda81d4f"
checksum = "e105d177a3871454f754b33bb0ee637ecaaac997446375fd3e5d43a2ed00c909"
dependencies = [
"libc",
"linux-raw-sys",
@ -6423,7 +6440,7 @@ checksum = "2380878cad4ac9aac1e2435f3eb4020e8374b5f13c296cb75b4620ff8e229154"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
"synstructure",
]
@ -6445,7 +6462,7 @@ checksum = "fa4f8080344d4671fb4e831a13ad1e68092748387dfc4f55e356242fae12ce3e"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]
[[package]]
@ -6465,7 +6482,7 @@ checksum = "595eed982f7d355beb85837f651fa22e90b3c044842dc7f2c2842c086f295808"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
"synstructure",
]
@ -6488,5 +6505,5 @@ checksum = "6eafa6dfb17584ea3e2bd6e76e0cc15ad7af12b09abdd1ca55961bed9b1063c6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.94",
"syn 2.0.96",
]

2
LICENSE-MIT
View file

@ -1,3 +1,5 @@
Copyright (c) The Rust Project Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the

35
compiler/rustc_abi/src/extern_abi/mod.rs
View file

@ -45,6 +45,9 @@ pub enum ExternAbi {
PtxKernel,
Msp430Interrupt,
X86Interrupt,
/// An entry-point function called by the GPU's host
// FIXME: should not be callable from Rust on GPU targets, is for host's use only
GpuKernel,
EfiApi,
AvrInterrupt,
AvrNonBlockingInterrupt,
@ -122,6 +125,7 @@ const AbiDatas: &[AbiData] = &[
AbiData { abi: Abi::PtxKernel, name: "ptx-kernel" },
AbiData { abi: Abi::Msp430Interrupt, name: "msp430-interrupt" },
AbiData { abi: Abi::X86Interrupt, name: "x86-interrupt" },
AbiData { abi: Abi::GpuKernel, name: "gpu-kernel" },
AbiData { abi: Abi::EfiApi, name: "efiapi" },
AbiData { abi: Abi::AvrInterrupt, name: "avr-interrupt" },
AbiData { abi: Abi::AvrNonBlockingInterrupt, name: "avr-non-blocking-interrupt" },
@ -239,6 +243,10 @@ pub fn is_stable(name: &str) -> Result<(), AbiDisabled> {
feature: sym::abi_x86_interrupt,
explain: "x86-interrupt ABI is experimental and subject to change",
}),
"gpu-kernel" => Err(AbiDisabled::Unstable {
feature: sym::abi_gpu_kernel,
explain: "gpu-kernel ABI is experimental and subject to change",
}),
"avr-interrupt" | "avr-non-blocking-interrupt" => Err(AbiDisabled::Unstable {
feature: sym::abi_avr_interrupt,
explain: "avr-interrupt and avr-non-blocking-interrupt ABIs are experimental and subject to change",
@ -293,20 +301,21 @@ impl Abi {
PtxKernel => 19,
Msp430Interrupt => 20,
X86Interrupt => 21,
EfiApi => 22,
AvrInterrupt => 23,
AvrNonBlockingInterrupt => 24,
CCmseNonSecureCall => 25,
CCmseNonSecureEntry => 26,
GpuKernel => 22,
EfiApi => 23,
AvrInterrupt => 24,
AvrNonBlockingInterrupt => 25,
CCmseNonSecureCall => 26,
CCmseNonSecureEntry => 27,
// Cross-platform ABIs
System { unwind: false } => 27,
System { unwind: true } => 28,
RustIntrinsic => 29,
RustCall => 30,
Unadjusted => 31,
RustCold => 32,
RiscvInterruptM => 33,
RiscvInterruptS => 34,
System { unwind: false } => 28,
System { unwind: true } => 29,
RustIntrinsic => 30,
RustCall => 31,
Unadjusted => 32,
RustCold => 33,
RiscvInterruptM => 34,
RiscvInterruptS => 35,
};
debug_assert!(
AbiDatas

2
compiler/rustc_ast/src/attr/mod.rs
View file

@ -723,6 +723,8 @@ impl MetaItemLit {
pub trait AttributeExt: Debug {
fn id(&self) -> AttrId;
/// For a single-segment attribute (i.e., `#[attr]` and not `#[path::atrr]`),
/// return the name of the attribute, else return the empty identifier.
fn name_or_empty(&self) -> Symbol {
self.ident().unwrap_or_else(Ident::empty).name
}

17
compiler/rustc_ast_lowering/src/delegation.rs
View file

@ -188,7 +188,14 @@ impl<'hir> LoweringContext<'_, 'hir> {
) -> hir::FnSig<'hir> {
let header = if let Some(local_sig_id) = sig_id.as_local() {
match self.resolver.delegation_fn_sigs.get(&local_sig_id) {
Some(sig) => self.lower_fn_header(sig.header, hir::Safety::Safe),
Some(sig) => self.lower_fn_header(
sig.header,
// HACK: we override the default safety instead of generating attributes from the ether.
// We are not forwarding the attributes, as the delegation fn sigs are collected on the ast,
// and here we need the hir attributes.
if sig.target_feature { hir::Safety::Unsafe } else { hir::Safety::Safe },
&[],
),
None => self.generate_header_error(),
}
} else {
@ -198,7 +205,11 @@ impl<'hir> LoweringContext<'_, 'hir> {
Asyncness::No => hir::IsAsync::NotAsync,
};
hir::FnHeader {
safety: sig.safety,
safety: if self.tcx.codegen_fn_attrs(sig_id).safe_target_features {
hir::HeaderSafety::SafeTargetFeatures
} else {
hir::HeaderSafety::Normal(sig.safety)
},
constness: self.tcx.constness(sig_id),
asyncness,
abi: sig.abi,
@ -384,7 +395,7 @@ impl<'hir> LoweringContext<'_, 'hir> {
fn generate_header_error(&self) -> hir::FnHeader {
hir::FnHeader {
safety: hir::Safety::Safe,
safety: hir::Safety::Safe.into(),
constness: hir::Constness::NotConst,
asyncness: hir::IsAsync::NotAsync,
abi: abi::Abi::Rust,

28
compiler/rustc_ast_lowering/src/item.rs
View file

@ -231,7 +231,7 @@ impl<'hir> LoweringContext<'_, 'hir> {
});
let sig = hir::FnSig {
decl,
header: this.lower_fn_header(*header, hir::Safety::Safe),
header: this.lower_fn_header(*header, hir::Safety::Safe, attrs),
span: this.lower_span(*fn_sig_span),
};
hir::ItemKind::Fn { sig, generics, body: body_id, has_body: body.is_some() }
@ -610,7 +610,7 @@ impl<'hir> LoweringContext<'_, 'hir> {
fn lower_foreign_item(&mut self, i: &ForeignItem) -> &'hir hir::ForeignItem<'hir> {
let hir_id = hir::HirId::make_owner(self.current_hir_id_owner.def_id);
let owner_id = hir_id.expect_owner();
self.lower_attrs(hir_id, &i.attrs);
let attrs = self.lower_attrs(hir_id, &i.attrs);
let item = hir::ForeignItem {
owner_id,
ident: self.lower_ident(i.ident),
@ -634,7 +634,7 @@ impl<'hir> LoweringContext<'_, 'hir> {
});
// Unmarked safety in unsafe block defaults to unsafe.
let header = self.lower_fn_header(sig.header, hir::Safety::Unsafe);
let header = self.lower_fn_header(sig.header, hir::Safety::Unsafe, attrs);
hir::ForeignItemKind::Fn(
hir::FnSig { header, decl, span: self.lower_span(sig.span) },
@ -776,6 +776,7 @@ impl<'hir> LoweringContext<'_, 'hir> {
i.id,
FnDeclKind::Trait,
sig.header.coroutine_kind,
attrs,
);
(generics, hir::TraitItemKind::Fn(sig, hir::TraitFn::Required(names)), false)
}
@ -795,6 +796,7 @@ impl<'hir> LoweringContext<'_, 'hir> {
i.id,
FnDeclKind::Trait,
sig.header.coroutine_kind,
attrs,
);
(generics, hir::TraitItemKind::Fn(sig, hir::TraitFn::Provided(body_id)), true)
}
@ -911,6 +913,7 @@ impl<'hir> LoweringContext<'_, 'hir> {
i.id,
if self.is_in_trait_impl { FnDeclKind::Impl } else { FnDeclKind::Inherent },
sig.header.coroutine_kind,
attrs,
);
(generics, hir::ImplItemKind::Fn(sig, body_id))
@ -1339,8 +1342,9 @@ impl<'hir> LoweringContext<'_, 'hir> {
id: NodeId,
kind: FnDeclKind,
coroutine_kind: Option<CoroutineKind>,
attrs: &[hir::Attribute],
) -> (&'hir hir::Generics<'hir>, hir::FnSig<'hir>) {
let header = self.lower_fn_header(sig.header, hir::Safety::Safe);
let header = self.lower_fn_header(sig.header, hir::Safety::Safe, attrs);
let itctx = ImplTraitContext::Universal;
let (generics, decl) = self.lower_generics(generics, id, itctx, |this| {
this.lower_fn_decl(&sig.decl, id, sig.span, kind, coroutine_kind)
@ -1352,14 +1356,28 @@ impl<'hir> LoweringContext<'_, 'hir> {
&mut self,
h: FnHeader,
default_safety: hir::Safety,
attrs: &[hir::Attribute],
) -> hir::FnHeader {
let asyncness = if let Some(CoroutineKind::Async { span, .. }) = h.coroutine_kind {
hir::IsAsync::Async(span)
} else {
hir::IsAsync::NotAsync
};
let safety = self.lower_safety(h.safety, default_safety);
// Treat safe `#[target_feature]` functions as unsafe, but also remember that we did so.
let safety = if attrs.iter().any(|attr| attr.has_name(sym::target_feature))
&& safety.is_safe()
&& !self.tcx.sess.target.is_like_wasm
{
hir::HeaderSafety::SafeTargetFeatures
} else {
safety.into()
};
hir::FnHeader {
safety: self.lower_safety(h.safety, default_safety),
safety,
asyncness,
constness: self.lower_constness(h.constness),
abi: self.lower_extern(h.ext),

15
compiler/rustc_attr_data_structures/src/stability.rs
View file

@ -101,6 +101,16 @@ impl PartialConstStability {
}
}
#[derive(Encodable, Decodable, PartialEq, Copy, Clone, Debug, Eq, Hash)]
#[derive(HashStable_Generic)]
pub enum AllowedThroughUnstableModules {
/// This does not get a deprecation warning. We still generally would prefer people to use the
/// fully stable path, and a warning will likely be emitted in the future.
WithoutDeprecation,
/// Emit the given deprecation warning.
WithDeprecation(Symbol),
}
/// The available stability levels.
#[derive(Encodable, Decodable, PartialEq, Copy, Clone, Debug, Eq, Hash)]
#[derive(HashStable_Generic)]
@ -137,9 +147,8 @@ pub enum StabilityLevel {
Stable {
/// Rust release which stabilized this feature.
since: StableSince,
/// Is this item allowed to be referred to on stable, despite being contained in unstable
/// modules?
allowed_through_unstable_modules: bool,
/// This is `Some` if this item allowed to be referred to on stable via unstable modules.
allowed_through_unstable_modules: Option<AllowedThroughUnstableModules>,
},
}

21
compiler/rustc_attr_parsing/src/attributes/stability.rs
View file

@ -6,8 +6,8 @@ use rustc_ast::MetaItem;
use rustc_ast::attr::AttributeExt;
use rustc_ast_pretty::pprust;
use rustc_attr_data_structures::{
ConstStability, DefaultBodyStability, Stability, StabilityLevel, StableSince, UnstableReason,
VERSION_PLACEHOLDER,
AllowedThroughUnstableModules, ConstStability, DefaultBodyStability, Stability, StabilityLevel,
StableSince, UnstableReason, VERSION_PLACEHOLDER,
};
use rustc_errors::ErrorGuaranteed;
use rustc_session::Session;
@ -24,11 +24,16 @@ pub fn find_stability(
item_sp: Span,
) -> Option<(Stability, Span)> {
let mut stab: Option<(Stability, Span)> = None;
let mut allowed_through_unstable_modules = false;
let mut allowed_through_unstable_modules = None;
for attr in attrs {
match attr.name_or_empty() {
sym::rustc_allowed_through_unstable_modules => allowed_through_unstable_modules = true,
sym::rustc_allowed_through_unstable_modules => {
allowed_through_unstable_modules = Some(match attr.value_str() {
Some(msg) => AllowedThroughUnstableModules::WithDeprecation(msg),
None => AllowedThroughUnstableModules::WithoutDeprecation,
})
}
sym::unstable => {
if stab.is_some() {
sess.dcx().emit_err(session_diagnostics::MultipleStabilityLevels {
@ -56,15 +61,15 @@ pub fn find_stability(
}
}
if allowed_through_unstable_modules {
if let Some(allowed_through_unstable_modules) = allowed_through_unstable_modules {
match &mut stab {
Some((
Stability {
level: StabilityLevel::Stable { allowed_through_unstable_modules, .. },
level: StabilityLevel::Stable { allowed_through_unstable_modules: in_stab, .. },
..
},
_,
)) => *allowed_through_unstable_modules = true,
)) => *in_stab = Some(allowed_through_unstable_modules),
_ => {
sess.dcx()
.emit_err(session_diagnostics::RustcAllowedUnstablePairing { span: item_sp });
@ -283,7 +288,7 @@ fn parse_stability(sess: &Session, attr: &impl AttributeExt) -> Option<(Symbol,
match feature {
Ok(feature) => {
let level = StabilityLevel::Stable { since, allowed_through_unstable_modules: false };
let level = StabilityLevel::Stable { since, allowed_through_unstable_modules: None };
Some((feature, level))
}
Err(ErrorGuaranteed { .. }) => None,

105
compiler/rustc_borrowck/src/dataflow.rs
View file

@ -187,19 +187,28 @@ struct OutOfScopePrecomputer<'a, 'tcx> {
borrows_out_of_scope_at_location: FxIndexMap<Location, Vec<BorrowIndex>>,
}
impl<'a, 'tcx> OutOfScopePrecomputer<'a, 'tcx> {
fn new(body: &'a Body<'tcx>, regioncx: &'a RegionInferenceContext<'tcx>) -> Self {
OutOfScopePrecomputer {
impl<'tcx> OutOfScopePrecomputer<'_, 'tcx> {
fn compute(
body: &Body<'tcx>,
regioncx: &RegionInferenceContext<'tcx>,
borrow_set: &BorrowSet<'tcx>,
) -> FxIndexMap<Location, Vec<BorrowIndex>> {
let mut prec = OutOfScopePrecomputer {
visited: DenseBitSet::new_empty(body.basic_blocks.len()),
visit_stack: vec![],
body,
regioncx,
borrows_out_of_scope_at_location: FxIndexMap::default(),
};
for (borrow_index, borrow_data) in borrow_set.iter_enumerated() {
let borrow_region = borrow_data.region;
let location = borrow_data.reserve_location;
prec.precompute_borrows_out_of_scope(borrow_index, borrow_region, location);
}
}
}
impl<'tcx> OutOfScopePrecomputer<'_, 'tcx> {
prec.borrows_out_of_scope_at_location
}
fn precompute_borrows_out_of_scope(
&mut self,
borrow_index: BorrowIndex,
@ -280,15 +289,7 @@ pub fn calculate_borrows_out_of_scope_at_location<'tcx>(
regioncx: &RegionInferenceContext<'tcx>,
borrow_set: &BorrowSet<'tcx>,
) -> FxIndexMap<Location, Vec<BorrowIndex>> {
let mut prec = OutOfScopePrecomputer::new(body, regioncx);
for (borrow_index, borrow_data) in borrow_set.iter_enumerated() {
let borrow_region = borrow_data.region;
let location = borrow_data.reserve_location;
prec.precompute_borrows_out_of_scope(borrow_index, borrow_region, location);
}
prec.borrows_out_of_scope_at_location
OutOfScopePrecomputer::compute(body, regioncx, borrow_set)
}
struct PoloniusOutOfScopePrecomputer<'a, 'tcx> {
@ -300,19 +301,30 @@ struct PoloniusOutOfScopePrecomputer<'a, 'tcx> {
loans_out_of_scope_at_location: FxIndexMap<Location, Vec<BorrowIndex>>,
}
impl<'a, 'tcx> PoloniusOutOfScopePrecomputer<'a, 'tcx> {
fn new(body: &'a Body<'tcx>, regioncx: &'a RegionInferenceContext<'tcx>) -> Self {
Self {
impl<'tcx> PoloniusOutOfScopePrecomputer<'_, 'tcx> {
fn compute(
body: &Body<'tcx>,
regioncx: &RegionInferenceContext<'tcx>,
borrow_set: &BorrowSet<'tcx>,
) -> FxIndexMap<Location, Vec<BorrowIndex>> {
// The in-tree polonius analysis computes loans going out of scope using the
// set-of-loans model.
let mut prec = PoloniusOutOfScopePrecomputer {
visited: DenseBitSet::new_empty(body.basic_blocks.len()),
visit_stack: vec![],
body,
regioncx,
loans_out_of_scope_at_location: FxIndexMap::default(),
};
for (loan_idx, loan_data) in borrow_set.iter_enumerated() {
let issuing_region = loan_data.region;
let loan_issued_at = loan_data.reserve_location;
prec.precompute_loans_out_of_scope(loan_idx, issuing_region, loan_issued_at);
}
}
}
impl<'tcx> PoloniusOutOfScopePrecomputer<'_, 'tcx> {
prec.loans_out_of_scope_at_location
}
/// Loans are in scope while they are live: whether they are contained within any live region.
/// In the location-insensitive analysis, a loan will be contained in a region if the issuing
/// region can reach it in the subset graph. So this is a reachability problem.
@ -325,10 +337,17 @@ impl<'tcx> PoloniusOutOfScopePrecomputer<'_, 'tcx> {
let sccs = self.regioncx.constraint_sccs();
let universal_regions = self.regioncx.universal_regions();
// The loop below was useful for the location-insensitive analysis but shouldn't be
// impactful in the location-sensitive case. It seems that it does, however, as without it a
// handful of tests fail. That likely means some liveness or outlives data related to choice
// regions is missing
// FIXME: investigate the impact of loans traversing applied member constraints and why some
// tests fail otherwise.
//
// We first handle the cases where the loan doesn't go out of scope, depending on the
// issuing region's successors.
for successor in graph::depth_first_search(&self.regioncx.region_graph(), issuing_region) {
// 1. Via applied member constraints
// Via applied member constraints
//
// The issuing region can flow into the choice regions, and they are either:
// - placeholders or free regions themselves,
@ -346,14 +365,6 @@ impl<'tcx> PoloniusOutOfScopePrecomputer<'_, 'tcx> {
return;
}
}
// 2. Via regions that are live at all points: placeholders and free regions.
//
// If the issuing region outlives such a region, its loan escapes the function and
// cannot go out of scope. We can early return.
if self.regioncx.is_region_live_at_all_points(successor) {
return;
}
}
let first_block = loan_issued_at.block;
@ -461,34 +472,12 @@ impl<'a, 'tcx> Borrows<'a, 'tcx> {
regioncx: &RegionInferenceContext<'tcx>,
borrow_set: &'a BorrowSet<'tcx>,
) -> Self {
let mut borrows_out_of_scope_at_location =
calculate_borrows_out_of_scope_at_location(body, regioncx, borrow_set);
// The in-tree polonius analysis computes loans going out of scope using the set-of-loans
// model, and makes sure they're identical to the existing computation of the set-of-points
// model.
if tcx.sess.opts.unstable_opts.polonius.is_next_enabled() {
let mut polonius_prec = PoloniusOutOfScopePrecomputer::new(body, regioncx);
for (loan_idx, loan_data) in borrow_set.iter_enumerated() {
let issuing_region = loan_data.region;
let loan_issued_at = loan_data.reserve_location;
polonius_prec.precompute_loans_out_of_scope(
loan_idx,
issuing_region,
loan_issued_at,
);
}
assert_eq!(
borrows_out_of_scope_at_location, polonius_prec.loans_out_of_scope_at_location,
"polonius loan scopes differ from NLL borrow scopes, for body {:?}",
body.span,
);
borrows_out_of_scope_at_location = polonius_prec.loans_out_of_scope_at_location;
}
let borrows_out_of_scope_at_location =
if !tcx.sess.opts.unstable_opts.polonius.is_next_enabled() {
calculate_borrows_out_of_scope_at_location(body, regioncx, borrow_set)
} else {
PoloniusOutOfScopePrecomputer::compute(body, regioncx, borrow_set)
};
Borrows { tcx, body, borrow_set, borrows_out_of_scope_at_location }
}

18
compiler/rustc_borrowck/src/lib.rs
View file

@ -182,9 +182,6 @@ fn do_mir_borrowck<'tcx>(
let location_table = PoloniusLocationTable::new(body);
let move_data = MoveData::gather_moves(body, tcx, |_| true);
let promoted_move_data = promoted
.iter_enumerated()
.map(|(idx, body)| (idx, MoveData::gather_moves(body, tcx, |_| true)));
let flow_inits = MaybeInitializedPlaces::new(tcx, body, &move_data)
.iterate_to_fixpoint(tcx, body, Some("borrowck"))
@ -242,10 +239,14 @@ fn do_mir_borrowck<'tcx>(
false
};
for (idx, move_data) in promoted_move_data {
// While promoteds should mostly be correct by construction, we need to check them for
// invalid moves to detect moving out of arrays:`struct S; fn main() { &([S][0]); }`.
for promoted_body in &promoted {
use rustc_middle::mir::visit::Visitor;
let promoted_body = &promoted[idx];
// This assumes that we won't use some of the fields of the `promoted_mbcx`
// when detecting and reporting move errors. While it would be nice to move
// this check out of `MirBorrowckCtxt`, actually doing so is far from trivial.
let move_data = MoveData::gather_moves(promoted_body, tcx, |_| true);
let mut promoted_mbcx = MirBorrowckCtxt {
infcx: &infcx,
body: promoted_body,
@ -270,9 +271,6 @@ fn do_mir_borrowck<'tcx>(
move_errors: Vec::new(),
diags_buffer,
};
MoveVisitor { ctxt: &mut promoted_mbcx }.visit_body(promoted_body);
promoted_mbcx.report_move_errors();
struct MoveVisitor<'a, 'b, 'infcx, 'tcx> {
ctxt: &'a mut MirBorrowckCtxt<'b, 'infcx, 'tcx>,
}
@ -284,6 +282,8 @@ fn do_mir_borrowck<'tcx>(
}
}
}
MoveVisitor { ctxt: &mut promoted_mbcx }.visit_body(promoted_body);
promoted_mbcx.report_move_errors();
}
let mut mbcx = MirBorrowckCtxt {

10
compiler/rustc_borrowck/src/nll.rs
View file

@ -103,7 +103,7 @@ pub(crate) fn compute_regions<'a, 'tcx>(
constraints,
universal_region_relations,
opaque_type_values,
mut polonius_context,
polonius_context,
} = type_check::type_check(
infcx,
body,
@ -142,10 +142,10 @@ pub(crate) fn compute_regions<'a, 'tcx>(
location_map,
);
// If requested for `-Zpolonius=next`, convert NLL constraints to localized outlives
// constraints.
let localized_outlives_constraints = polonius_context.as_mut().map(|polonius_context| {
polonius_context.create_localized_constraints(infcx.tcx, &regioncx, body)
// If requested for `-Zpolonius=next`, convert NLL constraints to localized outlives constraints
// and use them to compute loan liveness.
let localized_outlives_constraints = polonius_context.as_ref().map(|polonius_context| {
polonius_context.compute_loan_liveness(infcx.tcx, &mut regioncx, body, borrow_set)
});
// If requested: dump NLL facts, and run legacy polonius analysis.

307
compiler/rustc_borrowck/src/polonius/loan_liveness.rs Normal file
View file

@ -0,0 +1,307 @@
use std::collections::{BTreeMap, BTreeSet};
use rustc_data_structures::fx::{FxHashMap, FxHashSet, FxIndexSet};
use rustc_middle::mir::visit::Visitor;
use rustc_middle::mir::{
Body, Local, Location, Place, Rvalue, Statement, StatementKind, Terminator, TerminatorKind,
};
use rustc_middle::ty::{RegionVid, TyCtxt};
use rustc_mir_dataflow::points::PointIndex;
use super::{LiveLoans, LocalizedOutlivesConstraintSet};
use crate::constraints::OutlivesConstraint;
use crate::dataflow::BorrowIndex;
use crate::region_infer::values::LivenessValues;
use crate::type_check::Locations;
use crate::{BorrowSet, PlaceConflictBias, places_conflict};
/// Compute loan reachability, stop at kills, and trace loan liveness throughout the CFG, by
/// traversing the full graph of constraints that combines:
/// - the localized constraints (the physical edges),
/// - with the constraints that hold at all points (the logical edges).
pub(super) fn compute_loan_liveness<'tcx>(
tcx: TyCtxt<'tcx>,
body: &Body<'tcx>,
liveness: &LivenessValues,
outlives_constraints: impl Iterator<Item = OutlivesConstraint<'tcx>>,
borrow_set: &BorrowSet<'tcx>,
localized_outlives_constraints: &LocalizedOutlivesConstraintSet,
) -> LiveLoans {
let mut live_loans = LiveLoans::new(borrow_set.len());
// FIXME: it may be preferable for kills to be encoded in the edges themselves, to simplify and
// likely make traversal (and constraint generation) more efficient. We also display kills on
// edges when visualizing the constraint graph anyways.
let kills = collect_kills(body, tcx, borrow_set);
// Create the full graph with the physical edges we've localized earlier, and the logical edges
// of constraints that hold at all points.
let logical_constraints =
outlives_constraints.filter(|c| matches!(c.locations, Locations::All(_)));
let graph = LocalizedConstraintGraph::new(&localized_outlives_constraints, logical_constraints);
let mut visited = FxHashSet::default();
let mut stack = Vec::new();
// Compute reachability per loan by traversing each loan's subgraph starting from where it is
// introduced.
for (loan_idx, loan) in borrow_set.iter_enumerated() {
visited.clear();
stack.clear();
let start_node = LocalizedNode {
region: loan.region,
point: liveness.point_from_location(loan.reserve_location),
};
stack.push(start_node);
while let Some(node) = stack.pop() {
if !visited.insert(node) {
continue;
}
// Record the loan as being live on entry to this point.
live_loans.insert(node.point, loan_idx);
// Here, we have a conundrum. There's currently a weakness in our theory, in that
// we're using a single notion of reachability to represent what used to be _two_
// different transitive closures. It didn't seem impactful when coming up with the
// single-graph and reachability through space (regions) + time (CFG) concepts, but in
// practice the combination of time-traveling with kills is more impactful than
// initially anticipated.
//
// Kills should prevent a loan from reaching its successor points in the CFG, but not
// while time-traveling: we're not actually at that CFG point, but looking for
// predecessor regions that contain the loan. One of the two TCs we had pushed the
// transitive subset edges to each point instead of having backward edges, and the
// problem didn't exist before. In the abstract, naive reachability is not enough to
// model this, we'd need a slightly different solution. For example, maybe with a
// two-step traversal:
// - at each point we first traverse the subgraph (and possibly time-travel) looking for
// exit nodes while ignoring kills,
// - and then when we're back at the current point, we continue normally.
//
// Another (less annoying) subtlety is that kills and the loan use-map are
// flow-insensitive. Kills can actually appear in places before a loan is introduced, or
// at a location that is actually unreachable in the CFG from the introduction point,
// and these can also be encountered during time-traveling.
//
// The simplest change that made sense to "fix" the issues above is taking into
// account kills that are:
// - reachable from the introduction point
// - encountered during forward traversal. Note that this is not transitive like the
// two-step traversal described above: only kills encountered on exit via a backward
// edge are ignored.
//
// In our test suite, there are a couple of cases where kills are encountered while
// time-traveling, however as far as we can tell, always in cases where they would be
// unreachable. We have reason to believe that this is a property of the single-graph
// approach (but haven't proved it yet):
// - reachable kills while time-traveling would also be encountered via regular
// traversal
// - it makes _some_ sense to ignore unreachable kills, but subtleties around dead code
// in general need to be better thought through (like they were for NLLs).
// - ignoring kills is a conservative approximation: the loan is still live and could
// cause false positive errors at another place access. Soundness issues in this
// domain should look more like the absence of reachability instead.
//
// This is enough in practice to pass tests, and therefore is what we have implemented
// for now.
//
// FIXME: all of the above. Analyze potential unsoundness, possibly in concert with a
// borrowck implementation in a-mir-formality, fuzzing, or manually crafting
// counter-examples.
// Continuing traversal will depend on whether the loan is killed at this point, and
// whether we're time-traveling.
let current_location = liveness.location_from_point(node.point);
let is_loan_killed =
kills.get(&current_location).is_some_and(|kills| kills.contains(&loan_idx));
for succ in graph.outgoing_edges(node) {
// If the loan is killed at this point, it is killed _on exit_. But only during
// forward traversal.
if is_loan_killed {
let destination = liveness.location_from_point(succ.point);
if current_location.is_predecessor_of(destination, body) {
continue;
}
}
stack.push(succ);
}
}
}
live_loans
}
/// The localized constraint graph indexes the physical and logical edges to compute a given node's
/// successors during traversal.
struct LocalizedConstraintGraph {
/// The actual, physical, edges we have recorded for a given node.
edges: FxHashMap<LocalizedNode, FxIndexSet<LocalizedNode>>,
/// The logical edges representing the outlives constraints that hold at all points in the CFG,
/// which we don't localize to avoid creating a lot of unnecessary edges in the graph. Some CFGs
/// can be big, and we don't need to create such a physical edge for every point in the CFG.
logical_edges: FxHashMap<RegionVid, FxIndexSet<RegionVid>>,
}
/// A node in the graph to be traversed, one of the two vertices of a localized outlives constraint.
#[derive(Copy, Clone, PartialEq, Eq, Hash, Debug)]
struct LocalizedNode {
region: RegionVid,
point: PointIndex,
}
impl LocalizedConstraintGraph {
/// Traverses the constraints and returns the indexed graph of edges per node.
fn new<'tcx>(
constraints: &LocalizedOutlivesConstraintSet,
logical_constraints: impl Iterator<Item = OutlivesConstraint<'tcx>>,
) -> Self {
let mut edges: FxHashMap<_, FxIndexSet<_>> = FxHashMap::default();
for constraint in &constraints.outlives {
let source = LocalizedNode { region: constraint.source, point: constraint.from };
let target = LocalizedNode { region: constraint.target, point: constraint.to };
edges.entry(source).or_default().insert(target);
}
let mut logical_edges: FxHashMap<_, FxIndexSet<_>> = FxHashMap::default();
for constraint in logical_constraints {
logical_edges.entry(constraint.sup).or_default().insert(constraint.sub);
}
LocalizedConstraintGraph { edges, logical_edges }
}
/// Returns the outgoing edges of a given node, not its transitive closure.
fn outgoing_edges(&self, node: LocalizedNode) -> impl Iterator<Item = LocalizedNode> + use<'_> {
// The outgoing edges are:
// - the physical edges present at this node,
// - the materialized logical edges that exist virtually at all points for this node's
// region, localized at this point.
let physical_edges =
self.edges.get(&node).into_iter().flat_map(|targets| targets.iter().copied());
let materialized_edges =
self.logical_edges.get(&node.region).into_iter().flat_map(move |targets| {
targets
.iter()
.copied()
.map(move |target| LocalizedNode { point: node.point, region: target })
});
physical_edges.chain(materialized_edges)
}
}
/// Traverses the MIR and collects kills.
fn collect_kills<'tcx>(
body: &Body<'tcx>,
tcx: TyCtxt<'tcx>,
borrow_set: &BorrowSet<'tcx>,
) -> BTreeMap<Location, BTreeSet<BorrowIndex>> {
let mut collector = KillsCollector { borrow_set, tcx, body, kills: BTreeMap::default() };
for (block, data) in body.basic_blocks.iter_enumerated() {
collector.visit_basic_block_data(block, data);
}
collector.kills
}
struct KillsCollector<'a, 'tcx> {
body: &'a Body<'tcx>,
tcx: TyCtxt<'tcx>,
borrow_set: &'a BorrowSet<'tcx>,
/// The set of loans killed at each location.
kills: BTreeMap<Location, BTreeSet<BorrowIndex>>,
}
// This visitor has a similar structure to the `Borrows` dataflow computation with respect to kills,
// and the datalog polonius fact generation for the `loan_killed_at` relation.
impl<'tcx> KillsCollector<'_, 'tcx> {
/// Records the borrows on the specified place as `killed`. For example, when assigning to a
/// local, or on a call's return destination.
fn record_killed_borrows_for_place(&mut self, place: Place<'tcx>, location: Location) {
// For the reasons described in graph traversal, we also filter out kills
// unreachable from the loan's introduction point, as they would stop traversal when
// e.g. checking for reachability in the subset graph through invariance constraints
// higher up.
let filter_unreachable_kills = |loan| {
let introduction = self.borrow_set[loan].reserve_location;
let reachable = introduction.is_predecessor_of(location, self.body);
reachable
};
let other_borrows_of_local = self
.borrow_set
.local_map
.get(&place.local)
.into_iter()
.flat_map(|bs| bs.iter())
.copied();
// If the borrowed place is a local with no projections, all other borrows of this
// local must conflict. This is purely an optimization so we don't have to call
// `places_conflict` for every borrow.
if place.projection.is_empty() {
if !self.body.local_decls[place.local].is_ref_to_static() {
self.kills
.entry(location)
.or_default()
.extend(other_borrows_of_local.filter(|&loan| filter_unreachable_kills(loan)));
}
return;
}
// By passing `PlaceConflictBias::NoOverlap`, we conservatively assume that any given
// pair of array indices are not equal, so that when `places_conflict` returns true, we
// will be assured that two places being compared definitely denotes the same sets of
// locations.
let definitely_conflicting_borrows = other_borrows_of_local
.filter(|&i| {
places_conflict(
self.tcx,
self.body,
self.borrow_set[i].borrowed_place,
place,
PlaceConflictBias::NoOverlap,
)
})
.filter(|&loan| filter_unreachable_kills(loan));
self.kills.entry(location).or_default().extend(definitely_conflicting_borrows);
}
/// Records the borrows on the specified local as `killed`.
fn record_killed_borrows_for_local(&mut self, local: Local, location: Location) {
if let Some(borrow_indices) = self.borrow_set.local_map.get(&local) {
self.kills.entry(location).or_default().extend(borrow_indices.iter());
}
}
}
impl<'tcx> Visitor<'tcx> for KillsCollector<'_, 'tcx> {
fn visit_statement(&mut self, statement: &Statement<'tcx>, location: Location) {
// Make sure there are no remaining borrows for locals that have gone out of scope.
if let StatementKind::StorageDead(local) = statement.kind {
self.record_killed_borrows_for_local(local, location);
}
self.super_statement(statement, location);
}
fn visit_assign(&mut self, place: &Place<'tcx>, rvalue: &Rvalue<'tcx>, location: Location) {
// When we see `X = ...`, then kill borrows of `(*X).foo` and so forth.
self.record_killed_borrows_for_place(*place, location);
self.super_assign(place, rvalue, location);
}
fn visit_terminator(&mut self, terminator: &Terminator<'tcx>, location: Location) {
// A `Call` terminator's return value can be a local which has borrows, so we need to record
// those as killed as well.
if let TerminatorKind::Call { destination, .. } = terminator.kind {
self.record_killed_borrows_for_place(destination, location);
}
self.super_terminator(terminator, location);
}
}

32
compiler/rustc_borrowck/src/polonius/mod.rs
View file

@ -37,6 +37,7 @@ mod constraints;
mod dump;
pub(crate) mod legacy;
mod liveness_constraints;
mod loan_liveness;
mod typeck_constraints;
use std::collections::BTreeMap;
@ -49,8 +50,12 @@ use rustc_mir_dataflow::points::PointIndex;
pub(crate) use self::constraints::*;
pub(crate) use self::dump::dump_polonius_mir;
use self::liveness_constraints::create_liveness_constraints;
use self::loan_liveness::compute_loan_liveness;
use self::typeck_constraints::convert_typeck_constraints;
use crate::RegionInferenceContext;
use crate::dataflow::BorrowIndex;
use crate::{BorrowSet, RegionInferenceContext};
pub(crate) type LiveLoans = SparseBitMatrix<PointIndex, BorrowIndex>;
/// This struct holds the data needed to create the Polonius localized constraints.
pub(crate) struct PoloniusContext {
@ -82,14 +87,20 @@ impl PoloniusContext {
Self { live_region_variances: BTreeMap::new(), live_regions: None }
}
/// Creates a constraint set for `-Zpolonius=next` by:
/// Computes live loans using the set of loans model for `-Zpolonius=next`.
///
/// First, creates a constraint graph combining regions and CFG points, by:
/// - converting NLL typeck constraints to be localized
/// - encoding liveness constraints
pub(crate) fn create_localized_constraints<'tcx>(
///
/// Then, this graph is traversed, and combined with kills, reachability is recorded as loan
/// liveness, to be used by the loan scope and active loans computations.
pub(crate) fn compute_loan_liveness<'tcx>(
&self,
tcx: TyCtxt<'tcx>,
regioncx: &RegionInferenceContext<'tcx>,
regioncx: &mut RegionInferenceContext<'tcx>,
body: &Body<'tcx>,
borrow_set: &BorrowSet<'tcx>,
) -> LocalizedOutlivesConstraintSet {
let mut localized_outlives_constraints = LocalizedOutlivesConstraintSet::default();
convert_typeck_constraints(
@ -113,8 +124,17 @@ impl PoloniusContext {
&mut localized_outlives_constraints,
);
// FIXME: here, we can trace loan reachability in the constraint graph and record this as loan
// liveness for the next step in the chain, the NLL loan scope and active loans computations.
// Now that we have a complete graph, we can compute reachability to trace the liveness of
// loans for the next step in the chain, the NLL loan scope and active loans computations.
let live_loans = compute_loan_liveness(
tcx,
body,
regioncx.liveness_constraints(),
regioncx.outlives_constraints(),
borrow_set,
&localized_outlives_constraints,
);
regioncx.record_live_loans(live_loans);
localized_outlives_constraints
}

22
compiler/rustc_borrowck/src/polonius/typeck_constraints.rs
View file

@ -22,23 +22,11 @@ pub(super) fn convert_typeck_constraints<'tcx>(
for outlives_constraint in outlives_constraints {
match outlives_constraint.locations {
Locations::All(_) => {
// For now, turn logical constraints holding at all points into physical edges at
// every point in the graph.
// FIXME: encode this into *traversal* instead.
for (block, bb) in body.basic_blocks.iter_enumerated() {
let statement_count = bb.statements.len();
for statement_index in 0..=statement_count {
let current_location = Location { block, statement_index };
let current_point = liveness.point_from_location(current_location);
localized_outlives_constraints.push(LocalizedOutlivesConstraint {
source: outlives_constraint.sup,
from: current_point,
target: outlives_constraint.sub,
to: current_point,
});
}
}
// We don't turn constraints holding at all points into physical edges at every
// point in the graph. They are encoded into *traversal* instead: a given node's
// successors will combine these logical edges with the regular, physical, localized
// edges.
continue;
}
Locations::Single(location) => {

37
compiler/rustc_borrowck/src/region_infer/mod.rs
View file

@ -31,6 +31,7 @@ use crate::constraints::{ConstraintSccIndex, OutlivesConstraint, OutlivesConstra
use crate::dataflow::BorrowIndex;
use crate::diagnostics::{RegionErrorKind, RegionErrors, UniverseInfo};
use crate::member_constraints::{MemberConstraintSet, NllMemberConstraintIndex};
use crate::polonius::LiveLoans;
use crate::polonius::legacy::PoloniusOutput;
use crate::region_infer::reverse_sccs::ReverseSccGraph;
use crate::region_infer::values::{LivenessValues, RegionElement, RegionValues, ToElementIndex};
@ -2171,28 +2172,6 @@ impl<'tcx> RegionInferenceContext<'tcx> {
self.constraint_graph.region_graph(&self.constraints, self.universal_regions().fr_static)
}
/// Returns whether the given region is considered live at all points: whether it is a
/// placeholder or a free region.
pub(crate) fn is_region_live_at_all_points(&self, region: RegionVid) -> bool {
// FIXME: there must be a cleaner way to find this information. At least, when
// higher-ranked subtyping is abstracted away from the borrowck main path, we'll only
// need to check whether this is a universal region.
let origin = self.region_definition(region).origin;
let live_at_all_points = matches!(
origin,
NllRegionVariableOrigin::Placeholder(_) | NllRegionVariableOrigin::FreeRegion
);
live_at_all_points
}
/// Returns whether the `loan_idx` is live at the given `location`: whether its issuing
/// region is contained within the type of a variable that is live at this point.
/// Note: for now, the sets of live loans is only available when using `-Zpolonius=next`.
pub(crate) fn is_loan_live_at(&self, loan_idx: BorrowIndex, location: Location) -> bool {
let point = self.liveness_constraints.point_from_location(location);
self.liveness_constraints.is_loan_live_at(loan_idx, point)
}
/// Returns the representative `RegionVid` for a given SCC.
/// See `RegionTracker` for how a region variable ID is chosen.
///
@ -2208,6 +2187,20 @@ impl<'tcx> RegionInferenceContext<'tcx> {
pub(crate) fn liveness_constraints(&self) -> &LivenessValues {
&self.liveness_constraints
}
/// When using `-Zpolonius=next`, records the given live loans for the loan scopes and active
/// loans dataflow computations.
pub(crate) fn record_live_loans(&mut self, live_loans: LiveLoans) {
self.liveness_constraints.record_live_loans(live_loans);
}
/// Returns whether the `loan_idx` is live at the given `location`: whether its issuing
/// region is contained within the type of a variable that is live at this point.
/// Note: for now, the sets of live loans is only available when using `-Zpolonius=next`.
pub(crate) fn is_loan_live_at(&self, loan_idx: BorrowIndex, location: Location) -> bool {
let point = self.liveness_constraints.point_from_location(location);
self.liveness_constraints.is_loan_live_at(loan_idx, point)
}
}
impl<'tcx> RegionDefinition<'tcx> {

57
compiler/rustc_borrowck/src/region_infer/values.rs
View file

@ -11,6 +11,7 @@ use rustc_mir_dataflow::points::{DenseLocationMap, PointIndex};
use tracing::debug;
use crate::BorrowIndex;
use crate::polonius::LiveLoans;
rustc_index::newtype_index! {
/// A single integer representing a `ty::Placeholder`.
@ -50,29 +51,8 @@ pub(crate) struct LivenessValues {
/// region is live, only that it is.
points: Option<SparseIntervalMatrix<RegionVid, PointIndex>>,
/// When using `-Zpolonius=next`, for each point: the loans flowing into the live regions at
/// that point.
pub(crate) loans: Option<LiveLoans>,
}
/// Data used to compute the loans that are live at a given point in the CFG, when using
/// `-Zpolonius=next`.
pub(crate) struct LiveLoans {
/// The set of loans that flow into a given region. When individual regions are marked as live
/// in the CFG, these inflowing loans are recorded as live.
pub(crate) inflowing_loans: SparseBitMatrix<RegionVid, BorrowIndex>,
/// The set of loans that are live at a given point in the CFG.
pub(crate) live_loans: SparseBitMatrix<PointIndex, BorrowIndex>,
}
impl LiveLoans {
pub(crate) fn new(num_loans: usize) -> Self {
LiveLoans {
live_loans: SparseBitMatrix::new(num_loans),
inflowing_loans: SparseBitMatrix::new(num_loans),
}
}
/// When using `-Zpolonius=next`, the set of loans that are live at a given point in the CFG.
live_loans: Option<LiveLoans>,
}
impl LivenessValues {
@ -82,7 +62,7 @@ impl LivenessValues {
live_regions: None,
points: Some(SparseIntervalMatrix::new(location_map.num_points())),
location_map,
loans: None,
live_loans: None,
}
}
@ -95,7 +75,7 @@ impl LivenessValues {
live_regions: Some(Default::default()),
points: None,
location_map,
loans: None,
live_loans: None,
}
}
@ -129,13 +109,6 @@ impl LivenessValues {
} else if self.location_map.point_in_range(point) {
self.live_regions.as_mut().unwrap().insert(region);
}
// When available, record the loans flowing into this region as live at the given point.
if let Some(loans) = self.loans.as_mut() {
if let Some(inflowing) = loans.inflowing_loans.row(region) {
loans.live_loans.union_row(point, inflowing);
}
}
}
/// Records `region` as being live at all the given `points`.
@ -146,17 +119,6 @@ impl LivenessValues {
} else if points.iter().any(|point| self.location_map.point_in_range(point)) {
self.live_regions.as_mut().unwrap().insert(region);
}
// When available, record the loans flowing into this region as live at the given points.
if let Some(loans) = self.loans.as_mut() {
if let Some(inflowing) = loans.inflowing_loans.row(region) {
if !inflowing.is_empty() {
for point in points.iter() {
loans.live_loans.union_row(point, inflowing);
}
}
}
}
}
/// Records `region` as being live at all the control-flow points.
@ -213,12 +175,17 @@ impl LivenessValues {
self.location_map.to_location(point)
}
/// When using `-Zpolonius=next`, records the given live loans for the loan scopes and active
/// loans dataflow computations.
pub(crate) fn record_live_loans(&mut self, live_loans: LiveLoans) {
self.live_loans = Some(live_loans);
}
/// When using `-Zpolonius=next`, returns whether the `loan_idx` is live at the given `point`.
pub(crate) fn is_loan_live_at(&self, loan_idx: BorrowIndex, point: PointIndex) -> bool {
self.loans
self.live_loans
.as_ref()
.expect("Accessing live loans requires `-Zpolonius=next`")
.live_loans
.contains(point, loan_idx)
}
}

1
compiler/rustc_borrowck/src/renumber.rs
View file

@ -34,7 +34,6 @@ pub(crate) enum RegionCtxt {
Location(Location),
TyContext(TyContext),
Free(Symbol),
Bound(Symbol),
LateBound(Symbol),
Existential(Option<Symbol>),
Placeholder(Symbol),

67
compiler/rustc_borrowck/src/type_check/free_region_relations.rs
View file

@ -5,13 +5,14 @@ use rustc_infer::infer::canonical::QueryRegionConstraints;
use rustc_infer::infer::outlives::env::RegionBoundPairs;
use rustc_infer::infer::region_constraints::GenericKind;
use rustc_infer::infer::{InferCtxt, outlives};
use rustc_infer::traits::ScrubbedTraitError;
use rustc_middle::mir::ConstraintCategory;
use rustc_middle::traits::ObligationCause;
use rustc_middle::traits::query::OutlivesBound;
use rustc_middle::ty::{self, RegionVid, Ty, TypeVisitableExt};
use rustc_span::{ErrorGuaranteed, Span};
use rustc_trait_selection::error_reporting::InferCtxtErrorExt;
use rustc_trait_selection::solve::deeply_normalize;
use rustc_trait_selection::solve::NoSolution;
use rustc_trait_selection::traits::query::type_op::custom::CustomTypeOp;
use rustc_trait_selection::traits::query::type_op::{self, TypeOp};
use tracing::{debug, instrument};
use type_op::TypeOpOutput;
@ -229,24 +230,14 @@ impl<'tcx> UniversalRegionRelationsBuilder<'_, 'tcx> {
let mut constraints = vec![];
let mut known_type_outlives_obligations = vec![];
for bound in param_env.caller_bounds() {
let Some(mut outlives) = bound.as_type_outlives_clause() else { continue };
// In the new solver, normalize the type-outlives obligation assumptions.
if self.infcx.next_trait_solver() {
match deeply_normalize(
self.infcx.at(&ObligationCause::misc(span, defining_ty_def_id), param_env),
if let Some(outlives) = bound.as_type_outlives_clause() {
self.normalize_and_push_type_outlives_obligation(
outlives,
) {
Ok(normalized_outlives) => {
outlives = normalized_outlives;
}
Err(e) => {
self.infcx.err_ctxt().report_fulfillment_errors(e);
}
}
}
known_type_outlives_obligations.push(outlives);
span,
&mut known_type_outlives_obligations,
&mut constraints,
);
};
}
let unnormalized_input_output_tys = self
@ -356,6 +347,44 @@ impl<'tcx> UniversalRegionRelationsBuilder<'_, 'tcx> {
}
}
fn normalize_and_push_type_outlives_obligation(
&self,
mut outlives: ty::PolyTypeOutlivesPredicate<'tcx>,
span: Span,
known_type_outlives_obligations: &mut Vec<ty::PolyTypeOutlivesPredicate<'tcx>>,
constraints: &mut Vec<&QueryRegionConstraints<'tcx>>,
) {
// In the new solver, normalize the type-outlives obligation assumptions.
if self.infcx.next_trait_solver() {
let Ok(TypeOpOutput {
output: normalized_outlives,
constraints: constraints_normalize,
error_info: _,
}) = CustomTypeOp::new(
|ocx| {
ocx.deeply_normalize(
&ObligationCause::dummy_with_span(span),
self.param_env,
outlives,
)
.map_err(|_: Vec<ScrubbedTraitError<'tcx>>| NoSolution)
},
"normalize type outlives obligation",
)
.fully_perform(self.infcx, span)
else {
self.infcx.dcx().delayed_bug(format!("could not normalize {outlives:?}"));
return;
};
outlives = normalized_outlives;
if let Some(c) = constraints_normalize {
constraints.push(c);
}
}
known_type_outlives_obligations.push(outlives);
}
/// Update the type of a single local, which should represent
/// either the return type of the MIR or one of its arguments. At
/// the same time, compute and add any implied bounds that come

18
compiler/rustc_borrowck/src/type_check/liveness/mod.rs
View file

@ -38,11 +38,19 @@ pub(super) fn generate<'a, 'tcx>(
) {
debug!("liveness::generate");
let free_regions = regions_that_outlive_free_regions(
typeck.infcx.num_region_vars(),
&typeck.universal_regions,
&typeck.constraints.outlives_constraints,
);
// NLLs can avoid computing some liveness data here because its constraints are
// location-insensitive, but that doesn't work in polonius: locals whose type contains a region
// that outlives a free region are not necessarily live everywhere in a flow-sensitive setting,
// unlike NLLs.
let free_regions = if !typeck.tcx().sess.opts.unstable_opts.polonius.is_next_enabled() {
regions_that_outlive_free_regions(
typeck.infcx.num_region_vars(),
&typeck.universal_regions,
&typeck.constraints.outlives_constraints,
)
} else {
typeck.universal_regions.universal_regions_iter().collect()
};
let (relevant_live_locals, boring_locals) =
compute_relevant_live_locals(typeck.tcx(), &free_regions, body);

33
compiler/rustc_borrowck/src/type_check/liveness/trace.rs
View file

@ -16,7 +16,7 @@ use rustc_trait_selection::traits::query::type_op::{DropckOutlives, TypeOp, Type
use tracing::debug;
use crate::polonius;
use crate::region_infer::values::{self, LiveLoans};
use crate::region_infer::values;
use crate::type_check::liveness::local_use_map::LocalUseMap;
use crate::type_check::{NormalizeLocation, TypeChecker};
@ -44,37 +44,6 @@ pub(super) fn trace<'a, 'tcx>(
boring_locals: Vec<Local>,
) {
let local_use_map = &LocalUseMap::build(&relevant_live_locals, location_map, body);
// When using `-Zpolonius=next`, compute the set of loans that can reach a given region.
if typeck.tcx().sess.opts.unstable_opts.polonius.is_next_enabled() {
let borrow_set = &typeck.borrow_set;
let mut live_loans = LiveLoans::new(borrow_set.len());
let outlives_constraints = &typeck.constraints.outlives_constraints;
let graph = outlives_constraints.graph(typeck.infcx.num_region_vars());
let region_graph =
graph.region_graph(outlives_constraints, typeck.universal_regions.fr_static);
// Traverse each issuing region's constraints, and record the loan as flowing into the
// outlived region.
for (loan, issuing_region_data) in borrow_set.iter_enumerated() {
for succ in rustc_data_structures::graph::depth_first_search(
&region_graph,
issuing_region_data.region,
) {
// We don't need to mention that a loan flows into its issuing region.
if succ == issuing_region_data.region {
continue;
}
live_loans.inflowing_loans.insert(succ, loan);
}
}
// Store the inflowing loans in the liveness constraints: they will be used to compute live
// loans when liveness data is recorded there.
typeck.constraints.liveness_constraints.loans = Some(live_loans);
};
let cx = LivenessContext {
typeck,
body,

15
compiler/rustc_borrowck/src/type_check/mod.rs
View file

@ -1654,7 +1654,20 @@ impl<'a, 'tcx> TypeChecker<'a, 'tcx> {
match *cast_kind {
CastKind::PointerCoercion(PointerCoercion::ReifyFnPointer, coercion_source) => {
let is_implicit_coercion = coercion_source == CoercionSource::Implicit;
let src_sig = op.ty(body, tcx).fn_sig(tcx);
let src_ty = op.ty(body, tcx);
let mut src_sig = src_ty.fn_sig(tcx);
if let ty::FnDef(def_id, _) = src_ty.kind()
&& let ty::FnPtr(_, target_hdr) = *ty.kind()
&& tcx.codegen_fn_attrs(def_id).safe_target_features
&& target_hdr.safety.is_safe()
&& let Some(safe_sig) = tcx.adjust_target_feature_sig(
*def_id,
src_sig,
body.source.def_id(),
)
{
src_sig = safe_sig;
}
// HACK: This shouldn't be necessary... We can remove this when we actually
// get binders with where clauses, then elaborate implied bounds into that

63
compiler/rustc_borrowck/src/universal_regions.rs
View file

@ -467,15 +467,13 @@ impl<'cx, 'tcx> UniversalRegionsBuilder<'cx, 'tcx> {
self.infcx.tcx.local_parent(self.mir_def),
|r| {
debug!(?r);
if !indices.indices.contains_key(&r) {
let region_vid = {
let name = r.get_name_or_anon();
self.infcx.next_nll_region_var(FR, || RegionCtxt::LateBound(name))
};
let region_vid = {
let name = r.get_name_or_anon();
self.infcx.next_nll_region_var(FR, || RegionCtxt::LateBound(name))
};
debug!(?region_vid);
indices.insert_late_bound_region(r, region_vid.as_var());
}
debug!(?region_vid);
indices.insert_late_bound_region(r, region_vid.as_var());
},
);
@ -484,21 +482,17 @@ impl<'cx, 'tcx> UniversalRegionsBuilder<'cx, 'tcx> {
self.infcx.num_region_vars()
};
// "Liberate" the late-bound regions. These correspond to
// "local" free regions.
// Converse of above, if this is a function/closure then the late-bound regions declared
// on its signature are local.
//
// We manually loop over `bound_inputs_and_output` instead of using
// `for_each_late_bound_region_in_item` as we may need to add the otherwise
// implicit `ClosureEnv` region.
let bound_inputs_and_output = self.compute_inputs_and_output(&indices, defining_ty);
let inputs_and_output = self.infcx.replace_bound_regions_with_nll_infer_vars(
FR,
self.mir_def,
bound_inputs_and_output,
&mut indices,
);
// Converse of above, if this is a function/closure then the late-bound regions declared on its
// signature are local.
for_each_late_bound_region_in_item(self.infcx.tcx, self.mir_def, |r| {
debug!(?r);
if !indices.indices.contains_key(&r) {
for (idx, bound_var) in bound_inputs_and_output.bound_vars().iter().enumerate() {
if let ty::BoundVariableKind::Region(kind) = bound_var {
let kind = ty::LateParamRegionKind::from_bound(ty::BoundVar::from_usize(idx), kind);
let r = ty::Region::new_late_param(self.infcx.tcx, self.mir_def.to_def_id(), kind);
let region_vid = {
let name = r.get_name_or_anon();
self.infcx.next_nll_region_var(FR, || RegionCtxt::LateBound(name))
@ -507,7 +501,12 @@ impl<'cx, 'tcx> UniversalRegionsBuilder<'cx, 'tcx> {
debug!(?region_vid);
indices.insert_late_bound_region(r, region_vid.as_var());
}
});
}
let inputs_and_output = self.infcx.replace_bound_regions_with_nll_infer_vars(
self.mir_def,
bound_inputs_and_output,
&indices,
);
let (unnormalized_output_ty, mut unnormalized_input_tys) =
inputs_and_output.split_last().unwrap();
@ -832,10 +831,9 @@ impl<'tcx> BorrowckInferCtxt<'tcx> {
#[instrument(level = "debug", skip(self, indices))]
fn replace_bound_regions_with_nll_infer_vars<T>(
&self,
origin: NllRegionVariableOrigin,
all_outlive_scope: LocalDefId,
value: ty::Binder<'tcx, T>,
indices: &mut UniversalRegionIndices<'tcx>,
indices: &UniversalRegionIndices<'tcx>,
) -> T
where
T: TypeFoldable<TyCtxt<'tcx>>,
@ -845,18 +843,7 @@ impl<'tcx> BorrowckInferCtxt<'tcx> {
let kind = ty::LateParamRegionKind::from_bound(br.var, br.kind);
let liberated_region =
ty::Region::new_late_param(self.tcx, all_outlive_scope.to_def_id(), kind);
let region_vid = {
let name = match br.kind.get_name() {
Some(name) => name,
_ => sym::anon,
};
self.next_nll_region_var(origin, || RegionCtxt::Bound(name))
};
indices.insert_late_bound_region(liberated_region, region_vid.as_var());
debug!(?liberated_region, ?region_vid);
region_vid
ty::Region::new_var(self.tcx, indices.to_region_vid(liberated_region))
});
value
}
@ -870,7 +857,7 @@ impl<'tcx> UniversalRegionIndices<'tcx> {
/// well. These are used for error reporting.
fn insert_late_bound_region(&mut self, r: ty::Region<'tcx>, vid: ty::RegionVid) {
debug!("insert_late_bound_region({:?}, {:?})", r, vid);
self.indices.insert(r, vid);
assert_eq!(self.indices.insert(r, vid), None);
}
/// Converts `r` into a local inference variable: `r` can either

4
compiler/rustc_codegen_cranelift/patches/0029-stdlib-Disable-f16-and-f128-in-compiler-builtins.patch
View file

@ -16,8 +16,8 @@ index 7165c3e48af..968552ad435 100644
[dependencies]
core = { path = "../core" }
-compiler_builtins = { version = "=0.1.140", features = ['rustc-dep-of-std'] }
+compiler_builtins = { version = "=0.1.140", features = ['rustc-dep-of-std', 'no-f16-f128'] }
-compiler_builtins = { version = "=0.1.143", features = ['rustc-dep-of-std'] }
+compiler_builtins = { version = "=0.1.143", features = ['rustc-dep-of-std', 'no-f16-f128'] }
[dev-dependencies]
rand = { version = "0.8.5", default-features = false, features = ["alloc"] }

6
compiler/rustc_codegen_cranelift/src/abi/mod.rs
View file

@ -65,7 +65,11 @@ pub(crate) fn conv_to_call_conv(sess: &Session, c: Conv, default_call_conv: Call
sess.dcx().fatal("C-cmse-nonsecure-entry call conv is not yet implemented");
}
Conv::Msp430Intr | Conv::PtxKernel | Conv::AvrInterrupt | Conv::AvrNonBlockingInterrupt => {
Conv::Msp430Intr
| Conv::PtxKernel
| Conv::GpuKernel
| Conv::AvrInterrupt
| Conv::AvrNonBlockingInterrupt => {
unreachable!("tried to use {c:?} call conv which only exists on an unsupported target");
}
}

18
compiler/rustc_codegen_cranelift/src/codegen_i128.rs
View file

@ -76,20 +76,22 @@ pub(crate) fn maybe_codegen_mul_checked<'tcx>(
}
let is_signed = type_sign(lhs.layout().ty);
let out_ty = Ty::new_tup(fx.tcx, &[lhs.layout().ty, fx.tcx.types.bool]);
let out_place = CPlace::new_stack_slot(fx, fx.layout_of(out_ty));
let oflow_out_place = CPlace::new_stack_slot(fx, fx.layout_of(fx.tcx.types.i32));
let param_types = vec![
AbiParam::special(fx.pointer_type, ArgumentPurpose::StructReturn),
AbiParam::new(types::I128),
AbiParam::new(types::I128),
AbiParam::special(fx.pointer_type, ArgumentPurpose::Normal),
];
let args = [out_place.to_ptr().get_addr(fx), lhs.load_scalar(fx), rhs.load_scalar(fx)];
fx.lib_call(
let args = [lhs.load_scalar(fx), rhs.load_scalar(fx), oflow_out_place.to_ptr().get_addr(fx)];
let ret = fx.lib_call(
if is_signed { "__rust_i128_mulo" } else { "__rust_u128_mulo" },
param_types,
vec![],
vec![AbiParam::new(types::I128)],
&args,
);
Some(out_place.to_cvalue(fx))
let mul = ret[0];
let oflow = oflow_out_place.to_cvalue(fx).load_scalar(fx);
let oflow = clif_intcast(fx, oflow, types::I8, false);
let layout = fx.layout_of(Ty::new_tup(fx.tcx, &[lhs.layout().ty, fx.tcx.types.bool]));
Some(CValue::by_val_pair(mul, oflow, layout))
}

2
compiler/rustc_codegen_cranelift/src/compiler_builtins.rs
View file

@ -43,7 +43,7 @@ builtin_functions! {
fn __divti3(n: i128, d: i128) -> i128;
fn __umodti3(n: u128, d: u128) -> u128;
fn __modti3(n: i128, d: i128) -> i128;
fn __rust_u128_mulo(a: u128, b: u128) -> (u128, bool);
fn __rust_u128_mulo(a: u128, b: u128, oflow: &mut i32) -> u128;
// floats
fn __floattisf(i: i128) -> f32;

2
compiler/rustc_codegen_gcc/example/mini_core.rs
View file

@ -689,7 +689,7 @@ impl<T> Index<usize> for [T] {
}
}
extern {
extern "C" {
type VaListImpl;
}

4
compiler/rustc_codegen_gcc/example/mini_core_hello_world.rs
View file

@ -258,13 +258,13 @@ fn main() {
assert_eq!(((|()| 42u8) as fn(()) -> u8)(()), 42);
extern {
extern "C" {
#[linkage = "weak"]
static ABC: *const u8;
}
{
extern {
extern "C" {
#[linkage = "weak"]
static ABC: *const u8;
}

2
compiler/rustc_codegen_gcc/example/mod_bench.rs
View file

@ -3,7 +3,7 @@
#![allow(internal_features)]
#[link(name = "c")]
extern {}
extern "C" {}
#[panic_handler]
fn panic_handler(_: &core::panic::PanicInfo<'_>) -> ! {

2
compiler/rustc_codegen_gcc/example/std_example.rs
View file

@ -7,7 +7,7 @@ use std::arch::x86_64::*;
use std::io::Write;
use std::ops::Coroutine;
extern {
extern "C" {
pub fn printf(format: *const i8, ...) -> i32;
}

119
compiler/rustc_codegen_gcc/src/int.rs
View file

@ -322,36 +322,26 @@ impl<'a, 'gcc, 'tcx> Builder<'a, 'gcc, 'tcx> {
},
}
} else {
match new_kind {
Int(I128) | Uint(U128) => {
let func_name = match oop {
OverflowOp::Add => match new_kind {
Int(I128) => "__rust_i128_addo",
Uint(U128) => "__rust_u128_addo",
_ => unreachable!(),
},
OverflowOp::Sub => match new_kind {
Int(I128) => "__rust_i128_subo",
Uint(U128) => "__rust_u128_subo",
_ => unreachable!(),
},
OverflowOp::Mul => match new_kind {
Int(I128) => "__rust_i128_mulo", // TODO(antoyo): use __muloti4d instead?
Uint(U128) => "__rust_u128_mulo",
_ => unreachable!(),
},
};
return self.operation_with_overflow(func_name, lhs, rhs);
}
_ => match oop {
OverflowOp::Mul => match new_kind {
Int(I32) => "__mulosi4",
Int(I64) => "__mulodi4",
_ => unreachable!(),
},
_ => unimplemented!("overflow operation for {:?}", new_kind),
let (func_name, width) = match oop {
OverflowOp::Add => match new_kind {
Int(I128) => ("__rust_i128_addo", 128),
Uint(U128) => ("__rust_u128_addo", 128),
_ => unreachable!(),
},
}
OverflowOp::Sub => match new_kind {
Int(I128) => ("__rust_i128_subo", 128),
Uint(U128) => ("__rust_u128_subo", 128),
_ => unreachable!(),
},
OverflowOp::Mul => match new_kind {
Int(I32) => ("__mulosi4", 32),
Int(I64) => ("__mulodi4", 64),
Int(I128) => ("__rust_i128_mulo", 128), // TODO(antoyo): use __muloti4d instead?
Uint(U128) => ("__rust_u128_mulo", 128),
_ => unreachable!(),
},
};
return self.operation_with_overflow(func_name, lhs, rhs, width);
};
let intrinsic = self.context.get_builtin_function(name);
@ -364,80 +354,87 @@ impl<'a, 'gcc, 'tcx> Builder<'a, 'gcc, 'tcx> {
(res.dereference(self.location).to_rvalue(), overflow)
}
/// Non-`__builtin_*` overflow operations with a `fn(T, T, &mut i32) -> T` signature.
pub fn operation_with_overflow(
&self,
func_name: &str,
lhs: RValue<'gcc>,
rhs: RValue<'gcc>,
width: u64,
) -> (RValue<'gcc>, RValue<'gcc>) {
let a_type = lhs.get_type();
let b_type = rhs.get_type();
debug_assert!(a_type.dyncast_array().is_some());
debug_assert!(b_type.dyncast_array().is_some());
let overflow_type = self.i32_type;
let overflow_param_type = overflow_type.make_pointer();
let res_type = a_type;
let overflow_value =
self.current_func().new_local(self.location, overflow_type, "overflow");
let overflow_addr = overflow_value.get_address(self.location);
let param_a = self.context.new_parameter(self.location, a_type, "a");
let param_b = self.context.new_parameter(self.location, b_type, "b");
let result_field = self.context.new_field(self.location, a_type, "result");
let overflow_field = self.context.new_field(self.location, self.bool_type, "overflow");
let param_overflow =
self.context.new_parameter(self.location, overflow_param_type, "overflow");
let ret_ty = Ty::new_tup(self.tcx, &[self.tcx.types.i128, self.tcx.types.bool]);
let a_elem_type = a_type.dyncast_array().expect("non-array a value");
debug_assert!(a_elem_type.is_integral());
let res_ty = match width {
32 => self.tcx.types.i32,
64 => self.tcx.types.i64,
128 => self.tcx.types.i128,
_ => unreachable!("unexpected integer size"),
};
let layout = self
.tcx
.layout_of(ty::TypingEnv::fully_monomorphized().as_query_input(ret_ty))
.layout_of(ty::TypingEnv::fully_monomorphized().as_query_input(res_ty))
.unwrap();
let arg_abi = ArgAbi { layout, mode: PassMode::Direct(ArgAttributes::new()) };
let mut fn_abi = FnAbi {
args: vec![arg_abi.clone(), arg_abi.clone()].into_boxed_slice(),
args: vec![arg_abi.clone(), arg_abi.clone(), arg_abi.clone()].into_boxed_slice(),
ret: arg_abi,
c_variadic: false,
fixed_count: 2,
fixed_count: 3,
conv: Conv::C,
can_unwind: false,
};
fn_abi.adjust_for_foreign_abi(self.cx, spec::abi::Abi::C { unwind: false }).unwrap();
let indirect = matches!(fn_abi.ret.mode, PassMode::Indirect { .. });
let ret_indirect = matches!(fn_abi.ret.mode, PassMode::Indirect { .. });
let result = if ret_indirect {
let res_value = self.current_func().new_local(self.location, res_type, "result_value");
let res_addr = res_value.get_address(self.location);
let res_param_type = res_type.make_pointer();
let param_res = self.context.new_parameter(self.location, res_param_type, "result");
let return_type = self
.context
.new_struct_type(self.location, "result_overflow", &[result_field, overflow_field]);
let result = if indirect {
let return_value =
self.current_func().new_local(self.location, return_type.as_type(), "return_value");
let return_param_type = return_type.as_type().make_pointer();
let return_param =
self.context.new_parameter(self.location, return_param_type, "return_value");
let func = self.context.new_function(
self.location,
FunctionType::Extern,
self.type_void(),
&[return_param, param_a, param_b],
&[param_res, param_a, param_b, param_overflow],
func_name,
false,
);
self.llbb().add_eval(
self.location,
self.context.new_call(self.location, func, &[
return_value.get_address(self.location),
lhs,
rhs,
]),
);
return_value.to_rvalue()
let _void =
self.context.new_call(self.location, func, &[res_addr, lhs, rhs, overflow_addr]);
res_value.to_rvalue()
} else {
let func = self.context.new_function(
self.location,
FunctionType::Extern,
return_type.as_type(),
&[param_a, param_b],
res_type,
&[param_a, param_b, param_overflow],
func_name,
false,
);
self.context.new_call(self.location, func, &[lhs, rhs])
self.context.new_call(self.location, func, &[lhs, rhs, overflow_addr])
};
let overflow = result.access_field(self.location, overflow_field);
let int_result = result.access_field(self.location, result_field);
(int_result, overflow)
(result, self.context.new_cast(self.location, overflow_value, self.bool_type).to_rvalue())
}
pub fn gcc_icmp(

6
compiler/rustc_codegen_gcc/src/intrinsic/mod.rs
View file

@ -1001,7 +1001,8 @@ impl<'a, 'gcc, 'tcx> Builder<'a, 'gcc, 'tcx> {
128 => "__rust_i128_addo",
_ => unreachable!(),
};
let (int_result, overflow) = self.operation_with_overflow(func_name, lhs, rhs);
let (int_result, overflow) =
self.operation_with_overflow(func_name, lhs, rhs, width);
self.llbb().add_assignment(self.location, res, int_result);
overflow
};
@ -1071,7 +1072,8 @@ impl<'a, 'gcc, 'tcx> Builder<'a, 'gcc, 'tcx> {
128 => "__rust_i128_subo",
_ => unreachable!(),
};
let (int_result, overflow) = self.operation_with_overflow(func_name, lhs, rhs);
let (int_result, overflow) =
self.operation_with_overflow(func_name, lhs, rhs, width);
self.llbb().add_assignment(self.location, res, int_result);
overflow
};

22
compiler/rustc_codegen_llvm/src/abi.rs
View file

@ -1,3 +1,4 @@
use std::borrow::Borrow;
use std::cmp;
use libc::c_uint;
@ -312,7 +313,7 @@ impl<'ll, 'tcx> ArgAbiBuilderMethods<'tcx> for Builder<'_, 'll, 'tcx> {
pub(crate) trait FnAbiLlvmExt<'ll, 'tcx> {
fn llvm_type(&self, cx: &CodegenCx<'ll, 'tcx>) -> &'ll Type;
fn ptr_to_llvm_type(&self, cx: &CodegenCx<'ll, 'tcx>) -> &'ll Type;
fn llvm_cconv(&self) -> llvm::CallConv;
fn llvm_cconv(&self, cx: &CodegenCx<'ll, 'tcx>) -> llvm::CallConv;
/// Apply attributes to a function declaration/definition.
fn apply_attrs_llfn(
@ -404,8 +405,8 @@ impl<'ll, 'tcx> FnAbiLlvmExt<'ll, 'tcx> for FnAbi<'tcx, Ty<'tcx>> {
cx.type_ptr_ext(cx.data_layout().instruction_address_space)
}
fn llvm_cconv(&self) -> llvm::CallConv {
self.conv.into()
fn llvm_cconv(&self, cx: &CodegenCx<'ll, 'tcx>) -> llvm::CallConv {
llvm::CallConv::from_conv(self.conv, cx.tcx.sess.target.arch.borrow())
}
fn apply_attrs_llfn(
@ -617,7 +618,7 @@ impl<'ll, 'tcx> FnAbiLlvmExt<'ll, 'tcx> for FnAbi<'tcx, Ty<'tcx>> {
}
}
let cconv = self.llvm_cconv();
let cconv = self.llvm_cconv(&bx.cx);
if cconv != llvm::CCallConv {
llvm::SetInstructionCallConv(callsite, cconv);
}
@ -655,8 +656,8 @@ impl<'tcx> AbiBuilderMethods<'tcx> for Builder<'_, '_, 'tcx> {
}
}
impl From<Conv> for llvm::CallConv {
fn from(conv: Conv) -> Self {
impl llvm::CallConv {
pub fn from_conv(conv: Conv, arch: &str) -> Self {
match conv {
Conv::C
| Conv::Rust
@ -666,6 +667,15 @@ impl From<Conv> for llvm::CallConv {
Conv::Cold => llvm::ColdCallConv,
Conv::PreserveMost => llvm::PreserveMost,
Conv::PreserveAll => llvm::PreserveAll,
Conv::GpuKernel => {
if arch == "amdgpu" {
llvm::AmdgpuKernel
} else if arch == "nvptx64" {
llvm::PtxKernel
} else {
panic!("Architecture {arch} does not support GpuKernel calling convention");
}
}
Conv::AvrInterrupt => llvm::AvrInterrupt,
Conv::AvrNonBlockingInterrupt => llvm::AvrNonBlockingInterrupt,
Conv::ArmAapcs => llvm::ArmAapcsCallConv,

5
compiler/rustc_codegen_llvm/src/context.rs
View file

@ -741,7 +741,10 @@ impl<'ll, 'tcx> MiscCodegenMethods<'tcx> for CodegenCx<'ll, 'tcx> {
if self.get_declared_value(entry_name).is_none() {
Some(self.declare_entry_fn(
entry_name,
self.sess().target.entry_abi.into(),
llvm::CallConv::from_conv(
self.sess().target.entry_abi,
self.sess().target.arch.borrow(),
),
llvm::UnnamedAddr::Global,
fn_type,
))

2
compiler/rustc_codegen_llvm/src/declare.rs
View file

@ -125,7 +125,7 @@ impl<'ll, 'tcx> CodegenCx<'ll, 'tcx> {
let llfn = declare_raw_fn(
self,
name,
fn_abi.llvm_cconv(),
fn_abi.llvm_cconv(self),
llvm::UnnamedAddr::Global,
llvm::Visibility::Default,
fn_abi.llvm_type(self),

1
compiler/rustc_codegen_llvm/src/llvm/ffi.rs
View file

@ -120,6 +120,7 @@ pub enum CallConv {
X86_Intr = 83,
AvrNonBlockingInterrupt = 84,
AvrInterrupt = 85,
AmdgpuKernel = 91,
}
/// Must match the layout of `LLVMLinkage`.

12
compiler/rustc_codegen_ssa/src/codegen_attrs.rs
View file

@ -250,10 +250,14 @@ fn codegen_fn_attrs(tcx: TyCtxt<'_>, did: LocalDefId) -> CodegenFnAttrs {
}
}
sym::target_feature => {
if !tcx.is_closure_like(did.to_def_id())
&& let Some(fn_sig) = fn_sig()
&& fn_sig.skip_binder().safety().is_safe()
{
let Some(sig) = tcx.hir_node_by_def_id(did).fn_sig() else {
tcx.dcx().span_delayed_bug(attr.span, "target_feature applied to non-fn");
continue;
};
let safe_target_features =
matches!(sig.header.safety, hir::HeaderSafety::SafeTargetFeatures);
codegen_fn_attrs.safe_target_features = safe_target_features;
if safe_target_features {
if tcx.sess.target.is_like_wasm || tcx.sess.opts.actually_rustdoc {
// The `#[target_feature]` attribute is allowed on
// WebAssembly targets on all functions, including safe

26
compiler/rustc_const_eval/src/check_consts/check.rs
View file

@ -35,6 +35,12 @@ use crate::errors;
type QualifResults<'mir, 'tcx, Q> =
rustc_mir_dataflow::ResultsCursor<'mir, 'tcx, FlowSensitiveAnalysis<'mir, 'mir, 'tcx, Q>>;
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
enum ConstConditionsHold {
Yes,
No,
}
#[derive(Default)]
pub(crate) struct Qualifs<'mir, 'tcx> {
has_mut_interior: Option<QualifResults<'mir, 'tcx, HasMutInterior>>,
@ -376,15 +382,15 @@ impl<'mir, 'tcx> Checker<'mir, 'tcx> {
callee: DefId,
callee_args: ty::GenericArgsRef<'tcx>,
call_span: Span,
) -> bool {
) -> Option<ConstConditionsHold> {
let tcx = self.tcx;
if !tcx.is_conditionally_const(callee) {
return false;
return None;
}
let const_conditions = tcx.const_conditions(callee).instantiate(tcx, callee_args);
if const_conditions.is_empty() {
return false;
return None;
}
let (infcx, param_env) = tcx.infer_ctxt().build_with_typing_env(self.body.typing_env(tcx));
@ -413,12 +419,13 @@ impl<'mir, 'tcx> Checker<'mir, 'tcx> {
}));
let errors = ocx.select_all_or_error();
if !errors.is_empty() {
if errors.is_empty() {
Some(ConstConditionsHold::Yes)
} else {
tcx.dcx()
.span_delayed_bug(call_span, "this should have reported a ~const error in HIR");
Some(ConstConditionsHold::No)
}
true
}
pub fn check_drop_terminator(
@ -706,7 +713,10 @@ impl<'tcx> Visitor<'tcx> for Checker<'_, 'tcx> {
trace!("attempting to call a trait method");
let trait_is_const = tcx.is_const_trait(trait_did);
if trait_is_const {
// Only consider a trait to be const if the const conditions hold.
// Otherwise, it's really misleading to call something "conditionally"
// const when it's very obviously not conditionally const.
if trait_is_const && has_const_conditions == Some(ConstConditionsHold::Yes) {
// Trait calls are always conditionally-const.
self.check_op(ops::ConditionallyConstCall {
callee,
@ -730,7 +740,7 @@ impl<'tcx> Visitor<'tcx> for Checker<'_, 'tcx> {
}
// Even if we know the callee, ensure we can use conditionally-const calls.
if has_const_conditions {
if has_const_conditions.is_some() {
self.check_op(ops::ConditionallyConstCall {
callee,
args: fn_args,

20
compiler/rustc_const_eval/src/const_eval/fn_queries.rs
View file

@ -4,14 +4,18 @@ use rustc_hir::def_id::{DefId, LocalDefId};
use rustc_middle::query::Providers;
use rustc_middle::ty::TyCtxt;
fn parent_impl_constness(tcx: TyCtxt<'_>, def_id: LocalDefId) -> hir::Constness {
fn parent_impl_or_trait_constness(tcx: TyCtxt<'_>, def_id: LocalDefId) -> hir::Constness {
let parent_id = tcx.local_parent(def_id);
if matches!(tcx.def_kind(parent_id), DefKind::Impl { .. })
&& let Some(header) = tcx.impl_trait_header(parent_id)
{
header.constness
} else {
hir::Constness::NotConst
match tcx.def_kind(parent_id) {
DefKind::Impl { of_trait: true } => tcx.impl_trait_header(parent_id).unwrap().constness,
DefKind::Trait => {
if tcx.is_const_trait(parent_id.into()) {
hir::Constness::Const
} else {
hir::Constness::NotConst
}
}
_ => hir::Constness::NotConst,
}
}
@ -34,7 +38,7 @@ fn constness(tcx: TyCtxt<'_>, def_id: LocalDefId) -> hir::Constness {
// If the function itself is not annotated with `const`, it may still be a `const fn`
// if it resides in a const trait impl.
parent_impl_constness(tcx, def_id)
parent_impl_or_trait_constness(tcx, def_id)
} else {
tcx.dcx().span_bug(
tcx.def_span(def_id),

5
compiler/rustc_const_eval/src/const_eval/machine.rs
View file

@ -360,10 +360,7 @@ impl<'tcx> interpret::Machine<'tcx> for CompileTimeMachine<'tcx> {
// sensitive check here. But we can at least rule out functions that are not const at
// all. That said, we have to allow calling functions inside a trait marked with
// #[const_trait]. These *are* const-checked!
// FIXME(const_trait_impl): why does `is_const_fn` not classify them as const?
if (!ecx.tcx.is_const_fn(def) && !ecx.tcx.is_const_default_method(def))
|| ecx.tcx.has_attr(def, sym::rustc_do_not_const_check)
{
if !ecx.tcx.is_const_fn(def) || ecx.tcx.has_attr(def, sym::rustc_do_not_const_check) {
// We certainly do *not* want to actually call the fn
// though, so be sure we return here.
throw_unsup_format!("calling non-const function `{}`", instance)

25
compiler/rustc_const_eval/src/interpret/memory.rs
View file

@ -1481,22 +1481,31 @@ impl<'tcx, M: Machine<'tcx>> InterpCx<'tcx, M> {
/// Test if this value might be null.
/// If the machine does not support ptr-to-int casts, this is conservative.
pub fn scalar_may_be_null(&self, scalar: Scalar<M::Provenance>) -> InterpResult<'tcx, bool> {
interp_ok(match scalar.try_to_scalar_int() {
Ok(int) => int.is_null(),
match scalar.try_to_scalar_int() {
Ok(int) => interp_ok(int.is_null()),
Err(_) => {
// Can only happen during CTFE.
// We can't cast this pointer to an integer. Can only happen during CTFE.
let ptr = scalar.to_pointer(self)?;
match self.ptr_try_get_alloc_id(ptr, 0) {
Ok((alloc_id, offset, _)) => {
let size = self.get_alloc_info(alloc_id).size;
// If the pointer is out-of-bounds, it may be null.
// Note that one-past-the-end (offset == size) is still inbounds, and never null.
offset > size
let info = self.get_alloc_info(alloc_id);
// If the pointer is in-bounds (including "at the end"), it is definitely not null.
if offset <= info.size {
return interp_ok(false);
}
// If the allocation is N-aligned, and the offset is not divisible by N,
// then `base + offset` has a non-zero remainder after division by `N`,
// which means `base + offset` cannot be null.
if offset.bytes() % info.align.bytes() != 0 {
return interp_ok(false);
}
// We don't know enough, this might be null.
interp_ok(true)
}
Err(_offset) => bug!("a non-int scalar is always a pointer"),
}
}
})
}
}
/// Turning a "maybe pointer" into a proper pointer (and some information

25
compiler/rustc_data_structures/src/flock.rs
View file

@ -4,6 +4,7 @@
//! green/native threading. This is just a bare-bones enough solution for
//! librustdoc, it is not production quality at all.
#[cfg(bootstrap)]
cfg_match! {
cfg(target_os = "linux") => {
mod linux;
@ -27,4 +28,28 @@ cfg_match! {
}
}
#[cfg(not(bootstrap))]
cfg_match! {
target_os = "linux" => {
mod linux;
use linux as imp;
}
target_os = "redox" => {
mod linux;
use linux as imp;
}
unix => {
mod unix;
use unix as imp;
}
windows => {
mod windows;
use self::windows as imp;
}
_ => {
mod unsupported;
use unsupported as imp;
}
}
pub use imp::Lock;

10
compiler/rustc_data_structures/src/graph/iterate/mod.rs
View file

@ -125,6 +125,16 @@ where
pub fn visited(&self, node: G::Node) -> bool {
self.visited.contains(node)
}
/// Returns a reference to the set of nodes that have been visited, with
/// the same caveats as [`Self::visited`].
///
/// When incorporating the visited nodes into another bitset, using bulk
/// operations like `union` or `intersect` can be more efficient than
/// processing each node individually.
pub fn visited_set(&self) -> &DenseBitSet<G::Node> {
&self.visited
}
}
impl<G> std::fmt::Debug for DepthFirstSearch<G>

1
compiler/rustc_data_structures/src/graph/mod.rs
View file

@ -4,6 +4,7 @@ pub mod dominators;
pub mod implementation;
pub mod iterate;
mod reference;
pub mod reversed;
pub mod scc;
pub mod vec_graph;

42
compiler/rustc_data_structures/src/graph/reversed.rs Normal file
View file

@ -0,0 +1,42 @@
use crate::graph::{DirectedGraph, Predecessors, Successors};
/// View that reverses the direction of edges in its underlying graph, so that
/// successors become predecessors and vice-versa.
///
/// Because of `impl<G: Graph> Graph for &G`, the underlying graph can be
/// wrapped by-reference instead of by-value if desired.
#[derive(Clone, Copy, Debug)]
pub struct ReversedGraph<G> {
pub inner: G,
}
impl<G> ReversedGraph<G> {
pub fn new(inner: G) -> Self {
Self { inner }
}
}
impl<G: DirectedGraph> DirectedGraph for ReversedGraph<G> {
type Node = G::Node;
fn num_nodes(&self) -> usize {
self.inner.num_nodes()
}
}
// Implementing `StartNode` is not possible in general, because the start node
// of an underlying graph is instead an _end_ node in the reversed graph.
// But would be possible to define another wrapper type that adds an explicit
// start node to its underlying graph, if desired.
impl<G: Predecessors> Successors for ReversedGraph<G> {
fn successors(&self, node: Self::Node) -> impl Iterator<Item = Self::Node> {
self.inner.predecessors(node)
}
}
impl<G: Successors> Predecessors for ReversedGraph<G> {
fn predecessors(&self, node: Self::Node) -> impl Iterator<Item = Self::Node> {
self.inner.successors(node)
}
}

63
compiler/rustc_data_structures/src/profiling.rs
View file

@ -860,6 +860,7 @@ fn get_thread_id() -> u32 {
}
// Memory reporting
#[cfg(bootstrap)]
cfg_match! {
cfg(windows) => {
pub fn get_resident_set_size() -> Option<usize> {
@ -921,5 +922,67 @@ cfg_match! {
}
}
#[cfg(not(bootstrap))]
cfg_match! {
windows => {
pub fn get_resident_set_size() -> Option<usize> {
use std::mem;
use windows::{
Win32::System::ProcessStatus::{K32GetProcessMemoryInfo, PROCESS_MEMORY_COUNTERS},
Win32::System::Threading::GetCurrentProcess,
};
let mut pmc = PROCESS_MEMORY_COUNTERS::default();
let pmc_size = mem::size_of_val(&pmc);
unsafe {
K32GetProcessMemoryInfo(
GetCurrentProcess(),
&mut pmc,
pmc_size as u32,
)
}
.ok()
.ok()?;
Some(pmc.WorkingSetSize)
}
}
target_os = "macos" => {
pub fn get_resident_set_size() -> Option<usize> {
use libc::{c_int, c_void, getpid, proc_pidinfo, proc_taskinfo, PROC_PIDTASKINFO};
use std::mem;
const PROC_TASKINFO_SIZE: c_int = mem::size_of::<proc_taskinfo>() as c_int;
unsafe {
let mut info: proc_taskinfo = mem::zeroed();
let info_ptr = &mut info as *mut proc_taskinfo as *mut c_void;
let pid = getpid() as c_int;
let ret = proc_pidinfo(pid, PROC_PIDTASKINFO, 0, info_ptr, PROC_TASKINFO_SIZE);
if ret == PROC_TASKINFO_SIZE {
Some(info.pti_resident_size as usize)
} else {
None
}
}
}
}
unix => {
pub fn get_resident_set_size() -> Option<usize> {
let field = 1;
let contents = fs::read("/proc/self/statm").ok()?;
let contents = String::from_utf8(contents).ok()?;
let s = contents.split_whitespace().nth(field)?;
let npages = s.parse::<usize>().ok()?;
Some(npages * 4096)
}
}
_ => {
pub fn get_resident_set_size() -> Option<usize> {
None
}
}
}
#[cfg(test)]
mod tests;

24
compiler/rustc_error_codes/src/error_codes/E0207.md
View file

@ -195,6 +195,30 @@ impl<'a> Contains for Foo {
Please note that unconstrained lifetime parameters are not supported if they are
being used by an associated type.
In cases where the associated type's lifetime is meant to be tied to the the
self type, and none of the methods on the trait need ownership or different
mutability, then an option is to implement the trait on a borrowed type:
```rust
struct Foo(i32);
trait Contents {
type Item;
fn get(&self) -> Self::Item;
}
// Note the lifetime `'a` is used both for the self type...
impl<'a> Contents for &'a Foo {
// ...and the associated type.
type Item = &'a i32;
fn get(&self) -> Self::Item {
&self.0
}
}
```
### Additional information
For more information, please see [RFC 447].

2
compiler/rustc_feature/src/builtin_attrs.rs
View file

@ -623,7 +623,7 @@ pub const BUILTIN_ATTRIBUTES: &[BuiltinAttribute] = &[
EncodeCrossCrate::No, "allow_internal_unsafe side-steps the unsafe_code lint",
),
rustc_attr!(
rustc_allowed_through_unstable_modules, Normal, template!(Word),
rustc_allowed_through_unstable_modules, Normal, template!(Word, NameValueStr: "deprecation message"),
WarnFollowing, EncodeCrossCrate::No,
"rustc_allowed_through_unstable_modules special cases accidental stabilizations of stable items \
through unstable paths"

2
compiler/rustc_feature/src/unstable.rs
View file

@ -361,6 +361,8 @@ declare_features! (
(unstable, abi_avr_interrupt, "1.45.0", Some(69664)),
/// Allows `extern "C-cmse-nonsecure-call" fn()`.
(unstable, abi_c_cmse_nonsecure_call, "1.51.0", Some(81391)),
/// Allows `extern "gpu-kernel" fn()`.
(unstable, abi_gpu_kernel, "CURRENT_RUSTC_VERSION", Some(135467)),
/// Allows `extern "msp430-interrupt" fn()`.
(unstable, abi_msp430_interrupt, "1.16.0", Some(38487)),
/// Allows `extern "ptx-*" fn()`.

58
compiler/rustc_hir/src/hir.rs
View file

@ -34,6 +34,7 @@ use crate::intravisit::FnKind;
#[derive(Debug, Copy, Clone, HashStable_Generic)]
pub struct Lifetime {
#[stable_hasher(ignore)]
pub hir_id: HirId,
/// Either "`'a`", referring to a named lifetime definition,
@ -214,6 +215,7 @@ impl Path<'_> {
pub struct PathSegment<'hir> {
/// The identifier portion of this path segment.
pub ident: Ident,
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub res: Res,
@ -304,6 +306,7 @@ pub enum ConstArgKind<'hir> {
#[derive(Clone, Copy, Debug, HashStable_Generic)]
pub struct InferArg {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub span: Span,
}
@ -592,6 +595,7 @@ pub enum GenericParamKind<'hir> {
#[derive(Debug, Clone, Copy, HashStable_Generic)]
pub struct GenericParam<'hir> {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub def_id: LocalDefId,
pub name: ParamName,
@ -850,6 +854,7 @@ impl<'hir> Generics<'hir> {
/// A single predicate in a where-clause.
#[derive(Debug, Clone, Copy, HashStable_Generic)]
pub struct WherePredicate<'hir> {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub span: Span,
pub kind: &'hir WherePredicateKind<'hir>,
@ -1521,6 +1526,7 @@ impl fmt::Debug for DotDotPos {
#[derive(Debug, Clone, Copy, HashStable_Generic)]
pub struct PatExpr<'hir> {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub span: Span,
pub kind: PatExprKind<'hir>,
@ -1610,6 +1616,7 @@ pub enum PatKind<'hir> {
/// A statement.
#[derive(Debug, Clone, Copy, HashStable_Generic)]
pub struct Stmt<'hir> {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub kind: StmtKind<'hir>,
pub span: Span,
@ -1641,6 +1648,7 @@ pub struct LetStmt<'hir> {
pub init: Option<&'hir Expr<'hir>>,
/// Else block for a `let...else` binding.
pub els: Option<&'hir Block<'hir>>,
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub span: Span,
/// Can be `ForLoopDesugar` if the `let` statement is part of a `for` loop
@ -1937,6 +1945,7 @@ pub type Lit = Spanned<LitKind>;
/// `const N: usize = { ... }` with `tcx.hir().opt_const_param_default_param_def_id(..)`
#[derive(Copy, Clone, Debug, HashStable_Generic)]
pub struct AnonConst {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub def_id: LocalDefId,
pub body: BodyId,
@ -1946,6 +1955,7 @@ pub struct AnonConst {
/// An inline constant expression `const { something }`.
#[derive(Copy, Clone, Debug, HashStable_Generic)]
pub struct ConstBlock {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub def_id: LocalDefId,
pub body: BodyId,
@ -1961,6 +1971,7 @@ pub struct ConstBlock {
/// [rust lang reference]: https://doc.rust-lang.org/reference/expressions.html
#[derive(Debug, Clone, Copy, HashStable_Generic)]
pub struct Expr<'hir> {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub kind: ExprKind<'hir>,
pub span: Span,
@ -2839,6 +2850,7 @@ pub enum ImplItemKind<'hir> {
/// * the `f(..): Bound` in `Trait<f(..): Bound>` (feature `return_type_notation`)
#[derive(Debug, Clone, Copy, HashStable_Generic)]
pub struct AssocItemConstraint<'hir> {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub ident: Ident,
pub gen_args: &'hir GenericArgs<'hir>,
@ -2907,6 +2919,7 @@ impl<'hir> AssocItemConstraintKind<'hir> {
#[derive(Debug, Clone, Copy, HashStable_Generic)]
pub struct Ty<'hir> {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub kind: TyKind<'hir>,
pub span: Span,
@ -3102,6 +3115,7 @@ pub struct UnsafeBinderTy<'hir> {
#[derive(Debug, Clone, Copy, HashStable_Generic)]
pub struct OpaqueTy<'hir> {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub def_id: LocalDefId,
pub bounds: GenericBounds<'hir>,
@ -3138,6 +3152,7 @@ impl PreciseCapturingArg<'_> {
/// since resolve_bound_vars operates on `Lifetime`s.
#[derive(Debug, Clone, Copy, HashStable_Generic)]
pub struct PreciseCapturingNonLifetimeArg {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub ident: Ident,
pub res: Res,
@ -3311,6 +3326,7 @@ impl InlineAsm<'_> {
/// Represents a parameter in a function header.
#[derive(Debug, Clone, Copy, HashStable_Generic)]
pub struct Param<'hir> {
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub pat: &'hir Pat<'hir>,
pub ty_span: Span,
@ -3468,6 +3484,7 @@ pub struct Variant<'hir> {
/// Name of the variant.
pub ident: Ident,
/// Id of the variant (not the constructor, see `VariantData::ctor_hir_id()`).
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub def_id: LocalDefId,
/// Fields and constructor id of the variant.
@ -3540,6 +3557,7 @@ pub struct FieldDef<'hir> {
pub span: Span,
pub vis_span: Span,
pub ident: Ident,
#[stable_hasher(ignore)]
pub hir_id: HirId,
pub def_id: LocalDefId,
pub ty: &'hir Ty<'hir>,
@ -3564,11 +3582,11 @@ pub enum VariantData<'hir> {
/// A tuple variant.
///
/// E.g., `Bar(..)` as in `enum Foo { Bar(..) }`.
Tuple(&'hir [FieldDef<'hir>], HirId, LocalDefId),
Tuple(&'hir [FieldDef<'hir>], #[stable_hasher(ignore)] HirId, LocalDefId),
/// A unit variant.
///
/// E.g., `Bar = ..` as in `enum Foo { Bar = .. }`.
Unit(HirId, LocalDefId),
Unit(#[stable_hasher(ignore)] HirId, LocalDefId),
}
impl<'hir> VariantData<'hir> {
@ -3762,9 +3780,30 @@ impl fmt::Display for Constness {
}
}
/// The actualy safety specified in syntax. We may treat
/// its safety different within the type system to create a
/// "sound by default" system that needs checking this enum
/// explicitly to allow unsafe operations.
#[derive(Copy, Clone, Debug, HashStable_Generic, PartialEq, Eq)]
pub enum HeaderSafety {
/// A safe function annotated with `#[target_features]`.
/// The type system treats this function as an unsafe function,
/// but safety checking will check this enum to treat it as safe
/// and allowing calling other safe target feature functions with
/// the same features without requiring an additional unsafe block.
SafeTargetFeatures,
Normal(Safety),
}
impl From<Safety> for HeaderSafety {
fn from(v: Safety) -> Self {
Self::Normal(v)
}
}
#[derive(Copy, Clone, Debug, HashStable_Generic)]
pub struct FnHeader {
pub safety: Safety,
pub safety: HeaderSafety,
pub constness: Constness,
pub asyncness: IsAsync,
pub abi: ExternAbi,
@ -3780,7 +3819,18 @@ impl FnHeader {
}
pub fn is_unsafe(&self) -> bool {
self.safety.is_unsafe()
self.safety().is_unsafe()
}
pub fn is_safe(&self) -> bool {
self.safety().is_safe()
}
pub fn safety(&self) -> Safety {
match self.safety {
HeaderSafety::SafeTargetFeatures => Safety::Unsafe,
HeaderSafety::Normal(safety) => safety,
}
}
}

2
compiler/rustc_hir_analysis/messages.ftl
View file

@ -135,7 +135,7 @@ hir_analysis_dispatch_from_dyn_multi = implementing the `DispatchFromDyn` trait
hir_analysis_dispatch_from_dyn_repr = structs implementing `DispatchFromDyn` may not have `#[repr(packed)]` or `#[repr(C)]`
hir_analysis_dispatch_from_dyn_zst = the trait `DispatchFromDyn` may only be implemented for structs containing the field being coerced, ZST fields with 1 byte alignment, and nothing else
hir_analysis_dispatch_from_dyn_zst = the trait `DispatchFromDyn` may only be implemented for structs containing the field being coerced, ZST fields with 1 byte alignment that don't mention type/const generics, and nothing else
.note = extra field `{$name}` of type `{$ty}` is not allowed
hir_analysis_drop_impl_negative = negative `Drop` impls are not supported

1
compiler/rustc_hir_analysis/src/check/check.rs
View file

@ -1637,7 +1637,6 @@ fn check_type_alias_type_params_are_used<'tcx>(tcx: TyCtxt<'tcx>, def_id: LocalD
let ty = tcx.type_of(def_id).instantiate_identity();
if ty.references_error() {
// If there is already another error, do not emit an error for not using a type parameter.
assert!(tcx.dcx().has_errors().is_some());
return;
}

42
compiler/rustc_hir_analysis/src/coherence/builtin.rs
View file

@ -259,19 +259,37 @@ fn visit_implementation_of_dispatch_from_dyn(checker: &Checker<'_>) -> Result<()
let coerced_fields = fields
.iter()
.filter(|field| {
// Ignore PhantomData fields
let unnormalized_ty = tcx.type_of(field.did).instantiate_identity();
if tcx
.try_normalize_erasing_regions(
ty::TypingEnv::non_body_analysis(tcx, def_a.did()),
unnormalized_ty,
)
.unwrap_or(unnormalized_ty)
.is_phantom_data()
{
return false;
}
let ty_a = field.ty(tcx, args_a);
let ty_b = field.ty(tcx, args_b);
if let Ok(layout) =
tcx.layout_of(infcx.typing_env(param_env).as_query_input(ty_a))
{
if layout.is_1zst() {
// FIXME: We could do normalization here, but is it really worth it?
if ty_a == ty_b {
// Allow 1-ZSTs that don't mention type params.
//
// Allowing type params here would allow us to possibly transmute
// between ZSTs, which may be used to create library unsoundness.
if let Ok(layout) =
tcx.layout_of(infcx.typing_env(param_env).as_query_input(ty_a))
&& layout.is_1zst()
&& !ty_a.has_non_region_param()
{
// ignore 1-ZST fields
return false;
}
}
if ty_a == ty_b {
res = Err(tcx.dcx().emit_err(errors::DispatchFromDynZST {
span,
name: field.name,
@ -460,8 +478,16 @@ pub(crate) fn coerce_unsized_info<'tcx>(
.filter_map(|(i, f)| {
let (a, b) = (f.ty(tcx, args_a), f.ty(tcx, args_b));
if tcx.type_of(f.did).instantiate_identity().is_phantom_data() {
// Ignore PhantomData fields
// Ignore PhantomData fields
let unnormalized_ty = tcx.type_of(f.did).instantiate_identity();
if tcx
.try_normalize_erasing_regions(
ty::TypingEnv::non_body_analysis(tcx, def_a.did()),
unnormalized_ty,
)
.unwrap_or(unnormalized_ty)
.is_phantom_data()
{
return None;
}

17
compiler/rustc_hir_analysis/src/collect.rs
View file

@ -1336,7 +1336,7 @@ fn fn_sig(tcx: TyCtxt<'_>, def_id: LocalDefId) -> ty::EarlyBinder<'_, ty::PolyFn
{
icx.lowerer().lower_fn_ty(
hir_id,
sig.header.safety,
sig.header.safety(),
sig.header.abi,
sig.decl,
Some(generics),
@ -1351,13 +1351,18 @@ fn fn_sig(tcx: TyCtxt<'_>, def_id: LocalDefId) -> ty::EarlyBinder<'_, ty::PolyFn
kind: TraitItemKind::Fn(FnSig { header, decl, span: _ }, _),
generics,
..
}) => {
icx.lowerer().lower_fn_ty(hir_id, header.safety, header.abi, decl, Some(generics), None)
}
}) => icx.lowerer().lower_fn_ty(
hir_id,
header.safety(),
header.abi,
decl,
Some(generics),
None,
),
ForeignItem(&hir::ForeignItem { kind: ForeignItemKind::Fn(sig, _, _), .. }) => {
let abi = tcx.hir().get_foreign_abi(hir_id);
compute_sig_of_foreign_fn_decl(tcx, def_id, sig.decl, abi, sig.header.safety)
compute_sig_of_foreign_fn_decl(tcx, def_id, sig.decl, abi, sig.header.safety())
}
Ctor(data) | Variant(hir::Variant { data, .. }) if data.ctor().is_some() => {
@ -1405,7 +1410,7 @@ fn lower_fn_sig_recovering_infer_ret_ty<'tcx>(
icx.lowerer().lower_fn_ty(
icx.tcx().local_def_id_to_hir_id(def_id),
sig.header.safety,
sig.header.safety(),
sig.header.abi,
sig.decl,
Some(generics),

2
compiler/rustc_hir_analysis/src/collect/predicates_of.rs
View file

@ -653,7 +653,7 @@ pub(super) fn implied_predicates_with_filter<'tcx>(
}
}
}
PredicateFilter::SelfAndAssociatedTypeBounds => {
PredicateFilter::All | PredicateFilter::SelfAndAssociatedTypeBounds => {
for &(pred, span) in implied_bounds {
debug!("superbound: {:?}", pred);
if let ty::ClauseKind::Trait(bound) = pred.kind().skip_binder()

12
compiler/rustc_hir_pretty/src/lib.rs
View file

@ -2407,7 +2407,7 @@ impl<'a> State<'a> {
self.print_fn(
decl,
hir::FnHeader {
safety,
safety: safety.into(),
abi,
constness: hir::Constness::NotConst,
asyncness: hir::IsAsync::NotAsync,
@ -2423,12 +2423,20 @@ impl<'a> State<'a> {
fn print_fn_header_info(&mut self, header: hir::FnHeader) {
self.print_constness(header.constness);
let safety = match header.safety {
hir::HeaderSafety::SafeTargetFeatures => {
self.word_nbsp("#[target_feature]");
hir::Safety::Safe
}
hir::HeaderSafety::Normal(safety) => safety,
};
match header.asyncness {
hir::IsAsync::NotAsync => {}
hir::IsAsync::Async(_) => self.word_nbsp("async"),
}
self.print_safety(header.safety);
self.print_safety(safety);
if header.abi != ExternAbi::Rust {
self.word_nbsp("extern");

18
compiler/rustc_hir_typeck/src/coercion.rs
View file

@ -920,7 +920,7 @@ impl<'f, 'tcx> Coerce<'f, 'tcx> {
match b.kind() {
ty::FnPtr(_, b_hdr) => {
let a_sig = a.fn_sig(self.tcx);
let mut a_sig = a.fn_sig(self.tcx);
if let ty::FnDef(def_id, _) = *a.kind() {
// Intrinsics are not coercible to function pointers
if self.tcx.intrinsic(def_id).is_some() {
@ -932,12 +932,20 @@ impl<'f, 'tcx> Coerce<'f, 'tcx> {
return Err(TypeError::ForceInlineCast);
}
// Safe `#[target_feature]` functions are not assignable to safe fn pointers
// (RFC 2396).
if b_hdr.safety.is_safe()
&& !self.tcx.codegen_fn_attrs(def_id).target_features.is_empty()
&& self.tcx.codegen_fn_attrs(def_id).safe_target_features
{
return Err(TypeError::TargetFeatureCast(def_id));
// Allow the coercion if the current function has all the features that would be
// needed to call the coercee safely.
if let Some(safe_sig) = self.tcx.adjust_target_feature_sig(
def_id,
a_sig,
self.fcx.body_id.into(),
) {
a_sig = safe_sig;
} else {
return Err(TypeError::TargetFeatureCast(def_id));
}
}
}

1
compiler/rustc_hir_typeck/src/fn_ctxt/suggestions.rs
View file

@ -2682,6 +2682,7 @@ impl<'a, 'tcx> FnCtxt<'a, 'tcx> {
if let hir::ExprKind::Unary(hir::UnOp::Deref, inner) = expr.kind
&& let Some(1) = self.deref_steps_for_suggestion(expected, checked_ty)
&& self.typeck_results.borrow().expr_ty(inner).is_ref()
{
// We have `*&T`, check if what was expected was `&T`.
// If so, we may want to suggest removing a `*`.

2
compiler/rustc_hir_typeck/src/lib.rs
View file

@ -139,7 +139,7 @@ fn typeck_with_fallback<'tcx>(
// type that has an infer in it, lower the type directly so that it'll
// be correctly filled with infer. We'll use this inference to provide
// a suggestion later on.
fcx.lowerer().lower_fn_ty(id, header.safety, header.abi, decl, None, None)
fcx.lowerer().lower_fn_ty(id, header.safety(), header.abi, decl, None, None)
} else {
tcx.fn_sig(def_id).instantiate_identity()
};

30
compiler/rustc_index/src/bit_set.rs
View file

@ -281,6 +281,24 @@ impl<T: Idx> DenseBitSet<T> {
}
bit_relations_inherent_impls! {}
/// Sets `self = self | !other`.
///
/// FIXME: Incorporate this into [`BitRelations`] and fill out
/// implementations for other bitset types, if needed.
pub fn union_not(&mut self, other: &DenseBitSet<T>) {
assert_eq!(self.domain_size, other.domain_size);
// FIXME(Zalathar): If we were to forcibly _set_ all excess bits before
// the bitwise update, and then clear them again afterwards, we could
// quickly and accurately detect whether the update changed anything.
// But that's only worth doing if there's an actual use-case.
bitwise(&mut self.words, &other.words, |a, b| a | !b);
// The bitwise update `a | !b` can result in the last word containing
// out-of-domain bits, so we need to clear them.
self.clear_excess_bits();
}
}
// dense REL dense
@ -1087,6 +1105,18 @@ impl<T: Idx> fmt::Debug for ChunkedBitSet<T> {
}
}
/// Sets `out_vec[i] = op(out_vec[i], in_vec[i])` for each index `i` in both
/// slices. The slices must have the same length.
///
/// Returns true if at least one bit in `out_vec` was changed.
///
/// ## Warning
/// Some bitwise operations (e.g. union-not, xor) can set output bits that were
/// unset in in both inputs. If this happens in the last word/chunk of a bitset,
/// it can cause the bitset to contain out-of-domain values, which need to
/// be cleared with `clear_excess_bits_in_final_word`. This also makes the
/// "changed" return value unreliable, because the change might have only
/// affected excess bits.
#[inline]
fn bitwise<Op>(out_vec: &mut [Word], in_vec: &[Word], op: Op) -> bool
where

26
compiler/rustc_index/src/bit_set/tests.rs
View file

@ -75,6 +75,32 @@ fn union_two_sets() {
assert!(set1.contains(64));
}
#[test]
fn union_not() {
let mut a = DenseBitSet::<usize>::new_empty(100);
let mut b = DenseBitSet::<usize>::new_empty(100);
a.insert(3);
a.insert(5);
a.insert(80);
a.insert(81);
b.insert(5); // Already in `a`.
b.insert(7);
b.insert(63);
b.insert(81); // Already in `a`.
b.insert(90);
a.union_not(&b);
// After union-not, `a` should contain all values in the domain, except for
// the ones that are in `b` and were _not_ already in `a`.
assert_eq!(
a.iter().collect::<Vec<_>>(),
(0usize..100).filter(|&x| !matches!(x, 7 | 63 | 90)).collect::<Vec<_>>(),
);
}
#[test]
fn chunked_bitset() {
let mut b0 = ChunkedBitSet::<usize>::new_empty(0);

6
compiler/rustc_infer/src/traits/util.rs
View file

@ -1,5 +1,5 @@
use rustc_data_structures::fx::FxHashSet;
use rustc_middle::ty::{self, ToPolyTraitRef, TyCtxt};
use rustc_middle::ty::{self, TyCtxt};
use rustc_span::{Ident, Span};
pub use rustc_type_ir::elaborate::*;
@ -125,8 +125,8 @@ pub fn transitive_bounds_that_define_assoc_item<'tcx>(
.iter_identity_copied()
.map(|(clause, _)| clause.instantiate_supertrait(tcx, trait_ref))
.filter_map(|clause| clause.as_trait_clause())
// FIXME: Negative supertraits are elaborated here lol
.map(|trait_pred| trait_pred.to_poly_trait_ref()),
.filter(|clause| clause.polarity() == ty::PredicatePolarity::Positive)
.map(|clause| clause.map_bound(|clause| clause.trait_ref)),
);
return Some(trait_ref);

31
compiler/rustc_lint/src/types/literal.rs
View file

@ -204,20 +204,35 @@ fn get_type_suggestion(t: Ty<'_>, val: u128, negative: bool) -> Option<&'static
match t.kind() {
ty::Uint(ty::UintTy::Usize) | ty::Int(ty::IntTy::Isize) => None,
ty::Uint(_) => Some(Integer::fit_unsigned(val).uint_ty_str()),
ty::Int(_) if negative => Some(Integer::fit_signed(-(val as i128)).int_ty_str()),
ty::Int(int) => {
let signed = Integer::fit_signed(val as i128);
let unsigned = Integer::fit_unsigned(val);
Some(if Some(unsigned.size().bits()) == int.bit_width() {
unsigned.uint_ty_str()
ty::Int(_) => {
let signed = literal_to_i128(val, negative).map(Integer::fit_signed);
if negative {
signed.map(Integer::int_ty_str)
} else {
signed.int_ty_str()
})
let unsigned = Integer::fit_unsigned(val);
Some(if let Some(signed) = signed {
if unsigned.size() < signed.size() {
unsigned.uint_ty_str()
} else {
signed.int_ty_str()
}
} else {
unsigned.uint_ty_str()
})
}
}
_ => None,
}
}
fn literal_to_i128(val: u128, negative: bool) -> Option<i128> {
if negative {
(val <= i128::MAX as u128 + 1).then(|| val.wrapping_neg() as i128)
} else {
val.try_into().ok()
}
}
fn lint_int_literal<'tcx>(
cx: &LateContext<'tcx>,
type_limits: &TypeLimits,

2
compiler/rustc_lint_defs/src/builtin.rs
View file

@ -3597,7 +3597,7 @@ declare_lint! {
///
/// [Other ABIs]: https://doc.rust-lang.org/reference/items/external-blocks.html#abi
pub MISSING_ABI,
Allow,
Warn,
"No declared ABI for extern declaration"
}

3
compiler/rustc_middle/src/middle/codegen_fn_attrs.rs
View file

@ -30,6 +30,8 @@ pub struct CodegenFnAttrs {
/// features (only enabled features are supported right now).
/// Implied target features have already been applied.
pub target_features: Vec<TargetFeature>,
/// Whether the function was declared safe, but has target features
pub safe_target_features: bool,
/// The `#[linkage = "..."]` attribute on Rust-defined items and the value we found.
pub linkage: Option<Linkage>,
/// The `#[linkage = "..."]` attribute on foreign items and the value we found.
@ -150,6 +152,7 @@ impl CodegenFnAttrs {
link_name: None,
link_ordinal: None,
target_features: vec![],
safe_target_features: false,
linkage: None,
import_linkage: None,
link_section: None,

110
compiler/rustc_middle/src/middle/stability.rs
View file

@ -30,6 +30,14 @@ pub enum StabilityLevel {
Stable,
}
#[derive(Copy, Clone)]
pub enum UnstableKind {
/// Enforcing regular stability of an item
Regular,
/// Enforcing const stability of an item
Const(Span),
}
/// An entry in the `depr_map`.
#[derive(Copy, Clone, HashStable, Debug, Encodable, Decodable)]
pub struct DeprecationEntry {
@ -108,10 +116,16 @@ pub fn report_unstable(
is_soft: bool,
span: Span,
soft_handler: impl FnOnce(&'static Lint, Span, String),
kind: UnstableKind,
) {
let qual = match kind {
UnstableKind::Regular => "",
UnstableKind::Const(_) => " const",
};
let msg = match reason {
Some(r) => format!("use of unstable library feature `{feature}`: {r}"),
None => format!("use of unstable library feature `{feature}`"),
Some(r) => format!("use of unstable{qual} library feature `{feature}`: {r}"),
None => format!("use of unstable{qual} library feature `{feature}`"),
};
if is_soft {
@ -121,6 +135,9 @@ pub fn report_unstable(
if let Some((inner_types, msg, sugg, applicability)) = suggestion {
err.span_suggestion(inner_types, msg, sugg, applicability);
}
if let UnstableKind::Const(kw) = kind {
err.span_label(kw, "trait is not stable as const yet");
}
err.emit();
}
}
@ -232,9 +249,18 @@ fn late_report_deprecation(
return;
}
let is_in_effect = depr.is_in_effect();
let lint = deprecation_lint(is_in_effect);
// Calculating message for lint involves calling `self.def_path_str`,
// which will by default invoke the expensive `visible_parent_map` query.
// Skip all that work if the lint is allowed anyway.
if tcx.lint_level_at_node(lint, hir_id).0 == Level::Allow {
return;
}
let def_path = with_no_trimmed_paths!(tcx.def_path_str(def_id));
let def_kind = tcx.def_descr(def_id);
let is_in_effect = depr.is_in_effect();
let method_span = method_span.unwrap_or(span);
let suggestion =
@ -250,7 +276,7 @@ fn late_report_deprecation(
note: depr.note,
since_kind: deprecated_since_kind(is_in_effect, depr.since),
};
tcx.emit_node_span_lint(deprecation_lint(is_in_effect), hir_id, method_span, diag);
tcx.emit_node_span_lint(lint, hir_id, method_span, diag);
}
/// Result of `TyCtxt::eval_stability`.
@ -360,13 +386,7 @@ impl<'tcx> TyCtxt<'tcx> {
// hierarchy.
let depr_attr = &depr_entry.attr;
if !skip || depr_attr.is_since_rustc_version() {
// Calculating message for lint involves calling `self.def_path_str`.
// Which by default to calculate visible path will invoke expensive `visible_parent_map` query.
// So we skip message calculation altogether, if lint is allowed.
let lint = deprecation_lint(depr_attr.is_in_effect());
if self.lint_level_at_node(lint, id).0 != Level::Allow {
late_report_deprecation(self, depr_attr, span, method_span, id, def_id);
}
late_report_deprecation(self, depr_attr, span, method_span, id, def_id);
}
};
}
@ -587,6 +607,7 @@ impl<'tcx> TyCtxt<'tcx> {
is_soft,
span,
soft_handler,
UnstableKind::Regular,
),
EvalResult::Unmarked => unmarked(span, def_id),
}
@ -594,6 +615,73 @@ impl<'tcx> TyCtxt<'tcx> {
is_allowed
}
/// This function is analogous to `check_optional_stability` but with the logic in
/// `eval_stability_allow_unstable` inlined, and which operating on const stability
/// instead of regular stability.
///
/// This enforces *syntactical* const stability of const traits. In other words,
/// it enforces the ability to name `~const`/`const` traits in trait bounds in various
/// syntax positions in HIR (including in the trait of an impl header).
pub fn check_const_stability(self, def_id: DefId, span: Span, const_kw_span: Span) {
let is_staged_api = self.lookup_stability(def_id.krate.as_def_id()).is_some();
if !is_staged_api {
return;
}
// Only the cross-crate scenario matters when checking unstable APIs
let cross_crate = !def_id.is_local();
if !cross_crate {
return;
}
let stability = self.lookup_const_stability(def_id);
debug!(
"stability: \
inspecting def_id={:?} span={:?} of stability={:?}",
def_id, span, stability
);
match stability {
Some(ConstStability {
level: attr::StabilityLevel::Unstable { reason, issue, is_soft, implied_by, .. },
feature,
..
}) => {
assert!(!is_soft);
if span.allows_unstable(feature) {
debug!("body stability: skipping span={:?} since it is internal", span);
return;
}
if self.features().enabled(feature) {
return;
}
// If this item was previously part of a now-stabilized feature which is still
// enabled (i.e. the user hasn't removed the attribute for the stabilized feature
// yet) then allow use of this item.
if let Some(implied_by) = implied_by
&& self.features().enabled(implied_by)
{
return;
}
report_unstable(
self.sess,
feature,
reason.to_opt_reason(),
issue,
None,
false,
span,
|_, _, _| {},
UnstableKind::Const(const_kw_span),
);
}
Some(_) | None => {}
}
}
pub fn lookup_deprecation(self, id: DefId) -> Option<Deprecation> {
self.lookup_deprecation_entry(id).map(|depr| depr.attr)
}

8
compiler/rustc_middle/src/mir/coverage.rs
View file

@ -71,11 +71,7 @@ impl ConditionId {
/// Enum that can hold a constant zero value, the ID of an physical coverage
/// counter, or the ID of a coverage-counter expression.
///
/// This was originally only used for expression operands (and named `Operand`),
/// but the zero/counter/expression distinction is also useful for representing
/// the value of code/gap mappings, and the true/false arms of branch mappings.
#[derive(Copy, Clone, PartialEq, Eq)]
#[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord)]
#[derive(TyEncodable, TyDecodable, Hash, HashStable, TypeFoldable, TypeVisitable)]
pub enum CovTerm {
Zero,
@ -171,7 +167,7 @@ impl Op {
}
}
#[derive(Clone, Debug)]
#[derive(Clone, Debug, PartialEq, Eq)]
#[derive(TyEncodable, TyDecodable, Hash, HashStable, TypeFoldable, TypeVisitable)]
pub struct Expression {
pub lhs: CovTerm,

33
compiler/rustc_middle/src/ty/context.rs
View file

@ -60,7 +60,7 @@ use crate::dep_graph::{DepGraph, DepKindStruct};
use crate::infer::canonical::{CanonicalParamEnvCache, CanonicalVarInfo, CanonicalVarInfos};
use crate::lint::lint_level;
use crate::metadata::ModChild;
use crate::middle::codegen_fn_attrs::CodegenFnAttrs;
use crate::middle::codegen_fn_attrs::{CodegenFnAttrs, TargetFeature};
use crate::middle::{resolve_bound_vars, stability};
use crate::mir::interpret::{self, Allocation, ConstAllocation};
use crate::mir::{Body, Local, Place, PlaceElem, ProjectionKind, Promoted};
@ -1776,6 +1776,37 @@ impl<'tcx> TyCtxt<'tcx> {
pub fn dcx(self) -> DiagCtxtHandle<'tcx> {
self.sess.dcx()
}
pub fn is_target_feature_call_safe(
self,
callee_features: &[TargetFeature],
body_features: &[TargetFeature],
) -> bool {
// If the called function has target features the calling function hasn't,
// the call requires `unsafe`. Don't check this on wasm
// targets, though. For more information on wasm see the
// is_like_wasm check in hir_analysis/src/collect.rs
self.sess.target.options.is_like_wasm
|| callee_features
.iter()
.all(|feature| body_features.iter().any(|f| f.name == feature.name))
}
/// Returns the safe version of the signature of the given function, if calling it
/// would be safe in the context of the given caller.
pub fn adjust_target_feature_sig(
self,
fun_def: DefId,
fun_sig: ty::Binder<'tcx, ty::FnSig<'tcx>>,
caller: DefId,
) -> Option<ty::Binder<'tcx, ty::FnSig<'tcx>>> {
let fun_features = &self.codegen_fn_attrs(fun_def).target_features;
let callee_features = &self.codegen_fn_attrs(caller).target_features;
if self.is_target_feature_call_safe(&fun_features, &callee_features) {
return Some(fun_sig.map_bound(|sig| ty::FnSig { safety: hir::Safety::Safe, ..sig }));
}
None
}
}
impl<'tcx> TyCtxtAt<'tcx> {

1
compiler/rustc_middle/src/ty/layout.rs
View file

@ -1241,6 +1241,7 @@ pub fn fn_can_unwind(tcx: TyCtxt<'_>, fn_def_id: Option<DefId>, abi: ExternAbi)
PtxKernel
| Msp430Interrupt
| X86Interrupt
| GpuKernel
| EfiApi
| AvrInterrupt
| AvrNonBlockingInterrupt

4
compiler/rustc_middle/src/ty/mod.rs
View file

@ -79,8 +79,7 @@ pub use self::predicate::{
PolyExistentialPredicate, PolyExistentialProjection, PolyExistentialTraitRef,
PolyProjectionPredicate, PolyRegionOutlivesPredicate, PolySubtypePredicate, PolyTraitPredicate,
PolyTraitRef, PolyTypeOutlivesPredicate, Predicate, PredicateKind, ProjectionPredicate,
RegionOutlivesPredicate, SubtypePredicate, ToPolyTraitRef, TraitPredicate, TraitRef,
TypeOutlivesPredicate,
RegionOutlivesPredicate, SubtypePredicate, TraitPredicate, TraitRef, TypeOutlivesPredicate,
};
pub use self::region::{
BoundRegion, BoundRegionKind, EarlyParamRegion, LateParamRegion, LateParamRegionKind, Region,
@ -222,6 +221,7 @@ pub struct DelegationFnSig {
pub param_count: usize,
pub has_self: bool,
pub c_variadic: bool,
pub target_feature: bool,
}
#[derive(Clone, Copy, Debug)]

10
compiler/rustc_middle/src/ty/predicate.rs
View file

@ -476,16 +476,6 @@ impl<'tcx> Clause<'tcx> {
}
}
pub trait ToPolyTraitRef<'tcx> {
fn to_poly_trait_ref(&self) -> PolyTraitRef<'tcx>;
}
impl<'tcx> ToPolyTraitRef<'tcx> for PolyTraitPredicate<'tcx> {
fn to_poly_trait_ref(&self) -> PolyTraitRef<'tcx> {
self.map_bound_ref(|trait_pred| trait_pred.trait_ref)
}
}
impl<'tcx> UpcastFrom<TyCtxt<'tcx>, PredicateKind<'tcx>> for Predicate<'tcx> {
fn upcast_from(from: PredicateKind<'tcx>, tcx: TyCtxt<'tcx>) -> Self {
ty::Binder::dummy(from).upcast(tcx)

9
compiler/rustc_middle/src/ty/print/pretty.rs
View file

@ -690,7 +690,14 @@ pub trait PrettyPrinter<'tcx>: Printer<'tcx> + fmt::Write {
if with_reduced_queries() {
p!(print_def_path(def_id, args));
} else {
let sig = self.tcx().fn_sig(def_id).instantiate(self.tcx(), args);
let mut sig = self.tcx().fn_sig(def_id).instantiate(self.tcx(), args);
if self.tcx().codegen_fn_attrs(def_id).safe_target_features {
p!("#[target_features] ");
sig = sig.map_bound(|mut sig| {
sig.safety = hir::Safety::Safe;
sig
});
}
p!(print(sig), " {{", print_value_path(def_id, args), "}}");
}
}

39
compiler/rustc_middle/src/ty/sty.rs
View file

@ -401,7 +401,7 @@ impl<'tcx> Ty<'tcx> {
/// The more specific methods will often optimize their creation.
#[allow(rustc::usage_of_ty_tykind)]
#[inline]
pub fn new(tcx: TyCtxt<'tcx>, st: TyKind<'tcx>) -> Ty<'tcx> {
fn new(tcx: TyCtxt<'tcx>, st: TyKind<'tcx>) -> Ty<'tcx> {
tcx.mk_ty_from_kind(st)
}
@ -613,6 +613,41 @@ impl<'tcx> Ty<'tcx> {
#[inline]
pub fn new_adt(tcx: TyCtxt<'tcx>, def: AdtDef<'tcx>, args: GenericArgsRef<'tcx>) -> Ty<'tcx> {
tcx.debug_assert_args_compatible(def.did(), args);
if cfg!(debug_assertions) {
match tcx.def_kind(def.did()) {
DefKind::Struct | DefKind::Union | DefKind::Enum => {}
DefKind::Mod
| DefKind::Variant
| DefKind::Trait
| DefKind::TyAlias
| DefKind::ForeignTy
| DefKind::TraitAlias
| DefKind::AssocTy
| DefKind::TyParam
| DefKind::Fn
| DefKind::Const
| DefKind::ConstParam
| DefKind::Static { .. }
| DefKind::Ctor(..)
| DefKind::AssocFn
| DefKind::AssocConst
| DefKind::Macro(..)
| DefKind::ExternCrate
| DefKind::Use
| DefKind::ForeignMod
| DefKind::AnonConst
| DefKind::InlineConst
| DefKind::OpaqueTy
| DefKind::Field
| DefKind::LifetimeParam
| DefKind::GlobalAsm
| DefKind::Impl { .. }
| DefKind::Closure
| DefKind::SyntheticCoroutineBody => {
bug!("not an adt: {def:?} ({:?})", tcx.def_kind(def.did()))
}
}
}
Ty::new(tcx, Adt(def, args))
}
@ -772,7 +807,7 @@ impl<'tcx> Ty<'tcx> {
}
}
});
Ty::new(tcx, Adt(adt_def, args))
Ty::new_adt(tcx, adt_def, args)
}
#[inline]

45
compiler/rustc_mir_build/src/check_unsafety.rs
View file

@ -478,23 +478,26 @@ impl<'a, 'tcx> Visitor<'a, 'tcx> for UnsafetyVisitor<'a, 'tcx> {
return; // don't visit the whole expression
}
ExprKind::Call { fun, ty: _, args: _, from_hir_call: _, fn_span: _ } => {
if self.thir[fun].ty.fn_sig(self.tcx).safety().is_unsafe() {
let func_id = if let ty::FnDef(func_id, _) = self.thir[fun].ty.kind() {
let fn_ty = self.thir[fun].ty;
let sig = fn_ty.fn_sig(self.tcx);
let (callee_features, safe_target_features): (&[_], _) = match fn_ty.kind() {
ty::FnDef(func_id, ..) => {
let cg_attrs = self.tcx.codegen_fn_attrs(func_id);
(&cg_attrs.target_features, cg_attrs.safe_target_features)
}
_ => (&[], false),
};
if sig.safety().is_unsafe() && !safe_target_features {
let func_id = if let ty::FnDef(func_id, _) = fn_ty.kind() {
Some(*func_id)
} else {
None
};
self.requires_unsafe(expr.span, CallToUnsafeFunction(func_id));
} else if let &ty::FnDef(func_did, _) = self.thir[fun].ty.kind() {
// If the called function has target features the calling function hasn't,
// the call requires `unsafe`. Don't check this on wasm
// targets, though. For more information on wasm see the
// is_like_wasm check in hir_analysis/src/collect.rs
let callee_features = &self.tcx.codegen_fn_attrs(func_did).target_features;
if !self.tcx.sess.target.options.is_like_wasm
&& !callee_features.iter().all(|feature| {
self.body_target_features.iter().any(|f| f.name == feature.name)
})
} else if let &ty::FnDef(func_did, _) = fn_ty.kind() {
if !self
.tcx
.is_target_feature_call_safe(callee_features, self.body_target_features)
{
let missing: Vec<_> = callee_features
.iter()
@ -739,7 +742,10 @@ impl UnsafeOpKind {
) {
let parent_id = tcx.hir().get_parent_item(hir_id);
let parent_owner = tcx.hir_owner_node(parent_id);
let should_suggest = parent_owner.fn_sig().is_some_and(|sig| sig.header.is_unsafe());
let should_suggest = parent_owner.fn_sig().is_some_and(|sig| {
// Do not suggest for safe target_feature functions
matches!(sig.header.safety, hir::HeaderSafety::Normal(hir::Safety::Unsafe))
});
let unsafe_not_inherited_note = if should_suggest {
suggest_unsafe_block.then(|| {
let body_span = tcx.hir().body(parent_owner.body_id().unwrap()).value.span;
@ -902,7 +908,7 @@ impl UnsafeOpKind {
{
true
} else if let Some(sig) = tcx.hir().fn_sig_by_hir_id(*id)
&& sig.header.is_unsafe()
&& matches!(sig.header.safety, hir::HeaderSafety::Normal(hir::Safety::Unsafe))
{
true
} else {
@ -1111,7 +1117,16 @@ pub(crate) fn check_unsafety(tcx: TyCtxt<'_>, def: LocalDefId) {
let hir_id = tcx.local_def_id_to_hir_id(def);
let safety_context = tcx.hir().fn_sig_by_hir_id(hir_id).map_or(SafetyContext::Safe, |fn_sig| {
if fn_sig.header.safety.is_unsafe() { SafetyContext::UnsafeFn } else { SafetyContext::Safe }
match fn_sig.header.safety {
// We typeck the body as safe, but otherwise treat it as unsafe everywhere else.
// Call sites to other SafeTargetFeatures functions are checked explicitly and don't need
// to care about safety of the body.
hir::HeaderSafety::SafeTargetFeatures => SafetyContext::Safe,
hir::HeaderSafety::Normal(safety) => match safety {
hir::Safety::Unsafe => SafetyContext::UnsafeFn,
hir::Safety::Safe => SafetyContext::Safe,
},
}
});
let body_target_features = &tcx.body_codegen_attrs(def.to_def_id()).target_features;
let mut warnings = Vec::new();

640
compiler/rustc_mir_transform/src/coverage/counters.rs
View file

@ -1,117 +1,181 @@
use std::cmp::Ordering;
use std::fmt::{self, Debug};
use either::Either;
use itertools::Itertools;
use rustc_data_structures::captures::Captures;
use rustc_data_structures::fx::FxHashMap;
use rustc_data_structures::fx::{FxHashMap, FxIndexMap};
use rustc_data_structures::graph::DirectedGraph;
use rustc_index::IndexVec;
use rustc_index::bit_set::DenseBitSet;
use rustc_middle::mir::coverage::{CounterId, CovTerm, Expression, ExpressionId, Op};
use tracing::{debug, debug_span, instrument};
use crate::coverage::graph::{BasicCoverageBlock, CoverageGraph, ReadyFirstTraversal};
use crate::coverage::counters::balanced_flow::BalancedFlowGraph;
use crate::coverage::counters::iter_nodes::IterNodes;
use crate::coverage::counters::node_flow::{CounterTerm, MergedNodeFlowGraph, NodeCounters};
use crate::coverage::graph::{BasicCoverageBlock, CoverageGraph};
#[cfg(test)]
mod tests;
mod balanced_flow;
mod iter_nodes;
mod node_flow;
mod union_find;
/// The coverage counter or counter expression associated with a particular
/// BCB node or BCB edge.
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
enum BcbCounter {
Counter { id: CounterId },
Expression { id: ExpressionId },
/// Ensures that each BCB node needing a counter has one, by creating physical
/// counters or counter expressions for nodes as required.
pub(super) fn make_bcb_counters(
graph: &CoverageGraph,
bcb_needs_counter: &DenseBitSet<BasicCoverageBlock>,
) -> CoverageCounters {
// Create the derived graphs that are necessary for subsequent steps.
let balanced_graph = BalancedFlowGraph::for_graph(graph, |n| !graph[n].is_out_summable);
let merged_graph = MergedNodeFlowGraph::for_balanced_graph(&balanced_graph);
// Use those graphs to determine which nodes get physical counters, and how
// to compute the execution counts of other nodes from those counters.
let nodes = make_node_counter_priority_list(graph, balanced_graph);
let node_counters = merged_graph.make_node_counters(&nodes);
// Convert the counters into a form suitable for embedding into MIR.
transcribe_counters(&node_counters, bcb_needs_counter)
}
impl BcbCounter {
fn as_term(&self) -> CovTerm {
match *self {
BcbCounter::Counter { id, .. } => CovTerm::Counter(id),
BcbCounter::Expression { id, .. } => CovTerm::Expression(id),
/// Arranges the nodes in `balanced_graph` into a list, such that earlier nodes
/// take priority in being given a counter expression instead of a physical counter.
fn make_node_counter_priority_list(
graph: &CoverageGraph,
balanced_graph: BalancedFlowGraph<&CoverageGraph>,
) -> Vec<BasicCoverageBlock> {
// A "reloop" node has exactly one out-edge, which jumps back to the top
// of an enclosing loop. Reloop nodes are typically visited more times
// than loop-exit nodes, so try to avoid giving them physical counters.
let is_reloop_node = IndexVec::from_fn_n(
|node| match graph.successors[node].as_slice() {
&[succ] => graph.dominates(succ, node),
_ => false,
},
graph.num_nodes(),
);
let mut nodes = balanced_graph.iter_nodes().rev().collect::<Vec<_>>();
// The first node is the sink, which must not get a physical counter.
assert_eq!(nodes[0], balanced_graph.sink);
// Sort the real nodes, such that earlier (lesser) nodes take priority
// in being given a counter expression instead of a physical counter.
nodes[1..].sort_by(|&a, &b| {
// Start with a dummy `Equal` to make the actual tests line up nicely.
Ordering::Equal
// Prefer a physical counter for return/yield nodes.
.then_with(|| Ord::cmp(&graph[a].is_out_summable, &graph[b].is_out_summable))
// Prefer an expression for reloop nodes (see definition above).
.then_with(|| Ord::cmp(&is_reloop_node[a], &is_reloop_node[b]).reverse())
// Otherwise, prefer a physical counter for dominating nodes.
.then_with(|| graph.cmp_in_dominator_order(a, b).reverse())
});
nodes
}
// Converts node counters into a form suitable for embedding into MIR.
fn transcribe_counters(
old: &NodeCounters<BasicCoverageBlock>,
bcb_needs_counter: &DenseBitSet<BasicCoverageBlock>,
) -> CoverageCounters {
let mut new = CoverageCounters::with_num_bcbs(bcb_needs_counter.domain_size());
for bcb in bcb_needs_counter.iter() {
// Our counter-creation algorithm doesn't guarantee that a counter
// expression starts or ends with a positive term, so partition the
// counters into "positive" and "negative" lists for easier handling.
let (mut pos, mut neg): (Vec<_>, Vec<_>) =
old.counter_expr(bcb).iter().partition_map(|&CounterTerm { node, op }| match op {
Op::Add => Either::Left(node),
Op::Subtract => Either::Right(node),
});
if pos.is_empty() {
// If we somehow end up with no positive terms, fall back to
// creating a physical counter. There's no known way for this
// to happen, but we can avoid an ICE if it does.
debug_assert!(false, "{bcb:?} has no positive counter terms");
pos = vec![bcb];
neg = vec![];
}
// These intermediate sorts are not strictly necessary, but were helpful
// in reducing churn when switching to the current counter-creation scheme.
// They also help to slightly decrease the overall size of the expression
// table, due to more subexpressions being shared.
pos.sort();
neg.sort();
let mut new_counters_for_sites = |sites: Vec<BasicCoverageBlock>| {
sites.into_iter().map(|node| new.ensure_phys_counter(node)).collect::<Vec<_>>()
};
let mut pos = new_counters_for_sites(pos);
let mut neg = new_counters_for_sites(neg);
// These sorts are also not strictly necessary; see above.
pos.sort();
neg.sort();
let pos_counter = new.make_sum(&pos).expect("`pos` should not be empty");
let new_counter = new.make_subtracted_sum(pos_counter, &neg);
new.set_node_counter(bcb, new_counter);
}
}
impl Debug for BcbCounter {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Counter { id, .. } => write!(fmt, "Counter({:?})", id.index()),
Self::Expression { id } => write!(fmt, "Expression({:?})", id.index()),
}
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
struct BcbExpression {
lhs: BcbCounter,
op: Op,
rhs: BcbCounter,
}
/// Enum representing either a node or an edge in the coverage graph.
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub(super) enum Site {
Node { bcb: BasicCoverageBlock },
Edge { from_bcb: BasicCoverageBlock, to_bcb: BasicCoverageBlock },
new
}
/// Generates and stores coverage counter and coverage expression information
/// associated with nodes/edges in the BCB graph.
/// associated with nodes in the coverage graph.
pub(super) struct CoverageCounters {
/// List of places where a counter-increment statement should be injected
/// into MIR, each with its corresponding counter ID.
counter_increment_sites: IndexVec<CounterId, Site>,
phys_counter_for_node: FxIndexMap<BasicCoverageBlock, CounterId>,
next_counter_id: CounterId,
/// Coverage counters/expressions that are associated with individual BCBs.
node_counters: IndexVec<BasicCoverageBlock, Option<BcbCounter>>,
node_counters: IndexVec<BasicCoverageBlock, Option<CovTerm>>,
/// Table of expression data, associating each expression ID with its
/// corresponding operator (+ or -) and its LHS/RHS operands.
expressions: IndexVec<ExpressionId, BcbExpression>,
expressions: IndexVec<ExpressionId, Expression>,
/// Remember expressions that have already been created (or simplified),
/// so that we don't create unnecessary duplicates.
expressions_memo: FxHashMap<BcbExpression, BcbCounter>,
expressions_memo: FxHashMap<Expression, CovTerm>,
}
impl CoverageCounters {
/// Ensures that each BCB node needing a counter has one, by creating physical
/// counters or counter expressions for nodes and edges as required.
pub(super) fn make_bcb_counters(
graph: &CoverageGraph,
bcb_needs_counter: &DenseBitSet<BasicCoverageBlock>,
) -> Self {
let mut builder = CountersBuilder::new(graph, bcb_needs_counter);
builder.make_bcb_counters();
builder.into_coverage_counters()
}
fn with_num_bcbs(num_bcbs: usize) -> Self {
Self {
counter_increment_sites: IndexVec::new(),
phys_counter_for_node: FxIndexMap::default(),
next_counter_id: CounterId::ZERO,
node_counters: IndexVec::from_elem_n(None, num_bcbs),
expressions: IndexVec::new(),
expressions_memo: FxHashMap::default(),
}
}
/// Creates a new physical counter for a BCB node or edge.
fn make_phys_counter(&mut self, site: Site) -> BcbCounter {
let id = self.counter_increment_sites.push(site);
BcbCounter::Counter { id }
/// Returns the physical counter for the given node, creating it if necessary.
fn ensure_phys_counter(&mut self, bcb: BasicCoverageBlock) -> CovTerm {
let id = *self.phys_counter_for_node.entry(bcb).or_insert_with(|| {
let id = self.next_counter_id;
self.next_counter_id = id + 1;
id
});
CovTerm::Counter(id)
}
fn make_expression(&mut self, lhs: BcbCounter, op: Op, rhs: BcbCounter) -> BcbCounter {
let new_expr = BcbExpression { lhs, op, rhs };
*self.expressions_memo.entry(new_expr).or_insert_with(|| {
fn make_expression(&mut self, lhs: CovTerm, op: Op, rhs: CovTerm) -> CovTerm {
let new_expr = Expression { lhs, op, rhs };
*self.expressions_memo.entry(new_expr.clone()).or_insert_with(|| {
let id = self.expressions.push(new_expr);
BcbCounter::Expression { id }
CovTerm::Expression(id)
})
}
/// Creates a counter that is the sum of the given counters.
///
/// Returns `None` if the given list of counters was empty.
fn make_sum(&mut self, counters: &[BcbCounter]) -> Option<BcbCounter> {
fn make_sum(&mut self, counters: &[CovTerm]) -> Option<CovTerm> {
counters
.iter()
.copied()
@ -119,16 +183,18 @@ impl CoverageCounters {
}
/// Creates a counter whose value is `lhs - SUM(rhs)`.
fn make_subtracted_sum(&mut self, lhs: BcbCounter, rhs: &[BcbCounter]) -> BcbCounter {
fn make_subtracted_sum(&mut self, lhs: CovTerm, rhs: &[CovTerm]) -> CovTerm {
let Some(rhs_sum) = self.make_sum(rhs) else { return lhs };
self.make_expression(lhs, Op::Subtract, rhs_sum)
}
pub(super) fn num_counters(&self) -> usize {
self.counter_increment_sites.len()
let num_counters = self.phys_counter_for_node.len();
assert_eq!(num_counters, self.next_counter_id.as_usize());
num_counters
}
fn set_node_counter(&mut self, bcb: BasicCoverageBlock, counter: BcbCounter) -> BcbCounter {
fn set_node_counter(&mut self, bcb: BasicCoverageBlock, counter: CovTerm) -> CovTerm {
let existing = self.node_counters[bcb].replace(counter);
assert!(
existing.is_none(),
@ -138,16 +204,16 @@ impl CoverageCounters {
}
pub(super) fn term_for_bcb(&self, bcb: BasicCoverageBlock) -> Option<CovTerm> {
self.node_counters[bcb].map(|counter| counter.as_term())
self.node_counters[bcb]
}
/// Returns an iterator over all the nodes/edges in the coverage graph that
/// Returns an iterator over all the nodes in the coverage graph that
/// should have a counter-increment statement injected into MIR, along with
/// each site's corresponding counter ID.
pub(super) fn counter_increment_sites(
&self,
) -> impl Iterator<Item = (CounterId, Site)> + Captures<'_> {
self.counter_increment_sites.iter_enumerated().map(|(id, &site)| (id, site))
) -> impl Iterator<Item = (CounterId, BasicCoverageBlock)> + Captures<'_> {
self.phys_counter_for_node.iter().map(|(&site, &id)| (id, site))
}
/// Returns an iterator over the subset of BCB nodes that have been associated
@ -157,435 +223,13 @@ impl CoverageCounters {
) -> impl Iterator<Item = (BasicCoverageBlock, ExpressionId)> + Captures<'_> {
self.node_counters.iter_enumerated().filter_map(|(bcb, &counter)| match counter {
// Yield the BCB along with its associated expression ID.
Some(BcbCounter::Expression { id }) => Some((bcb, id)),
Some(CovTerm::Expression(id)) => Some((bcb, id)),
// This BCB is associated with a counter or nothing, so skip it.
Some(BcbCounter::Counter { .. }) | None => None,
Some(CovTerm::Counter { .. } | CovTerm::Zero) | None => None,
})
}
pub(super) fn into_expressions(self) -> IndexVec<ExpressionId, Expression> {
let old_len = self.expressions.len();
let expressions = self
.expressions
.into_iter()
.map(|BcbExpression { lhs, op, rhs }| Expression {
lhs: lhs.as_term(),
op,
rhs: rhs.as_term(),
})
.collect::<IndexVec<ExpressionId, _>>();
// Expression IDs are indexes into this vector, so make sure we didn't
// accidentally invalidate them by changing its length.
assert_eq!(old_len, expressions.len());
expressions
self.expressions
}
}
/// Symbolic representation of the coverage counter to be used for a particular
/// node or edge in the coverage graph. The same site counter can be used for
/// multiple sites, if they have been determined to have the same count.
#[derive(Clone, Copy, Debug)]
enum SiteCounter {
/// A physical counter at some node/edge.
Phys { site: Site },
/// A counter expression for a node that takes the sum of all its in-edge
/// counters.
NodeSumExpr { bcb: BasicCoverageBlock },
/// A counter expression for an edge that takes the counter of its source
/// node, and subtracts the counters of all its sibling out-edges.
EdgeDiffExpr { from_bcb: BasicCoverageBlock, to_bcb: BasicCoverageBlock },
}
/// Yields the graph successors of `from_bcb` that aren't `to_bcb`. This is
/// used when creating a counter expression for [`SiteCounter::EdgeDiffExpr`].
///
/// For example, in this diagram the sibling out-edge targets of edge `AC` are
/// the nodes `B` and `D`.
///
/// ```text
/// A
/// / | \
/// B C D
/// ```
fn sibling_out_edge_targets(
graph: &CoverageGraph,
from_bcb: BasicCoverageBlock,
to_bcb: BasicCoverageBlock,
) -> impl Iterator<Item = BasicCoverageBlock> + Captures<'_> {
graph.successors[from_bcb].iter().copied().filter(move |&t| t != to_bcb)
}
/// Helper struct that allows counter creation to inspect the BCB graph, and
/// the set of nodes that need counters.
struct CountersBuilder<'a> {
graph: &'a CoverageGraph,
bcb_needs_counter: &'a DenseBitSet<BasicCoverageBlock>,
site_counters: FxHashMap<Site, SiteCounter>,
}
impl<'a> CountersBuilder<'a> {
fn new(
graph: &'a CoverageGraph,
bcb_needs_counter: &'a DenseBitSet<BasicCoverageBlock>,
) -> Self {
assert_eq!(graph.num_nodes(), bcb_needs_counter.domain_size());
Self { graph, bcb_needs_counter, site_counters: FxHashMap::default() }
}
fn make_bcb_counters(&mut self) {
debug!("make_bcb_counters(): adding a counter or expression to each BasicCoverageBlock");
// Traverse the coverage graph, ensuring that every node that needs a
// coverage counter has one.
for bcb in ReadyFirstTraversal::new(self.graph) {
let _span = debug_span!("traversal", ?bcb).entered();
if self.bcb_needs_counter.contains(bcb) {
self.make_node_counter_and_out_edge_counters(bcb);
}
}
}
/// Make sure the given node has a node counter, and then make sure each of
/// its out-edges has an edge counter (if appropriate).
#[instrument(level = "debug", skip(self))]
fn make_node_counter_and_out_edge_counters(&mut self, from_bcb: BasicCoverageBlock) {
// First, ensure that this node has a counter of some kind.
// We might also use that counter to compute one of the out-edge counters.
self.get_or_make_node_counter(from_bcb);
// If this node's out-edges won't sum to the node's counter,
// then there's no reason to create edge counters here.
if !self.graph[from_bcb].is_out_summable {
return;
}
// When choosing which out-edge should be given a counter expression, ignore edges that
// already have counters, or could use the existing counter of their target node.
let out_edge_has_counter = |to_bcb| {
if self.site_counters.contains_key(&Site::Edge { from_bcb, to_bcb }) {
return true;
}
self.graph.sole_predecessor(to_bcb) == Some(from_bcb)
&& self.site_counters.contains_key(&Site::Node { bcb: to_bcb })
};
// Determine the set of out-edges that could benefit from being given an expression.
let candidate_successors = self.graph.successors[from_bcb]
.iter()
.copied()
.filter(|&to_bcb| !out_edge_has_counter(to_bcb))
.collect::<Vec<_>>();
debug!(?candidate_successors);
// If there are out-edges without counters, choose one to be given an expression
// (computed from this node and the other out-edges) instead of a physical counter.
let Some(to_bcb) = self.choose_out_edge_for_expression(from_bcb, &candidate_successors)
else {
return;
};
// For each out-edge other than the one that was chosen to get an expression,
// ensure that it has a counter (existing counter/expression or a new counter).
for target in sibling_out_edge_targets(self.graph, from_bcb, to_bcb) {
self.get_or_make_edge_counter(from_bcb, target);
}
// Now create an expression for the chosen edge, by taking the counter
// for its source node and subtracting the sum of its sibling out-edges.
let counter = SiteCounter::EdgeDiffExpr { from_bcb, to_bcb };
self.site_counters.insert(Site::Edge { from_bcb, to_bcb }, counter);
}
#[instrument(level = "debug", skip(self))]
fn get_or_make_node_counter(&mut self, bcb: BasicCoverageBlock) -> SiteCounter {
// If the BCB already has a counter, return it.
if let Some(&counter) = self.site_counters.get(&Site::Node { bcb }) {
debug!("{bcb:?} already has a counter: {counter:?}");
return counter;
}
let counter = self.make_node_counter_inner(bcb);
self.site_counters.insert(Site::Node { bcb }, counter);
counter
}
fn make_node_counter_inner(&mut self, bcb: BasicCoverageBlock) -> SiteCounter {
// If the node's sole in-edge already has a counter, use that.
if let Some(sole_pred) = self.graph.sole_predecessor(bcb)
&& let Some(&edge_counter) =
self.site_counters.get(&Site::Edge { from_bcb: sole_pred, to_bcb: bcb })
{
return edge_counter;
}
let predecessors = self.graph.predecessors[bcb].as_slice();
// Handle cases where we can't compute a node's count from its in-edges:
// - START_BCB has no in-edges, so taking the sum would panic (or be wrong).
// - For nodes with one in-edge, or that directly loop to themselves,
// trying to get the in-edge counts would require this node's counter,
// leading to infinite recursion.
if predecessors.len() <= 1 || predecessors.contains(&bcb) {
debug!(?bcb, ?predecessors, "node has <=1 predecessors or is its own predecessor");
let counter = SiteCounter::Phys { site: Site::Node { bcb } };
debug!(?bcb, ?counter, "node gets a physical counter");
return counter;
}
// A BCB with multiple incoming edges can compute its count by ensuring that counters
// exist for each of those edges, and then adding them up to get a total count.
for &from_bcb in predecessors {
self.get_or_make_edge_counter(from_bcb, bcb);
}
let sum_of_in_edges = SiteCounter::NodeSumExpr { bcb };
debug!("{bcb:?} gets a new counter (sum of predecessor counters): {sum_of_in_edges:?}");
sum_of_in_edges
}
#[instrument(level = "debug", skip(self))]
fn get_or_make_edge_counter(
&mut self,
from_bcb: BasicCoverageBlock,
to_bcb: BasicCoverageBlock,
) -> SiteCounter {
// If the edge already has a counter, return it.
if let Some(&counter) = self.site_counters.get(&Site::Edge { from_bcb, to_bcb }) {
debug!("Edge {from_bcb:?}->{to_bcb:?} already has a counter: {counter:?}");
return counter;
}
let counter = self.make_edge_counter_inner(from_bcb, to_bcb);
self.site_counters.insert(Site::Edge { from_bcb, to_bcb }, counter);
counter
}
fn make_edge_counter_inner(
&mut self,
from_bcb: BasicCoverageBlock,
to_bcb: BasicCoverageBlock,
) -> SiteCounter {
// If the target node has exactly one in-edge (i.e. this one), then just
// use the node's counter, since it will have the same value.
if let Some(sole_pred) = self.graph.sole_predecessor(to_bcb) {
assert_eq!(sole_pred, from_bcb);
// This call must take care not to invoke `get_or_make_edge` for
// this edge, since that would result in infinite recursion!
return self.get_or_make_node_counter(to_bcb);
}
// If the source node has exactly one out-edge (i.e. this one) and would have
// the same execution count as that edge, then just use the node's counter.
if let Some(simple_succ) = self.graph.simple_successor(from_bcb) {
assert_eq!(simple_succ, to_bcb);
return self.get_or_make_node_counter(from_bcb);
}
// Make a new counter to count this edge.
let counter = SiteCounter::Phys { site: Site::Edge { from_bcb, to_bcb } };
debug!(?from_bcb, ?to_bcb, ?counter, "edge gets a physical counter");
counter
}
/// Given a set of candidate out-edges (represented by their successor node),
/// choose one to be given a counter expression instead of a physical counter.
fn choose_out_edge_for_expression(
&self,
from_bcb: BasicCoverageBlock,
candidate_successors: &[BasicCoverageBlock],
) -> Option<BasicCoverageBlock> {
// Try to find a candidate that leads back to the top of a loop,
// because reloop edges tend to be executed more times than loop-exit edges.
if let Some(reloop_target) = self.find_good_reloop_edge(from_bcb, &candidate_successors) {
debug!("Selecting reloop target {reloop_target:?} to get an expression");
return Some(reloop_target);
}
// We couldn't identify a "good" edge, so just choose an arbitrary one.
let arbitrary_target = candidate_successors.first().copied()?;
debug!(?arbitrary_target, "selecting arbitrary out-edge to get an expression");
Some(arbitrary_target)
}
/// Given a set of candidate out-edges (represented by their successor node),
/// tries to find one that leads back to the top of a loop.
///
/// Reloop edges are good candidates for counter expressions, because they
/// will tend to be executed more times than a loop-exit edge, so it's nice
/// for them to be able to avoid a physical counter increment.
fn find_good_reloop_edge(
&self,
from_bcb: BasicCoverageBlock,
candidate_successors: &[BasicCoverageBlock],
) -> Option<BasicCoverageBlock> {
// If there are no candidates, avoid iterating over the loop stack.
if candidate_successors.is_empty() {
return None;
}
// Consider each loop on the current traversal context stack, top-down.
for loop_header_node in self.graph.loop_headers_containing(from_bcb) {
// Try to find a candidate edge that doesn't exit this loop.
for &target_bcb in candidate_successors {
// An edge is a reloop edge if its target dominates any BCB that has
// an edge back to the loop header. (Otherwise it's an exit edge.)
let is_reloop_edge = self
.graph
.reloop_predecessors(loop_header_node)
.any(|reloop_bcb| self.graph.dominates(target_bcb, reloop_bcb));
if is_reloop_edge {
// We found a good out-edge to be given an expression.
return Some(target_bcb);
}
}
// All of the candidate edges exit this loop, so keep looking
// for a good reloop edge for one of the outer loops.
}
None
}
fn into_coverage_counters(self) -> CoverageCounters {
Transcriber::new(&self).transcribe_counters()
}
}
/// Helper struct for converting `CountersBuilder` into a final `CoverageCounters`.
struct Transcriber<'a> {
old: &'a CountersBuilder<'a>,
new: CoverageCounters,
phys_counter_for_site: FxHashMap<Site, BcbCounter>,
}
impl<'a> Transcriber<'a> {
fn new(old: &'a CountersBuilder<'a>) -> Self {
Self {
old,
new: CoverageCounters::with_num_bcbs(old.graph.num_nodes()),
phys_counter_for_site: FxHashMap::default(),
}
}
fn transcribe_counters(mut self) -> CoverageCounters {
for bcb in self.old.bcb_needs_counter.iter() {
let site = Site::Node { bcb };
let site_counter = self.site_counter(site);
// Resolve the site counter into flat lists of nodes/edges whose
// physical counts contribute to the counter for this node.
// Distinguish between counts that will be added vs subtracted.
let mut pos = vec![];
let mut neg = vec![];
self.push_resolved_sites(site_counter, &mut pos, &mut neg);
// Simplify by cancelling out sites that appear on both sides.
let (mut pos, mut neg) = sort_and_cancel(pos, neg);
if pos.is_empty() {
// If we somehow end up with no positive terms after cancellation,
// fall back to creating a physical counter. There's no known way
// for this to happen, but it's hard to confidently rule it out.
debug_assert!(false, "{site:?} has no positive counter terms");
pos = vec![Some(site)];
neg = vec![];
}
let mut new_counters_for_sites = |sites: Vec<Option<Site>>| {
sites
.into_iter()
.filter_map(|id| try { self.ensure_phys_counter(id?) })
.collect::<Vec<_>>()
};
let mut pos = new_counters_for_sites(pos);
let mut neg = new_counters_for_sites(neg);
pos.sort();
neg.sort();
let pos_counter = self.new.make_sum(&pos).expect("`pos` should not be empty");
let new_counter = self.new.make_subtracted_sum(pos_counter, &neg);
self.new.set_node_counter(bcb, new_counter);
}
self.new
}
fn site_counter(&self, site: Site) -> SiteCounter {
self.old.site_counters.get(&site).copied().unwrap_or_else(|| {
// We should have already created all necessary site counters.
// But if we somehow didn't, avoid crashing in release builds,
// and just use an extra physical counter instead.
debug_assert!(false, "{site:?} should have a counter");
SiteCounter::Phys { site }
})
}
fn ensure_phys_counter(&mut self, site: Site) -> BcbCounter {
*self.phys_counter_for_site.entry(site).or_insert_with(|| self.new.make_phys_counter(site))
}
/// Resolves the given counter into flat lists of nodes/edges, whose counters
/// will then be added and subtracted to form a counter expression.
fn push_resolved_sites(&self, counter: SiteCounter, pos: &mut Vec<Site>, neg: &mut Vec<Site>) {
match counter {
SiteCounter::Phys { site } => pos.push(site),
SiteCounter::NodeSumExpr { bcb } => {
for &from_bcb in &self.old.graph.predecessors[bcb] {
let edge_counter = self.site_counter(Site::Edge { from_bcb, to_bcb: bcb });
self.push_resolved_sites(edge_counter, pos, neg);
}
}
SiteCounter::EdgeDiffExpr { from_bcb, to_bcb } => {
// First, add the count for `from_bcb`.
let node_counter = self.site_counter(Site::Node { bcb: from_bcb });
self.push_resolved_sites(node_counter, pos, neg);
// Then subtract the counts for the other out-edges.
for target in sibling_out_edge_targets(self.old.graph, from_bcb, to_bcb) {
let edge_counter = self.site_counter(Site::Edge { from_bcb, to_bcb: target });
// Swap `neg` and `pos` so that the counter is subtracted.
self.push_resolved_sites(edge_counter, neg, pos);
}
}
}
}
}
/// Given two lists:
/// - Sorts each list.
/// - Converts each list to `Vec<Option<T>>`.
/// - Scans for values that appear in both lists, and cancels them out by
/// replacing matching pairs of values with `None`.
fn sort_and_cancel<T: Ord>(mut pos: Vec<T>, mut neg: Vec<T>) -> (Vec<Option<T>>, Vec<Option<T>>) {
pos.sort();
neg.sort();
// Convert to `Vec<Option<T>>`. If `T` has a niche, this should be zero-cost.
let mut pos = pos.into_iter().map(Some).collect::<Vec<_>>();
let mut neg = neg.into_iter().map(Some).collect::<Vec<_>>();
// Scan through the lists using two cursors. When either cursor reaches the
// end of its list, there can be no more equal pairs, so stop.
let mut p = 0;
let mut n = 0;
while p < pos.len() && n < neg.len() {
// If the values are equal, remove them and advance both cursors.
// Otherwise, advance whichever cursor points to the lesser value.
// (Choosing which cursor to advance relies on both lists being sorted.)
match pos[p].cmp(&neg[n]) {
Ordering::Less => p += 1,
Ordering::Equal => {
pos[p] = None;
neg[n] = None;
p += 1;
n += 1;
}
Ordering::Greater => n += 1,
}
}
(pos, neg)
}

133
compiler/rustc_mir_transform/src/coverage/counters/balanced_flow.rs Normal file
View file

@ -0,0 +1,133 @@
//! A control-flow graph can be said to have “balanced flow” if the flow
//! (execution count) of each node is equal to the sum of its in-edge flows,
//! and also equal to the sum of its out-edge flows.
//!
//! Control-flow graphs typically have one or more nodes that don't satisfy the
//! balanced-flow property, e.g.:
//! - The start node has out-edges, but no in-edges.
//! - Return nodes have in-edges, but no out-edges.
//! - `Yield` nodes can have an out-flow that is less than their in-flow.
//! - Inescapable loops cause the in-flow/out-flow relationship to break down.
//!
//! Balanced-flow graphs are nevertheless useful for analysis, so this module
//! provides a wrapper type ([`BalancedFlowGraph`]) that imposes balanced flow
//! on an underlying graph. This is done by non-destructively adding synthetic
//! nodes and edges as necessary.
use rustc_data_structures::graph;
use rustc_data_structures::graph::iterate::DepthFirstSearch;
use rustc_data_structures::graph::reversed::ReversedGraph;
use rustc_index::Idx;
use rustc_index::bit_set::DenseBitSet;
use crate::coverage::counters::iter_nodes::IterNodes;
/// A view of an underlying graph that has been augmented to have “balanced flow”.
/// This means that the flow (execution count) of each node is equal to the
/// sum of its in-edge flows, and also equal to the sum of its out-edge flows.
///
/// To achieve this, a synthetic "sink" node is non-destructively added to the
/// graph, with synthetic in-edges from these nodes:
/// - Any node that has no out-edges.
/// - Any node that explicitly requires a sink edge, as indicated by a
/// caller-supplied `force_sink_edge` function.
/// - Any node that would otherwise be unable to reach the sink, because it is
/// part of an inescapable loop.
///
/// To make the graph fully balanced, there is also a synthetic edge from the
/// sink node back to the start node.
///
/// ---
/// The benefit of having a balanced-flow graph is that it can be subsequently
/// transformed in ways that are guaranteed to preserve balanced flow
/// (e.g. merging nodes together), which is useful for discovering relationships
/// between the node flows of different nodes in the graph.
pub(crate) struct BalancedFlowGraph<G: graph::DirectedGraph> {
graph: G,
sink_edge_nodes: DenseBitSet<G::Node>,
pub(crate) sink: G::Node,
}
impl<G: graph::DirectedGraph> BalancedFlowGraph<G> {
/// Creates a balanced view of an underlying graph, by adding a synthetic
/// sink node that has in-edges from nodes that need or request such an edge,
/// and a single out-edge to the start node.
///
/// Assumes that all nodes in the underlying graph are reachable from the
/// start node.
pub(crate) fn for_graph(graph: G, force_sink_edge: impl Fn(G::Node) -> bool) -> Self
where
G: graph::ControlFlowGraph,
{
let mut sink_edge_nodes = DenseBitSet::new_empty(graph.num_nodes());
let mut dfs = DepthFirstSearch::new(ReversedGraph::new(&graph));
// First, determine the set of nodes that explicitly request or require
// an out-edge to the sink.
for node in graph.iter_nodes() {
if force_sink_edge(node) || graph.successors(node).next().is_none() {
sink_edge_nodes.insert(node);
dfs.push_start_node(node);
}
}
// Next, find all nodes that are currently not reverse-reachable from
// `sink_edge_nodes`, and add them to the set as well.
dfs.complete_search();
sink_edge_nodes.union_not(dfs.visited_set());
// The sink node is 1 higher than the highest real node.
let sink = G::Node::new(graph.num_nodes());
BalancedFlowGraph { graph, sink_edge_nodes, sink }
}
}
impl<G> graph::DirectedGraph for BalancedFlowGraph<G>
where
G: graph::DirectedGraph,
{
type Node = G::Node;
/// Returns the number of nodes in this balanced-flow graph, which is 1
/// more than the number of nodes in the underlying graph, to account for
/// the synthetic sink node.
fn num_nodes(&self) -> usize {
// The sink node's index is already the size of the underlying graph,
// so just add 1 to that instead.
self.sink.index() + 1
}
}
impl<G> graph::StartNode for BalancedFlowGraph<G>
where
G: graph::StartNode,
{
fn start_node(&self) -> Self::Node {
self.graph.start_node()
}
}
impl<G> graph::Successors for BalancedFlowGraph<G>
where
G: graph::StartNode + graph::Successors,
{
fn successors(&self, node: Self::Node) -> impl Iterator<Item = Self::Node> {
let real_edges;
let sink_edge;
if node == self.sink {
// The sink node has no real out-edges, and one synthetic out-edge
// to the start node.
real_edges = None;
sink_edge = Some(self.graph.start_node());
} else {
// Real nodes have their real out-edges, and possibly one synthetic
// out-edge to the sink node.
real_edges = Some(self.graph.successors(node));
sink_edge = self.sink_edge_nodes.contains(node).then_some(self.sink);
}
real_edges.into_iter().flatten().chain(sink_edge)
}
}

16
compiler/rustc_mir_transform/src/coverage/counters/iter_nodes.rs Normal file
View file

@ -0,0 +1,16 @@
use rustc_data_structures::graph;
use rustc_index::Idx;
pub(crate) trait IterNodes: graph::DirectedGraph {
/// Iterates over all nodes of a graph in ascending numeric order.
/// Assumes that nodes are densely numbered, i.e. every index in
/// `0..num_nodes` is a valid node.
///
/// FIXME: Can this just be part of [`graph::DirectedGraph`]?
fn iter_nodes(
&self,
) -> impl Iterator<Item = Self::Node> + DoubleEndedIterator + ExactSizeIterator {
(0..self.num_nodes()).map(<Self::Node as Idx>::new)
}
}
impl<G: graph::DirectedGraph> IterNodes for G {}

306
compiler/rustc_mir_transform/src/coverage/counters/node_flow.rs Normal file
View file

@ -0,0 +1,306 @@
//! For each node in a control-flow graph, determines whether that node should
//! have a physical counter, or a counter expression that is derived from the
//! physical counters of other nodes.
//!
//! Based on the algorithm given in
//! "Optimal measurement points for program frequency counts"
//! (Knuth & Stevenson, 1973).
use rustc_data_structures::graph;
use rustc_index::bit_set::DenseBitSet;
use rustc_index::{Idx, IndexVec};
use rustc_middle::mir::coverage::Op;
use smallvec::SmallVec;
use crate::coverage::counters::iter_nodes::IterNodes;
use crate::coverage::counters::union_find::{FrozenUnionFind, UnionFind};
#[cfg(test)]
mod tests;
/// View of some underlying graph, in which each node's successors have been
/// merged into a single "supernode".
///
/// The resulting supernodes have no obvious meaning on their own.
/// However, merging successor nodes means that a node's out-edges can all
/// be combined into a single out-edge, whose flow is the same as the flow
/// (execution count) of its corresponding node in the original graph.
///
/// With all node flows now in the original graph now represented as edge flows
/// in the merged graph, it becomes possible to analyze the original node flows
/// using techniques for analyzing edge flows.
#[derive(Debug)]
pub(crate) struct MergedNodeFlowGraph<Node: Idx> {
/// Maps each node to the supernode that contains it, indicated by some
/// arbitrary "root" node that is part of that supernode.
supernodes: FrozenUnionFind<Node>,
/// For each node, stores the single supernode that all of its successors
/// have been merged into.
///
/// (Note that each node in a supernode can potentially have a _different_
/// successor supernode from its peers.)
succ_supernodes: IndexVec<Node, Node>,
}
impl<Node: Idx> MergedNodeFlowGraph<Node> {
/// Creates a "merged" view of an underlying graph.
///
/// The given graph is assumed to have [“balanced flow”](balanced-flow),
/// though it does not necessarily have to be a `BalancedFlowGraph`.
///
/// [balanced-flow]: `crate::coverage::counters::balanced_flow::BalancedFlowGraph`.
pub(crate) fn for_balanced_graph<G>(graph: G) -> Self
where
G: graph::DirectedGraph<Node = Node> + graph::Successors,
{
let mut supernodes = UnionFind::<G::Node>::new(graph.num_nodes());
// For each node, merge its successors into a single supernode, and
// arbitrarily choose one of those successors to represent all of them.
let successors = graph
.iter_nodes()
.map(|node| {
graph
.successors(node)
.reduce(|a, b| supernodes.unify(a, b))
.expect("each node in a balanced graph must have at least one out-edge")
})
.collect::<IndexVec<G::Node, G::Node>>();
// Now that unification is complete, freeze the supernode forest,
// and resolve each arbitrarily-chosen successor to its canonical root.
// (This avoids having to explicitly resolve them later.)
let supernodes = supernodes.freeze();
let succ_supernodes = successors.into_iter().map(|succ| supernodes.find(succ)).collect();
Self { supernodes, succ_supernodes }
}
fn num_nodes(&self) -> usize {
self.succ_supernodes.len()
}
fn is_supernode(&self, node: Node) -> bool {
self.supernodes.find(node) == node
}
/// Using the information in this merged graph, together with a given
/// permutation of all nodes in the graph, to create physical counters and
/// counter expressions for each node in the underlying graph.
///
/// The given list must contain exactly one copy of each node in the
/// underlying balanced-flow graph. The order of nodes is used as a hint to
/// influence counter allocation:
/// - Earlier nodes are more likely to receive counter expressions.
/// - Later nodes are more likely to receive physical counters.
pub(crate) fn make_node_counters(&self, all_nodes_permutation: &[Node]) -> NodeCounters<Node> {
let mut builder = SpantreeBuilder::new(self);
for &node in all_nodes_permutation {
builder.visit_node(node);
}
NodeCounters { counter_exprs: builder.finish() }
}
}
/// End result of allocating physical counters and counter expressions for the
/// nodes of a graph.
#[derive(Debug)]
pub(crate) struct NodeCounters<Node: Idx> {
counter_exprs: IndexVec<Node, CounterExprVec<Node>>,
}
impl<Node: Idx> NodeCounters<Node> {
/// For the given node, returns the finished list of terms that represent
/// its physical counter or counter expression. Always non-empty.
///
/// If a node was given a physical counter, its "expression" will contain
/// that counter as its sole element.
pub(crate) fn counter_expr(&self, this: Node) -> &[CounterTerm<Node>] {
self.counter_exprs[this].as_slice()
}
}
#[derive(Debug)]
struct SpantreeEdge<Node> {
/// If true, this edge in the spantree has been reversed an odd number of
/// times, so all physical counters added to its node's counter expression
/// need to be negated.
is_reversed: bool,
/// Each spantree edge is "claimed" by the (regular) node that caused it to
/// be created. When a node with a physical counter traverses this edge,
/// that counter is added to the claiming node's counter expression.
claiming_node: Node,
/// Supernode at the other end of this spantree edge. Transitively points
/// to the "root" of this supernode's spantree component.
span_parent: Node,
}
/// Part of a node's counter expression, which is a sum of counter terms.
#[derive(Debug)]
pub(crate) struct CounterTerm<Node> {
/// Whether to add or subtract the value of the node's physical counter.
pub(crate) op: Op,
/// The node whose physical counter is represented by this term.
pub(crate) node: Node,
}
/// Stores the list of counter terms that make up a node's counter expression.
type CounterExprVec<Node> = SmallVec<[CounterTerm<Node>; 2]>;
#[derive(Debug)]
struct SpantreeBuilder<'a, Node: Idx> {
graph: &'a MergedNodeFlowGraph<Node>,
is_unvisited: DenseBitSet<Node>,
/// Links supernodes to each other, gradually forming a spanning tree of
/// the merged-flow graph.
///
/// A supernode without a span edge is the root of its component of the
/// spantree. Nodes that aren't supernodes cannot have a spantree edge.
span_edges: IndexVec<Node, Option<SpantreeEdge<Node>>>,
/// Shared path buffer recycled by all calls to `yank_to_spantree_root`.
yank_buffer: Vec<Node>,
/// An in-progress counter expression for each node. Each expression is
/// initially empty, and will be filled in as relevant nodes are visited.
counter_exprs: IndexVec<Node, CounterExprVec<Node>>,
}
impl<'a, Node: Idx> SpantreeBuilder<'a, Node> {
fn new(graph: &'a MergedNodeFlowGraph<Node>) -> Self {
let num_nodes = graph.num_nodes();
Self {
graph,
is_unvisited: DenseBitSet::new_filled(num_nodes),
span_edges: IndexVec::from_fn_n(|_| None, num_nodes),
yank_buffer: vec![],
counter_exprs: IndexVec::from_fn_n(|_| SmallVec::new(), num_nodes),
}
}
/// Given a supernode, finds the supernode that is the "root" of its
/// spantree component. Two nodes that have the same spantree root are
/// connected in the spantree.
fn spantree_root(&self, this: Node) -> Node {
debug_assert!(self.graph.is_supernode(this));
match self.span_edges[this] {
None => this,
Some(SpantreeEdge { span_parent, .. }) => self.spantree_root(span_parent),
}
}
/// Rotates edges in the spantree so that `this` is the root of its
/// spantree component.
fn yank_to_spantree_root(&mut self, this: Node) {
debug_assert!(self.graph.is_supernode(this));
// The rotation is done iteratively, by first traversing from `this` to
// its root and storing the path in a buffer, and then traversing the
// path buffer backwards to reverse all the edges.
// Recycle the same path buffer for all calls to this method.
let path_buf = &mut self.yank_buffer;
path_buf.clear();
path_buf.push(this);
// Traverse the spantree until we reach a supernode that has no
// span-parent, which must be the root.
let mut curr = this;
while let &Some(SpantreeEdge { span_parent, .. }) = &self.span_edges[curr] {
path_buf.push(span_parent);
curr = span_parent;
}
// For each spantree edge `a -> b` in the path that was just traversed,
// reverse it to become `a <- b`, while preserving `claiming_node`.
for &[a, b] in path_buf.array_windows::<2>().rev() {
let SpantreeEdge { is_reversed, claiming_node, span_parent } = self.span_edges[a]
.take()
.expect("all nodes in the path (except the last) have a `span_parent`");
debug_assert_eq!(span_parent, b);
debug_assert!(self.span_edges[b].is_none());
self.span_edges[b] =
Some(SpantreeEdge { is_reversed: !is_reversed, claiming_node, span_parent: a });
}
// The result of the rotation is that `this` is now a spantree root.
debug_assert!(self.span_edges[this].is_none());
}
/// Must be called exactly once for each node in the balanced-flow graph.
fn visit_node(&mut self, this: Node) {
// Assert that this node was unvisited, and mark it visited.
assert!(self.is_unvisited.remove(this), "node has already been visited: {this:?}");
// Get the supernode containing `this`, and make it the root of its
// component of the spantree.
let this_supernode = self.graph.supernodes.find(this);
self.yank_to_spantree_root(this_supernode);
// Get the supernode containing all of this's successors.
let succ_supernode = self.graph.succ_supernodes[this];
debug_assert!(self.graph.is_supernode(succ_supernode));
// If two supernodes are already connected in the spantree, they will
// have the same spantree root. (Each supernode is connected to itself.)
if this_supernode != self.spantree_root(succ_supernode) {
// Adding this node's flow edge to the spantree would cause two
// previously-disconnected supernodes to become connected, so add
// it. That spantree-edge is now "claimed" by this node.
//
// Claiming a spantree-edge means that this node will get a counter
// expression instead of a physical counter. That expression is
// currently empty, but will be built incrementally as the other
// nodes are visited.
self.span_edges[this_supernode] = Some(SpantreeEdge {
is_reversed: false,
claiming_node: this,
span_parent: succ_supernode,
});
} else {
// This node's flow edge would join two supernodes that are already
// connected in the spantree (or are the same supernode). That would
// create a cycle in the spantree, so don't add an edge.
//
// Instead, create a physical counter for this node, and add that
// counter to all expressions on the path from `succ_supernode` to
// `this_supernode`.
// Instead of setting `this.measure = true` as in the original paper,
// we just add the node's ID to its own "expression".
self.counter_exprs[this].push(CounterTerm { node: this, op: Op::Add });
// Walk the spantree from `this.successor` back to `this`. For each
// spantree edge along the way, add this node's physical counter to
// the counter expression of the node that claimed the spantree edge.
let mut curr = succ_supernode;
while curr != this_supernode {
let &SpantreeEdge { is_reversed, claiming_node, span_parent } =
self.span_edges[curr].as_ref().unwrap();
let op = if is_reversed { Op::Subtract } else { Op::Add };
self.counter_exprs[claiming_node].push(CounterTerm { node: this, op });
curr = span_parent;
}
}
}
/// Asserts that all nodes have been visited, and returns the computed
/// counter expressions (made up of physical counters) for each node.
fn finish(self) -> IndexVec<Node, CounterExprVec<Node>> {
let Self { graph, is_unvisited, span_edges, yank_buffer: _, counter_exprs } = self;
assert!(is_unvisited.is_empty(), "some nodes were never visited: {is_unvisited:?}");
debug_assert!(
span_edges
.iter_enumerated()
.all(|(node, span_edge)| { span_edge.is_some() <= graph.is_supernode(node) }),
"only supernodes can have a span edge",
);
debug_assert!(
counter_exprs.iter().all(|expr| !expr.is_empty()),
"after visiting all nodes, every node should have a non-empty expression",
);
counter_exprs
}
}

64
compiler/rustc_mir_transform/src/coverage/counters/node_flow/tests.rs Normal file
View file

@ -0,0 +1,64 @@
use itertools::Itertools;
use rustc_data_structures::graph;
use rustc_data_structures::graph::vec_graph::VecGraph;
use rustc_index::Idx;
use rustc_middle::mir::coverage::Op;
use super::{CounterTerm, MergedNodeFlowGraph, NodeCounters};
fn merged_node_flow_graph<G: graph::Successors>(graph: G) -> MergedNodeFlowGraph<G::Node> {
MergedNodeFlowGraph::for_balanced_graph(graph)
}
fn make_graph<Node: Idx + Ord>(num_nodes: usize, edge_pairs: Vec<(Node, Node)>) -> VecGraph<Node> {
VecGraph::new(num_nodes, edge_pairs)
}
/// Example used in "Optimal Measurement Points for Program Frequency Counts"
/// (Knuth & Stevenson, 1973), but with 0-based node IDs.
#[test]
fn example_driver() {
let graph = make_graph::<u32>(5, vec![
(0, 1),
(0, 3),
(1, 0),
(1, 2),
(2, 1),
(2, 4),
(3, 3),
(3, 4),
(4, 0),
]);
let merged = merged_node_flow_graph(&graph);
let counters = merged.make_node_counters(&[3, 1, 2, 0, 4]);
assert_eq!(format_counter_expressions(&counters), &[
// (comment to force vertical formatting for clarity)
"[0]: +c0",
"[1]: +c0 +c2 -c4",
"[2]: +c2",
"[3]: +c3",
"[4]: +c4",
]);
}
fn format_counter_expressions<Node: Idx>(counters: &NodeCounters<Node>) -> Vec<String> {
let format_item = |&CounterTerm { node, op }| {
let op = match op {
Op::Subtract => '-',
Op::Add => '+',
};
format!("{op}c{node:?}")
};
counters
.counter_exprs
.indices()
.map(|node| {
let mut expr = counters.counter_expr(node).iter().collect::<Vec<_>>();
expr.sort_by_key(|item| item.node.index());
format!("[{node:?}]: {}", expr.into_iter().map(format_item).join(" "))
})
.collect()
}

41
compiler/rustc_mir_transform/src/coverage/counters/tests.rs
View file

@ -1,41 +0,0 @@
use std::fmt::Debug;
use super::sort_and_cancel;
fn flatten<T>(input: Vec<Option<T>>) -> Vec<T> {
input.into_iter().flatten().collect()
}
fn sort_and_cancel_and_flatten<T: Clone + Ord>(pos: Vec<T>, neg: Vec<T>) -> (Vec<T>, Vec<T>) {
let (pos_actual, neg_actual) = sort_and_cancel(pos, neg);
(flatten(pos_actual), flatten(neg_actual))
}
#[track_caller]
fn check_test_case<T: Clone + Debug + Ord>(
pos: Vec<T>,
neg: Vec<T>,
pos_expected: Vec<T>,
neg_expected: Vec<T>,
) {
eprintln!("pos = {pos:?}; neg = {neg:?}");
let output = sort_and_cancel_and_flatten(pos, neg);
assert_eq!(output, (pos_expected, neg_expected));
}
#[test]
fn cancellation() {
let cases: &[(Vec<u32>, Vec<u32>, Vec<u32>, Vec<u32>)] = &[
(vec![], vec![], vec![], vec![]),
(vec![4, 2, 1, 5, 3], vec![], vec![1, 2, 3, 4, 5], vec![]),
(vec![5, 5, 5, 5, 5], vec![5], vec![5, 5, 5, 5], vec![]),
(vec![1, 1, 2, 2, 3, 3], vec![1, 2, 3], vec![1, 2, 3], vec![]),
(vec![1, 1, 2, 2, 3, 3], vec![2, 4, 2], vec![1, 1, 3, 3], vec![4]),
];
for (pos, neg, pos_expected, neg_expected) in cases {
check_test_case(pos.to_vec(), neg.to_vec(), pos_expected.to_vec(), neg_expected.to_vec());
// Same test case, but with its inputs flipped and its outputs flipped.
check_test_case(neg.to_vec(), pos.to_vec(), neg_expected.to_vec(), pos_expected.to_vec());
}
}

116
compiler/rustc_mir_transform/src/coverage/counters/union_find.rs Normal file
View file

@ -0,0 +1,116 @@
use std::cmp::Ordering;
use std::mem;
use rustc_index::{Idx, IndexVec};
#[cfg(test)]
mod tests;
/// Simple implementation of a union-find data structure, i.e. a disjoint-set
/// forest.
#[derive(Debug)]
pub(crate) struct UnionFind<Key: Idx> {
table: IndexVec<Key, UnionFindEntry<Key>>,
}
#[derive(Debug)]
struct UnionFindEntry<Key> {
/// Transitively points towards the "root" of the set containing this key.
///
/// Invariant: A root key is its own parent.
parent: Key,
/// When merging two "root" keys, their ranks determine which key becomes
/// the new root, to prevent the parent tree from becoming unnecessarily
/// tall. See [`UnionFind::unify`] for details.
rank: u32,
}
impl<Key: Idx> UnionFind<Key> {
/// Creates a new disjoint-set forest containing the keys `0..num_keys`.
/// Initially, every key is part of its own one-element set.
pub(crate) fn new(num_keys: usize) -> Self {
// Initially, every key is the root of its own set, so its parent is itself.
Self { table: IndexVec::from_fn_n(|key| UnionFindEntry { parent: key, rank: 0 }, num_keys) }
}
/// Returns the "root" key of the disjoint-set containing the given key.
/// If two keys have the same root, they belong to the same set.
///
/// Also updates internal data structures to make subsequent `find`
/// operations faster.
pub(crate) fn find(&mut self, key: Key) -> Key {
// Loop until we find a key that is its own parent.
let mut curr = key;
while let parent = self.table[curr].parent
&& curr != parent
{
// Perform "path compression" by peeking one layer ahead, and
// setting the current key's parent to that value.
// (This works even when `parent` is the root of its set, because
// of the invariant that a root is its own parent.)
let parent_parent = self.table[parent].parent;
self.table[curr].parent = parent_parent;
// Advance by one step and continue.
curr = parent;
}
curr
}
/// Merges the set containing `a` and the set containing `b` into one set.
///
/// Returns the common root of both keys, after the merge.
pub(crate) fn unify(&mut self, a: Key, b: Key) -> Key {
let mut a = self.find(a);
let mut b = self.find(b);
// If both keys have the same root, they're already in the same set,
// so there's nothing more to do.
if a == b {
return a;
};
// Ensure that `a` has strictly greater rank, swapping if necessary.
// If both keys have the same rank, increment the rank of `a` so that
// future unifications will also prefer `a`, leading to flatter trees.
match Ord::cmp(&self.table[a].rank, &self.table[b].rank) {
Ordering::Less => mem::swap(&mut a, &mut b),
Ordering::Equal => self.table[a].rank += 1,
Ordering::Greater => {}
}
debug_assert!(self.table[a].rank > self.table[b].rank);
debug_assert_eq!(self.table[b].parent, b);
// Make `a` the parent of `b`.
self.table[b].parent = a;
a
}
/// Creates a snapshot of this disjoint-set forest that can no longer be
/// mutated, but can be queried without mutation.
pub(crate) fn freeze(&mut self) -> FrozenUnionFind<Key> {
// Just resolve each key to its actual root.
let roots = self.table.indices().map(|key| self.find(key)).collect();
FrozenUnionFind { roots }
}
}
/// Snapshot of a disjoint-set forest that can no longer be mutated, but can be
/// queried in O(1) time without mutation.
///
/// This is really just a wrapper around a direct mapping from keys to roots,
/// but with a [`Self::find`] method that resembles [`UnionFind::find`].
#[derive(Debug)]
pub(crate) struct FrozenUnionFind<Key: Idx> {
roots: IndexVec<Key, Key>,
}
impl<Key: Idx> FrozenUnionFind<Key> {
/// Returns the "root" key of the disjoint-set containing the given key.
/// If two keys have the same root, they belong to the same set.
pub(crate) fn find(&self, key: Key) -> Key {
self.roots[key]
}
}

32
compiler/rustc_mir_transform/src/coverage/counters/union_find/tests.rs Normal file
View file

@ -0,0 +1,32 @@
use super::UnionFind;
#[test]
fn empty() {
let mut sets = UnionFind::<u32>::new(10);
for i in 1..10 {
assert_eq!(sets.find(i), i);
}
}
#[test]
fn transitive() {
let mut sets = UnionFind::<u32>::new(10);
sets.unify(3, 7);
sets.unify(4, 2);
assert_eq!(sets.find(7), sets.find(3));
assert_eq!(sets.find(2), sets.find(4));
assert_ne!(sets.find(3), sets.find(4));
sets.unify(7, 4);
assert_eq!(sets.find(7), sets.find(3));
assert_eq!(sets.find(2), sets.find(4));
assert_eq!(sets.find(3), sets.find(4));
for i in [0, 1, 5, 6, 8, 9] {
assert_eq!(sets.find(i), i);
}
}

174
compiler/rustc_mir_transform/src/coverage/graph.rs
View file

@ -1,7 +1,6 @@
use std::cmp::Ordering;
use std::collections::VecDeque;
use std::ops::{Index, IndexMut};
use std::{iter, mem, slice};
use std::{mem, slice};
use rustc_data_structures::captures::Captures;
use rustc_data_structures::fx::FxHashSet;
@ -211,54 +210,6 @@ impl CoverageGraph {
self.dominator_order_rank[a].cmp(&self.dominator_order_rank[b])
}
/// Returns the source of this node's sole in-edge, if it has exactly one.
/// That edge can be assumed to have the same execution count as the node
/// itself (in the absence of panics).
pub(crate) fn sole_predecessor(
&self,
to_bcb: BasicCoverageBlock,
) -> Option<BasicCoverageBlock> {
// Unlike `simple_successor`, there is no need for extra checks here.
if let &[from_bcb] = self.predecessors[to_bcb].as_slice() { Some(from_bcb) } else { None }
}
/// Returns the target of this node's sole out-edge, if it has exactly
/// one, but only if that edge can be assumed to have the same execution
/// count as the node itself (in the absence of panics).
pub(crate) fn simple_successor(
&self,
from_bcb: BasicCoverageBlock,
) -> Option<BasicCoverageBlock> {
// If a node's count is the sum of its out-edges, and it has exactly
// one out-edge, then that edge has the same count as the node.
if self.bcbs[from_bcb].is_out_summable
&& let &[to_bcb] = self.successors[from_bcb].as_slice()
{
Some(to_bcb)
} else {
None
}
}
/// For each loop that contains the given node, yields the "loop header"
/// node representing that loop, from innermost to outermost. If the given
/// node is itself a loop header, it is yielded first.
pub(crate) fn loop_headers_containing(
&self,
bcb: BasicCoverageBlock,
) -> impl Iterator<Item = BasicCoverageBlock> + Captures<'_> {
let self_if_loop_header = self.is_loop_header.contains(bcb).then_some(bcb).into_iter();
let mut curr = Some(bcb);
let strictly_enclosing = iter::from_fn(move || {
let enclosing = self.enclosing_loop_header[curr?];
curr = enclosing;
enclosing
});
self_if_loop_header.chain(strictly_enclosing)
}
/// For the given node, yields the subset of its predecessor nodes that
/// it dominates. If that subset is non-empty, the node is a "loop header",
/// and each of those predecessors represents an in-edge that jumps back to
@ -489,126 +440,3 @@ impl<'a, 'tcx> graph::Successors for CoverageRelevantSubgraph<'a, 'tcx> {
self.coverage_successors(bb).into_iter()
}
}
/// State of a node in the coverage graph during ready-first traversal.
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)]
enum ReadyState {
/// This node has not yet been added to the fallback queue or ready queue.
Unqueued,
/// This node is currently in the fallback queue.
InFallbackQueue,
/// This node's predecessors have all been visited, so it is in the ready queue.
/// (It might also have a stale entry in the fallback queue.)
InReadyQueue,
/// This node has been visited.
/// (It might also have a stale entry in the fallback queue.)
Visited,
}
/// Iterator that visits nodes in the coverage graph, in an order that always
/// prefers "ready" nodes whose predecessors have already been visited.
pub(crate) struct ReadyFirstTraversal<'a> {
graph: &'a CoverageGraph,
/// For each node, the number of its predecessor nodes that haven't been visited yet.
n_unvisited_preds: IndexVec<BasicCoverageBlock, u32>,
/// Indicates whether a node has been visited, or which queue it is in.
state: IndexVec<BasicCoverageBlock, ReadyState>,
/// Holds unvisited nodes whose predecessors have all been visited.
ready_queue: VecDeque<BasicCoverageBlock>,
/// Holds unvisited nodes with some unvisited predecessors.
/// Also contains stale entries for nodes that were upgraded to ready.
fallback_queue: VecDeque<BasicCoverageBlock>,
}
impl<'a> ReadyFirstTraversal<'a> {
pub(crate) fn new(graph: &'a CoverageGraph) -> Self {
let num_nodes = graph.num_nodes();
let n_unvisited_preds =
IndexVec::from_fn_n(|node| graph.predecessors[node].len() as u32, num_nodes);
let mut state = IndexVec::from_elem_n(ReadyState::Unqueued, num_nodes);
// We know from coverage graph construction that the start node is the
// only node with no predecessors.
debug_assert!(
n_unvisited_preds.iter_enumerated().all(|(node, &n)| (node == START_BCB) == (n == 0))
);
let ready_queue = VecDeque::from(vec![START_BCB]);
state[START_BCB] = ReadyState::InReadyQueue;
Self { graph, state, n_unvisited_preds, ready_queue, fallback_queue: VecDeque::new() }
}
/// Returns the next node from the ready queue, or else the next unvisited
/// node from the fallback queue.
fn next_inner(&mut self) -> Option<BasicCoverageBlock> {
// Always prefer to yield a ready node if possible.
if let Some(node) = self.ready_queue.pop_front() {
assert_eq!(self.state[node], ReadyState::InReadyQueue);
return Some(node);
}
while let Some(node) = self.fallback_queue.pop_front() {
match self.state[node] {
// This entry in the fallback queue is not stale, so yield it.
ReadyState::InFallbackQueue => return Some(node),
// This node was added to the fallback queue, but later became
// ready and was visited via the ready queue. Ignore it here.
ReadyState::Visited => {}
// Unqueued nodes can't be in the fallback queue, by definition.
// We know that the ready queue is empty at this point.
ReadyState::Unqueued | ReadyState::InReadyQueue => unreachable!(
"unexpected state for {node:?} in the fallback queue: {:?}",
self.state[node]
),
}
}
None
}
fn mark_visited_and_enqueue_successors(&mut self, node: BasicCoverageBlock) {
assert!(self.state[node] < ReadyState::Visited);
self.state[node] = ReadyState::Visited;
// For each of this node's successors, decrease the successor's
// "unvisited predecessors" count, and enqueue it if appropriate.
for &succ in &self.graph.successors[node] {
let is_unqueued = match self.state[succ] {
ReadyState::Unqueued => true,
ReadyState::InFallbackQueue => false,
ReadyState::InReadyQueue => {
unreachable!("nodes in the ready queue have no unvisited predecessors")
}
// The successor was already visited via one of its other predecessors.
ReadyState::Visited => continue,
};
self.n_unvisited_preds[succ] -= 1;
if self.n_unvisited_preds[succ] == 0 {
// This node's predecessors have all been visited, so add it to
// the ready queue. If it's already in the fallback queue, that
// fallback entry will be ignored later.
self.state[succ] = ReadyState::InReadyQueue;
self.ready_queue.push_back(succ);
} else if is_unqueued {
// This node has unvisited predecessors, so add it to the
// fallback queue in case we run out of ready nodes later.
self.state[succ] = ReadyState::InFallbackQueue;
self.fallback_queue.push_back(succ);
}
}
}
}
impl<'a> Iterator for ReadyFirstTraversal<'a> {
type Item = BasicCoverageBlock;
fn next(&mut self) -> Option<Self::Item> {
let node = self.next_inner()?;
self.mark_visited_and_enqueue_successors(node);
Some(node)
}
}

58
compiler/rustc_mir_transform/src/coverage/mod.rs
View file

@ -15,16 +15,13 @@ use rustc_middle::hir::nested_filter;
use rustc_middle::mir::coverage::{
CoverageKind, DecisionInfo, FunctionCoverageInfo, Mapping, MappingKind,
};
use rustc_middle::mir::{
self, BasicBlock, BasicBlockData, SourceInfo, Statement, StatementKind, Terminator,
TerminatorKind,
};
use rustc_middle::mir::{self, BasicBlock, Statement, StatementKind, TerminatorKind};
use rustc_middle::ty::TyCtxt;
use rustc_span::Span;
use rustc_span::def_id::LocalDefId;
use tracing::{debug, debug_span, trace};
use crate::coverage::counters::{CoverageCounters, Site};
use crate::coverage::counters::CoverageCounters;
use crate::coverage::graph::CoverageGraph;
use crate::coverage::mappings::ExtractedMappings;
@ -92,8 +89,7 @@ fn instrument_function_for_coverage<'tcx>(tcx: TyCtxt<'tcx>, mir_body: &mut mir:
return;
}
let coverage_counters =
CoverageCounters::make_bcb_counters(&graph, &bcbs_with_counter_mappings);
let coverage_counters = counters::make_bcb_counters(&graph, &bcbs_with_counter_mappings);
let mappings = create_mappings(&extracted_mappings, &coverage_counters);
if mappings.is_empty() {
@ -242,27 +238,8 @@ fn inject_coverage_statements<'tcx>(
coverage_counters: &CoverageCounters,
) {
// Inject counter-increment statements into MIR.
for (id, site) in coverage_counters.counter_increment_sites() {
// Determine the block to inject a counter-increment statement into.
// For BCB nodes this is just their first block, but for edges we need
// to create a new block between the two BCBs, and inject into that.
let target_bb = match site {
Site::Node { bcb } => graph[bcb].leader_bb(),
Site::Edge { from_bcb, to_bcb } => {
// Create a new block between the last block of `from_bcb` and
// the first block of `to_bcb`.
let from_bb = graph[from_bcb].last_bb();
let to_bb = graph[to_bcb].leader_bb();
let new_bb = inject_edge_counter_basic_block(mir_body, from_bb, to_bb);
debug!(
"Edge {from_bcb:?} (last {from_bb:?}) -> {to_bcb:?} (leader {to_bb:?}) \
requires a new MIR BasicBlock {new_bb:?} for counter increment {id:?}",
);
new_bb
}
};
for (id, bcb) in coverage_counters.counter_increment_sites() {
let target_bb = graph[bcb].leader_bb();
inject_statement(mir_body, CoverageKind::CounterIncrement { id }, target_bb);
}
@ -335,31 +312,6 @@ fn inject_mcdc_statements<'tcx>(
}
}
/// Given two basic blocks that have a control-flow edge between them, creates
/// and returns a new block that sits between those blocks.
fn inject_edge_counter_basic_block(
mir_body: &mut mir::Body<'_>,
from_bb: BasicBlock,
to_bb: BasicBlock,
) -> BasicBlock {
let span = mir_body[from_bb].terminator().source_info.span.shrink_to_hi();
let new_bb = mir_body.basic_blocks_mut().push(BasicBlockData {
statements: vec![], // counter will be injected here
terminator: Some(Terminator {
source_info: SourceInfo::outermost(span),
kind: TerminatorKind::Goto { target: to_bb },
}),
is_cleanup: false,
});
let edge_ref = mir_body[from_bb]
.terminator_mut()
.successors_mut()
.find(|successor| **successor == to_bb)
.expect("from_bb should have a successor for to_bb");
*edge_ref = new_bb;
new_bb
}
fn inject_statement(mir_body: &mut mir::Body<'_>, counter_kind: CoverageKind, bb: BasicBlock) {
debug!(" injecting statement {counter_kind:?} for {bb:?}");
let data = &mut mir_body[bb];

1
compiler/rustc_mir_transform/src/lib.rs
View file

@ -1,4 +1,5 @@
// tidy-alphabetical-start
#![feature(array_windows)]
#![feature(assert_matches)]
#![feature(box_patterns)]
#![feature(const_type_name)]

22
compiler/rustc_monomorphize/src/collector.rs
View file

@ -1410,19 +1410,33 @@ impl<'v> RootCollector<'_, 'v> {
&& !self.tcx.generics_of(id.owner_id).requires_monomorphization(self.tcx)
{
debug!("RootCollector: ADT drop-glue for `{id:?}`",);
let id_args =
ty::GenericArgs::for_item(self.tcx, id.owner_id.to_def_id(), |param, _| {
match param.kind {
GenericParamDefKind::Lifetime => {
self.tcx.lifetimes.re_erased.into()
}
GenericParamDefKind::Type { .. }
| GenericParamDefKind::Const { .. } => {
unreachable!(
"`own_requires_monomorphization` check means that \
we should have no type/const params"
)
}
}
});
// This type is impossible to instantiate, so we should not try to
// generate a `drop_in_place` instance for it.
if self.tcx.instantiate_and_check_impossible_predicates((
id.owner_id.to_def_id(),
ty::List::empty(),
id_args,
)) {
return;
}
let ty = self.tcx.erase_regions(
self.tcx.type_of(id.owner_id.to_def_id()).instantiate_identity(),
);
let ty =
self.tcx.type_of(id.owner_id.to_def_id()).instantiate(self.tcx, id_args);
assert!(!ty.has_non_region_param());
visit_drop_use(self.tcx, ty, true, DUMMY_SP, self.output);
}

1
compiler/rustc_next_trait_solver/Cargo.toml
View file

@ -13,6 +13,7 @@ rustc_macros = { path = "../rustc_macros", optional = true }
rustc_serialize = { path = "../rustc_serialize", optional = true }
rustc_type_ir = { path = "../rustc_type_ir", default-features = false }
rustc_type_ir_macros = { path = "../rustc_type_ir_macros" }
smallvec = "1.8.1"
tracing = "0.1"
# tidy-alphabetical-end

36
compiler/rustc_next_trait_solver/src/solve/trait_goals.rs
View file

@ -8,6 +8,7 @@ use rustc_type_ir::lang_items::TraitSolverLangItem;
use rustc_type_ir::solve::CanonicalResponse;
use rustc_type_ir::visit::TypeVisitableExt as _;
use rustc_type_ir::{self as ty, Interner, TraitPredicate, TypingMode, Upcast as _, elaborate};
use smallvec::SmallVec;
use tracing::{instrument, trace};
use crate::delegate::SolverDelegate;
@ -225,7 +226,7 @@ where
}
ecx.probe_and_evaluate_goal_for_constituent_tys(
CandidateSource::BuiltinImpl(BuiltinImplSource::Misc),
CandidateSource::BuiltinImpl(BuiltinImplSource::Trivial),
goal,
structural_traits::instantiate_constituent_tys_for_sized_trait,
)
@ -741,12 +742,14 @@ where
a_data.principal(),
));
} else if let Some(a_principal) = a_data.principal() {
for new_a_principal in
elaborate::supertraits(self.cx(), a_principal.with_self_ty(cx, a_ty)).skip(1)
for (idx, new_a_principal) in
elaborate::supertraits(self.cx(), a_principal.with_self_ty(cx, a_ty))
.enumerate()
.skip(1)
{
responses.extend(self.consider_builtin_upcast_to_principal(
goal,
CandidateSource::BuiltinImpl(BuiltinImplSource::TraitUpcasting),
CandidateSource::BuiltinImpl(BuiltinImplSource::TraitUpcasting(idx)),
a_data,
a_region,
b_data,
@ -1192,7 +1195,30 @@ where
};
}
// FIXME: prefer trivial builtin impls
// We prefer trivial builtin candidates, i.e. builtin impls without any
// nested requirements, over all others. This is a fix for #53123 and
// prevents where-bounds from accidentally extending the lifetime of a
// variable.
if candidates
.iter()
.any(|c| matches!(c.source, CandidateSource::BuiltinImpl(BuiltinImplSource::Trivial)))
{
let trivial_builtin_impls: SmallVec<[_; 1]> = candidates
.iter()
.filter(|c| {
matches!(c.source, CandidateSource::BuiltinImpl(BuiltinImplSource::Trivial))
})
.map(|c| c.result)
.collect();
// There should only ever be a single trivial builtin candidate
// as they would otherwise overlap.
assert_eq!(trivial_builtin_impls.len(), 1);
return if let Some(response) = self.try_merge_responses(&trivial_builtin_impls) {
Ok((response, Some(TraitGoalProvenVia::Misc)))
} else {
Ok((self.bail_with_ambiguity(&trivial_builtin_impls), None))
};
}
// If there are non-global where-bounds, prefer where-bounds
// (including global ones) over everything else.

172
compiler/rustc_passes/src/stability.rs
View file

@ -5,8 +5,8 @@ use std::mem::replace;
use std::num::NonZero;
use rustc_attr_parsing::{
self as attr, ConstStability, DeprecatedSince, Stability, StabilityLevel, StableSince,
UnstableReason, VERSION_PLACEHOLDER,
self as attr, AllowedThroughUnstableModules, ConstStability, DeprecatedSince, Stability,
StabilityLevel, StableSince, UnstableReason, VERSION_PLACEHOLDER,
};
use rustc_data_structures::fx::FxIndexMap;
use rustc_data_structures::unord::{ExtendUnord, UnordMap, UnordSet};
@ -20,11 +20,16 @@ use rustc_hir::{FieldDef, Item, ItemKind, TraitRef, Ty, TyKind, Variant};
use rustc_middle::hir::nested_filter;
use rustc_middle::middle::lib_features::{FeatureStability, LibFeatures};
use rustc_middle::middle::privacy::EffectiveVisibilities;
use rustc_middle::middle::stability::{AllowUnstable, DeprecationEntry, Index};
use rustc_middle::middle::stability::{
AllowUnstable, Deprecated, DeprecationEntry, EvalResult, Index,
};
use rustc_middle::query::Providers;
use rustc_middle::ty::TyCtxt;
use rustc_middle::ty::print::with_no_trimmed_paths;
use rustc_session::lint;
use rustc_session::lint::builtin::{INEFFECTIVE_UNSTABLE_TRAIT_IMPL, USELESS_DEPRECATED};
use rustc_session::lint::builtin::{
DEPRECATED, INEFFECTIVE_UNSTABLE_TRAIT_IMPL, USELESS_DEPRECATED,
};
use rustc_span::{Span, Symbol, sym};
use tracing::{debug, info};
@ -593,9 +598,11 @@ impl<'tcx> MissingStabilityAnnotations<'tcx> {
}
fn check_missing_const_stability(&self, def_id: LocalDefId, span: Span) {
let is_const = self.tcx.is_const_fn(def_id.to_def_id());
let is_const = self.tcx.is_const_fn(def_id.to_def_id())
|| (self.tcx.def_kind(def_id.to_def_id()) == DefKind::Trait
&& self.tcx.is_const_trait(def_id.to_def_id()));
// Reachable const fn must have a stability attribute.
// Reachable const fn/trait must have a stability attribute.
if is_const
&& self.effective_visibilities.is_reachable(def_id)
&& self.tcx.lookup_const_stability(def_id).is_none()
@ -772,7 +779,13 @@ impl<'tcx> Visitor<'tcx> for Checker<'tcx> {
// For implementations of traits, check the stability of each item
// individually as it's possible to have a stable trait with unstable
// items.
hir::ItemKind::Impl(hir::Impl { of_trait: Some(ref t), self_ty, items, .. }) => {
hir::ItemKind::Impl(hir::Impl {
of_trait: Some(ref t),
self_ty,
items,
constness,
..
}) => {
let features = self.tcx.features();
if features.staged_api() {
let attrs = self.tcx.hir().attrs(item.hir_id());
@ -814,6 +827,16 @@ impl<'tcx> Visitor<'tcx> for Checker<'tcx> {
}
}
match constness {
rustc_hir::Constness::Const => {
if let Some(def_id) = t.trait_def_id() {
// FIXME(const_trait_impl): Improve the span here.
self.tcx.check_const_stability(def_id, t.path.span, t.path.span);
}
}
rustc_hir::Constness::NotConst => {}
}
for impl_item_ref in *items {
let impl_item = self.tcx.associated_item(impl_item_ref.id.owner_id);
@ -829,6 +852,18 @@ impl<'tcx> Visitor<'tcx> for Checker<'tcx> {
intravisit::walk_item(self, item);
}
fn visit_poly_trait_ref(&mut self, t: &'tcx hir::PolyTraitRef<'tcx>) {
match t.modifiers.constness {
hir::BoundConstness::Always(span) | hir::BoundConstness::Maybe(span) => {
if let Some(def_id) = t.trait_ref.trait_def_id() {
self.tcx.check_const_stability(def_id, t.trait_ref.path.span, span);
}
}
hir::BoundConstness::Never => {}
}
intravisit::walk_poly_trait_ref(self, t);
}
fn visit_path(&mut self, path: &hir::Path<'tcx>, id: hir::HirId) {
if let Some(def_id) = path.res.opt_def_id() {
let method_span = path.segments.last().map(|s| s.ident.span);
@ -844,42 +879,95 @@ impl<'tcx> Visitor<'tcx> for Checker<'tcx> {
},
);
let is_allowed_through_unstable_modules = |def_id| {
self.tcx.lookup_stability(def_id).is_some_and(|stab| match stab.level {
StabilityLevel::Stable { allowed_through_unstable_modules, .. } => {
allowed_through_unstable_modules
}
_ => false,
})
};
if item_is_allowed {
// The item itself is allowed; check whether the path there is also allowed.
let is_allowed_through_unstable_modules: Option<AllowedThroughUnstableModules> =
self.tcx.lookup_stability(def_id).and_then(|stab| match stab.level {
StabilityLevel::Stable { allowed_through_unstable_modules, .. } => {
allowed_through_unstable_modules
}
_ => None,
});
if item_is_allowed && !is_allowed_through_unstable_modules(def_id) {
// Check parent modules stability as well if the item the path refers to is itself
// stable. We only emit warnings for unstable path segments if the item is stable
// or allowed because stability is often inherited, so the most common case is that
// both the segments and the item are unstable behind the same feature flag.
//
// We check here rather than in `visit_path_segment` to prevent visiting the last
// path segment twice
//
// We include special cases via #[rustc_allowed_through_unstable_modules] for items
// that were accidentally stabilized through unstable paths before this check was
// added, such as `core::intrinsics::transmute`
let parents = path.segments.iter().rev().skip(1);
for path_segment in parents {
if let Some(def_id) = path_segment.res.opt_def_id() {
// use `None` for id to prevent deprecation check
self.tcx.check_stability_allow_unstable(
def_id,
None,
path.span,
None,
if is_unstable_reexport(self.tcx, id) {
AllowUnstable::Yes
} else {
AllowUnstable::No
},
);
if is_allowed_through_unstable_modules.is_none() {
// Check parent modules stability as well if the item the path refers to is itself
// stable. We only emit warnings for unstable path segments if the item is stable
// or allowed because stability is often inherited, so the most common case is that
// both the segments and the item are unstable behind the same feature flag.
//
// We check here rather than in `visit_path_segment` to prevent visiting the last
// path segment twice
//
// We include special cases via #[rustc_allowed_through_unstable_modules] for items
// that were accidentally stabilized through unstable paths before this check was
// added, such as `core::intrinsics::transmute`
let parents = path.segments.iter().rev().skip(1);
for path_segment in parents {
if let Some(def_id) = path_segment.res.opt_def_id() {
// use `None` for id to prevent deprecation check
self.tcx.check_stability_allow_unstable(
def_id,
None,
path.span,
None,
if is_unstable_reexport(self.tcx, id) {
AllowUnstable::Yes
} else {
AllowUnstable::No
},
);
}
}
} else if let Some(AllowedThroughUnstableModules::WithDeprecation(deprecation)) =
is_allowed_through_unstable_modules
{
// Similar to above, but we cannot use `check_stability_allow_unstable` as that would
// immediately show the stability error. We just want to know the result and disaplay
// our own kind of error.
let parents = path.segments.iter().rev().skip(1);
for path_segment in parents {
if let Some(def_id) = path_segment.res.opt_def_id() {
// use `None` for id to prevent deprecation check
let eval_result = self.tcx.eval_stability_allow_unstable(
def_id,
None,
path.span,
None,
if is_unstable_reexport(self.tcx, id) {
AllowUnstable::Yes
} else {
AllowUnstable::No
},
);
let is_allowed = matches!(eval_result, EvalResult::Allow);
if !is_allowed {
// Calculating message for lint involves calling `self.def_path_str`,
// which will by default invoke the expensive `visible_parent_map` query.
// Skip all that work if the lint is allowed anyway.
if self.tcx.lint_level_at_node(DEPRECATED, id).0
== lint::Level::Allow
{
return;
}
// Show a deprecation message.
let def_path =
with_no_trimmed_paths!(self.tcx.def_path_str(def_id));
let def_kind = self.tcx.def_descr(def_id);
let diag = Deprecated {
sub: None,
kind: def_kind.to_owned(),
path: def_path,
note: Some(deprecation),
since_kind: lint::DeprecatedSinceKind::InEffect,
};
self.tcx.emit_node_span_lint(
DEPRECATED,
id,
method_span.unwrap_or(path.span),
diag,
);
}
}
}
}
}

13
compiler/rustc_resolve/src/late.rs
View file

@ -3960,7 +3960,7 @@ impl<'a, 'ast, 'ra: 'ast, 'tcx> LateResolutionVisitor<'a, 'ast, 'ra, 'tcx> {
match res {
Res::SelfCtor(_) // See #70549.
| Res::Def(
DefKind::Ctor(_, CtorKind::Const) | DefKind::Const | DefKind::ConstParam,
DefKind::Ctor(_, CtorKind::Const) | DefKind::Const | DefKind::AssocConst | DefKind::ConstParam,
_,
) if is_syntactic_ambiguity => {
// Disambiguate in favor of a unit struct/variant or constant pattern.
@ -3969,7 +3969,7 @@ impl<'a, 'ast, 'ra: 'ast, 'tcx> LateResolutionVisitor<'a, 'ast, 'ra, 'tcx> {
}
Some(res)
}
Res::Def(DefKind::Ctor(..) | DefKind::Const | DefKind::Static { .. }, _) => {
Res::Def(DefKind::Ctor(..) | DefKind::Const | DefKind::AssocConst | DefKind::Static { .. }, _) => {
// This is unambiguously a fresh binding, either syntactically
// (e.g., `IDENT @ PAT` or `ref IDENT`) or because `IDENT` resolves
// to something unusable as a pattern (e.g., constructor function),
@ -4005,7 +4005,7 @@ impl<'a, 'ast, 'ra: 'ast, 'tcx> LateResolutionVisitor<'a, 'ast, 'ra, 'tcx> {
);
None
}
Res::Def(DefKind::Fn, _) | Res::Local(..) | Res::Err => {
Res::Def(DefKind::Fn | DefKind::AssocFn, _) | Res::Local(..) | Res::Err => {
// These entities are explicitly allowed to be shadowed by fresh bindings.
None
}
@ -5019,12 +5019,13 @@ struct ItemInfoCollector<'a, 'ra, 'tcx> {
}
impl ItemInfoCollector<'_, '_, '_> {
fn collect_fn_info(&mut self, sig: &FnSig, id: NodeId) {
fn collect_fn_info(&mut self, sig: &FnSig, id: NodeId, attrs: &[Attribute]) {
let sig = DelegationFnSig {
header: sig.header,
param_count: sig.decl.inputs.len(),
has_self: sig.decl.has_self(),
c_variadic: sig.decl.c_variadic(),
target_feature: attrs.iter().any(|attr| attr.has_name(sym::target_feature)),
};
self.r.delegation_fn_sigs.insert(self.r.local_def_id(id), sig);
}
@ -5043,7 +5044,7 @@ impl<'ast> Visitor<'ast> for ItemInfoCollector<'_, '_, '_> {
| ItemKind::Trait(box Trait { ref generics, .. })
| ItemKind::TraitAlias(ref generics, _) => {
if let ItemKind::Fn(box Fn { ref sig, .. }) = &item.kind {
self.collect_fn_info(sig, item.id);
self.collect_fn_info(sig, item.id, &item.attrs);
}
let def_id = self.r.local_def_id(item.id);
@ -5076,7 +5077,7 @@ impl<'ast> Visitor<'ast> for ItemInfoCollector<'_, '_, '_> {
fn visit_assoc_item(&mut self, item: &'ast AssocItem, ctxt: AssocCtxt) {
if let AssocItemKind::Fn(box Fn { ref sig, .. }) = &item.kind {
self.collect_fn_info(sig, item.id);
self.collect_fn_info(sig, item.id, &item.attrs);
}
visit::walk_assoc_item(self, item, ctxt);
}

1
compiler/rustc_resolve/src/macros.rs
View file

@ -1031,6 +1031,7 @@ impl<'ra, 'tcx> Resolver<'ra, 'tcx> {
is_soft,
span,
soft_handler,
stability::UnstableKind::Regular,
);
}
}

7
compiler/rustc_session/src/config.rs
View file

@ -132,13 +132,6 @@ pub enum LtoCli {
}
/// The different settings that the `-C instrument-coverage` flag can have.
///
/// Coverage instrumentation now supports combining `-C instrument-coverage`
/// with compiler and linker optimization (enabled with `-O` or `-C opt-level=1`
/// and higher). Nevertheless, there are many variables, depending on options
/// selected, code structure, and enabled attributes. If errors are encountered,
/// either while compiling or when generating `llvm-cov show` reports, consider
/// lowering the optimization level, or including/excluding `-C link-dead-code`.
#[derive(Clone, Copy, PartialEq, Hash, Debug)]
pub enum InstrumentCoverage {
/// `-C instrument-coverage=no` (or `off`, `false` etc.)

2
compiler/rustc_session/src/options.rs
View file

@ -1638,7 +1638,7 @@ options! {
"extra arguments to append to the linker invocation (space separated)"),
#[rustc_lint_opt_deny_field_access("use `Session::link_dead_code` instead of this field")]
link_dead_code: Option<bool> = (None, parse_opt_bool, [TRACKED],
"keep dead code at link time (useful for code coverage) (default: no)"),
"try to generate and link dead code (default: no)"),
link_self_contained: LinkSelfContained = (LinkSelfContained::default(), parse_link_self_contained, [UNTRACKED],
"control whether to link Rust provided C objects/libraries or rely \
on a C toolchain or linker installed in the system"),

1
compiler/rustc_smir/src/rustc_internal/internal.rs
View file

@ -472,6 +472,7 @@ impl RustcInternal for Abi {
Abi::PtxKernel => rustc_abi::ExternAbi::PtxKernel,
Abi::Msp430Interrupt => rustc_abi::ExternAbi::Msp430Interrupt,
Abi::X86Interrupt => rustc_abi::ExternAbi::X86Interrupt,
Abi::GpuKernel => rustc_abi::ExternAbi::GpuKernel,
Abi::EfiApi => rustc_abi::ExternAbi::EfiApi,
Abi::AvrInterrupt => rustc_abi::ExternAbi::AvrInterrupt,
Abi::AvrNonBlockingInterrupt => rustc_abi::ExternAbi::AvrNonBlockingInterrupt,

1
compiler/rustc_smir/src/rustc_smir/convert/abi.rs
View file

@ -113,6 +113,7 @@ impl<'tcx> Stable<'tcx> for callconv::Conv {
Conv::X86VectorCall => CallConvention::X86VectorCall,
Conv::X86_64SysV => CallConvention::X86_64SysV,
Conv::X86_64Win64 => CallConvention::X86_64Win64,
Conv::GpuKernel => CallConvention::GpuKernel,
Conv::AvrInterrupt => CallConvention::AvrInterrupt,
Conv::AvrNonBlockingInterrupt => CallConvention::AvrNonBlockingInterrupt,
Conv::RiscvInterrupt { .. } => CallConvention::RiscvInterrupt,

1
compiler/rustc_smir/src/rustc_smir/convert/ty.rs
View file

@ -911,6 +911,7 @@ impl<'tcx> Stable<'tcx> for rustc_abi::ExternAbi {
ExternAbi::Win64 { unwind } => Abi::Win64 { unwind },
ExternAbi::SysV64 { unwind } => Abi::SysV64 { unwind },
ExternAbi::PtxKernel => Abi::PtxKernel,
ExternAbi::GpuKernel => Abi::GpuKernel,
ExternAbi::Msp430Interrupt => Abi::Msp430Interrupt,
ExternAbi::X86Interrupt => Abi::X86Interrupt,
ExternAbi::EfiApi => Abi::EfiApi,

160
compiler/rustc_span/src/analyze_source_file.rs
View file

@ -29,6 +29,7 @@ pub(crate) fn analyze_source_file(src: &str) -> (Vec<RelativeBytePos>, Vec<Multi
(lines, multi_byte_chars)
}
#[cfg(bootstrap)]
cfg_match! {
cfg(any(target_arch = "x86", target_arch = "x86_64")) => {
fn analyze_source_file_dispatch(
@ -185,6 +186,165 @@ cfg_match! {
}
}
}
#[cfg(not(bootstrap))]
cfg_match! {
any(target_arch = "x86", target_arch = "x86_64") => {
fn analyze_source_file_dispatch(
src: &str,
lines: &mut Vec<RelativeBytePos>,
multi_byte_chars: &mut Vec<MultiByteChar>,
) {
if is_x86_feature_detected!("sse2") {
unsafe {
analyze_source_file_sse2(src, lines, multi_byte_chars);
}
} else {
analyze_source_file_generic(
src,
src.len(),
RelativeBytePos::from_u32(0),
lines,
multi_byte_chars,
);
}
}
/// Checks 16 byte chunks of text at a time. If the chunk contains
/// something other than printable ASCII characters and newlines, the
/// function falls back to the generic implementation. Otherwise it uses
/// SSE2 intrinsics to quickly find all newlines.
#[target_feature(enable = "sse2")]
unsafe fn analyze_source_file_sse2(
src: &str,
lines: &mut Vec<RelativeBytePos>,
multi_byte_chars: &mut Vec<MultiByteChar>,
) {
#[cfg(target_arch = "x86")]
use std::arch::x86::*;
#[cfg(target_arch = "x86_64")]
use std::arch::x86_64::*;
const CHUNK_SIZE: usize = 16;
let src_bytes = src.as_bytes();
let chunk_count = src.len() / CHUNK_SIZE;
// This variable keeps track of where we should start decoding a
// chunk. If a multi-byte character spans across chunk boundaries,
// we need to skip that part in the next chunk because we already
// handled it.
let mut intra_chunk_offset = 0;
for chunk_index in 0..chunk_count {
let ptr = src_bytes.as_ptr() as *const __m128i;
// We don't know if the pointer is aligned to 16 bytes, so we
// use `loadu`, which supports unaligned loading.
let chunk = unsafe { _mm_loadu_si128(ptr.add(chunk_index)) };
// For character in the chunk, see if its byte value is < 0, which
// indicates that it's part of a UTF-8 char.
let multibyte_test = unsafe { _mm_cmplt_epi8(chunk, _mm_set1_epi8(0)) };
// Create a bit mask from the comparison results.
let multibyte_mask = unsafe { _mm_movemask_epi8(multibyte_test) };
// If the bit mask is all zero, we only have ASCII chars here:
if multibyte_mask == 0 {
assert!(intra_chunk_offset == 0);
// Check if there are any control characters in the chunk. All
// control characters that we can encounter at this point have a
// byte value less than 32 or ...
let control_char_test0 = unsafe { _mm_cmplt_epi8(chunk, _mm_set1_epi8(32)) };
let control_char_mask0 = unsafe { _mm_movemask_epi8(control_char_test0) };
// ... it's the ASCII 'DEL' character with a value of 127.
let control_char_test1 = unsafe { _mm_cmpeq_epi8(chunk, _mm_set1_epi8(127)) };
let control_char_mask1 = unsafe { _mm_movemask_epi8(control_char_test1) };
let control_char_mask = control_char_mask0 | control_char_mask1;
if control_char_mask != 0 {
// Check for newlines in the chunk
let newlines_test = unsafe { _mm_cmpeq_epi8(chunk, _mm_set1_epi8(b'\n' as i8)) };
let newlines_mask = unsafe { _mm_movemask_epi8(newlines_test) };
if control_char_mask == newlines_mask {
// All control characters are newlines, record them
let mut newlines_mask = 0xFFFF0000 | newlines_mask as u32;
let output_offset = RelativeBytePos::from_usize(chunk_index * CHUNK_SIZE + 1);
loop {
let index = newlines_mask.trailing_zeros();
if index >= CHUNK_SIZE as u32 {
// We have arrived at the end of the chunk.
break;
}
lines.push(RelativeBytePos(index) + output_offset);
// Clear the bit, so we can find the next one.
newlines_mask &= (!1) << index;
}
// We are done for this chunk. All control characters were
// newlines and we took care of those.
continue;
} else {
// Some of the control characters are not newlines,
// fall through to the slow path below.
}
} else {
// No control characters, nothing to record for this chunk
continue;
}
}
// The slow path.
// There are control chars in here, fallback to generic decoding.
let scan_start = chunk_index * CHUNK_SIZE + intra_chunk_offset;
intra_chunk_offset = analyze_source_file_generic(
&src[scan_start..],
CHUNK_SIZE - intra_chunk_offset,
RelativeBytePos::from_usize(scan_start),
lines,
multi_byte_chars,
);
}
// There might still be a tail left to analyze
let tail_start = chunk_count * CHUNK_SIZE + intra_chunk_offset;
if tail_start < src.len() {
analyze_source_file_generic(
&src[tail_start..],
src.len() - tail_start,
RelativeBytePos::from_usize(tail_start),
lines,
multi_byte_chars,
);
}
}
}
_ => {
// The target (or compiler version) does not support SSE2 ...
fn analyze_source_file_dispatch(
src: &str,
lines: &mut Vec<RelativeBytePos>,
multi_byte_chars: &mut Vec<MultiByteChar>,
) {
analyze_source_file_generic(
src,
src.len(),
RelativeBytePos::from_u32(0),
lines,
multi_byte_chars,
);
}
}
}
// `scan_len` determines the number of bytes in `src` to scan. Note that the
// function can read past `scan_len` if a multi-byte character start within the
// range but extends past it. The overflow is returned by the function.

1
compiler/rustc_span/src/symbol.rs
View file

@ -379,6 +379,7 @@ symbols! {
abi_avr_interrupt,
abi_c_cmse_nonsecure_call,
abi_efiapi,
abi_gpu_kernel,
abi_msp430_interrupt,
abi_ptx,
abi_riscv_interrupt,

Some files were not shown because too many files have changed in this diff Show more