Rollup merge of #79570 - alexcrichton:split-debuginfo, r=bjorn3

rustc: Stabilize `-Zrun-dsymutil` as `-Csplit-debuginfo`

This commit adds a new stable codegen option to rustc,
`-Csplit-debuginfo`. The old `-Zrun-dsymutil` flag is deleted and now
subsumed by this stable flag. Additionally `-Zsplit-dwarf` is also
subsumed by this flag but still requires `-Zunstable-options` to
actually activate. The `-Csplit-debuginfo` flag takes one of
three values:

* `off` - This indicates that split-debuginfo from the final artifact is
  not desired. This is not supported on Windows and is the default on
  Unix platforms except macOS. On macOS this means that `dsymutil` is
  not executed.

* `packed` - This means that debuginfo is desired in one location
  separate from the main executable. This is the default on Windows
  (`*.pdb`) and macOS (`*.dSYM`). On other Unix platforms this subsumes
  `-Zsplit-dwarf=single` and produces a `*.dwp` file.

* `unpacked` - This means that debuginfo will be roughly equivalent to
  object files, meaning that it's throughout the build directory
  rather than in one location (often the fastest for local development).
  This is not the default on any platform and is not supported on Windows.

Each target can indicate its own default preference for how debuginfo is
handled. Almost all platforms default to `off` except for Windows and
macOS which default to `packed` for historical reasons.

Some equivalencies for previous unstable flags with the new flags are:

* `-Zrun-dsymutil=yes` -> `-Csplit-debuginfo=packed`
* `-Zrun-dsymutil=no` -> `-Csplit-debuginfo=unpacked`
* `-Zsplit-dwarf=single` -> `-Csplit-debuginfo=packed`
* `-Zsplit-dwarf=split` -> `-Csplit-debuginfo=unpacked`

Note that `-Csplit-debuginfo` still requires `-Zunstable-options` for
non-macOS platforms since split-dwarf support was *just* implemented in
rustc.

There's some more rationale listed on #79361, but the main gist of the
motivation for this commit is that `dsymutil` can take quite a long time
to execute in debug builds and provides little benefit. This means that
incremental compile times appear that much worse on macOS because the
compiler is constantly running `dsymutil` over every single binary it
produces during `cargo build` (even build scripts!). Ideally rustc would
switch to not running `dsymutil` by default, but that's a problem left
to get tackled another day.

Closes #79361

src/bootstrap/builder.rs

@@ -1139,10 +1139,18 @@ impl<'a> Builder<'a> {
         // itself, we skip it by default since we know it's safe to do so in that case.
         // See https://github.com/rust-lang/rust/issues/79361 for more info on this flag.
         if target.contains("apple") {
-            if self.config.rust_run_dsymutil {
-                rustflags.arg("-Zrun-dsymutil=yes");
+            if stage == 0 {
+                if self.config.rust_run_dsymutil {
+                    rustflags.arg("-Zrun-dsymutil=yes");
+                } else {
+                    rustflags.arg("-Zrun-dsymutil=no");
+                }
             } else {
-                rustflags.arg("-Zrun-dsymutil=no");
+                if self.config.rust_run_dsymutil {
+                    rustflags.arg("-Csplit-debuginfo=packed");
+                } else {
+                    rustflags.arg("-Csplit-debuginfo=unpacked");
+                }
             }
         }
 

src/doc/rustc/src/codegen-options/index.md

@@ -492,6 +492,34 @@ point instructions in software. It takes one of the following values:
 * `y`, `yes`, `on`, or no value: use soft floats.
 * `n`, `no`, or `off`: use hardware floats (the default).
 
+## split-debuginfo
+
+This option controls the emission of "split debuginfo" for debug information
+that `rustc` generates. The default behavior of this option is
+platform-specific, and not all possible values for this option work on all
+platform. Possible values are:
+
+* `off` - This is the default for platforms with ELF binaries and windows-gnu
+  (not Windows MSVC and not macOS). This typically means that dwarf debug
+  information can be found in the final artifact in sections of the executable.
+  This option is not supported on Windows MSVC. On macOS this options prevents
+  the final execution of `dsymutil` to generate debuginfo.
+
+* `packed` - This is the default for Windows MSVC and macOS platforms. The term
+  "packed" here means that all the debug information is packed into a separate
+  file from the main executable. On Windows MSVC this is a `*.pdb` file, on
+  macOS this is a `*.dSYM` folder, and on other platforms this is a `*.dwp`
+  files.
+
+* `unpacked` - This means that debug information will be found in separate
+  files for each compilation unit (object file). This is not supported on
+  Windows MSVC. On macOS this means the original object files will contain
+  debug information. On other Unix platforms this means that `*.dwo` files will
+  contain debug information.
+
+Note that `packed` and `unpacked` gated behind `-Zunstable-options` on
+non-macOS platforms at this time.
+
 ## target-cpu
 
 This instructs `rustc` to generate code specifically for a particular processor.
@@ -499,7 +527,7 @@ This instructs `rustc` to generate code specifically for a particular processor.
 You can run `rustc --print target-cpus` to see the valid options to pass
 here. Each target has a default base CPU. Special values include:
 
-* `native` can be passed to use the processor of the host machine. 
+* `native` can be passed to use the processor of the host machine.
 * `generic` refers to an LLVM target with minimal features but modern tuning.
 
 ## target-feature

src/test/run-make-fulldeps/split-debuginfo/Makefile

@@ -0,0 +1,59 @@
+-include ../tools.mk
+
+all: off packed unpacked
+
+ifeq ($(UNAME),Darwin)
+# If disabled, don't run dsymutil
+off:
+	rm -rf $(TMPDIR)/*.dSYM
+	$(RUSTC) foo.rs -g -C split-debuginfo=off
+	[ ! -d $(TMPDIR)/foo.dSYM ]
+
+# Packed by default, but only if debuginfo is requested
+packed:
+	rm -rf $(TMPDIR)/*.dSYM
+	$(RUSTC) foo.rs
+	[ ! -d $(TMPDIR)/foo.dSYM ]
+	rm -rf $(TMPDIR)/*.dSYM
+	$(RUSTC) foo.rs -g
+	[ -d $(TMPDIR)/foo.dSYM ]
+	rm -rf $(TMPDIR)/*.dSYM
+	$(RUSTC) foo.rs -g -C split-debuginfo=packed
+	[ -d $(TMPDIR)/foo.dSYM ]
+	rm -rf $(TMPDIR)/*.dSYM
+
+# Object files are preserved with unpacked and `dsymutil` isn't run
+unpacked:
+	$(RUSTC) foo.rs -g -C split-debuginfo=unpacked
+	ls $(TMPDIR)/*.o
+	[ ! -d $(TMPDIR)/foo.dSYM ]
+else
+ifdef IS_WINDOWS
+# Windows only supports =off
+off:
+packed:
+unpacked:
+else
+# If disabled, don't run dsymutil
+off:
+	$(RUSTC) foo.rs -g -C split-debuginfo=off -Z unstable-options
+	[ ! -f $(TMPDIR)/*.dwp ]
+	[ ! -f $(TMPDIR)/*.dwo ]
+
+	$(RUSTC) foo.rs -g
+	[ ! -f $(TMPDIR)/*.dwp ]
+	[ ! -f $(TMPDIR)/*.dwo ]
+
+packed:
+	$(RUSTC) foo.rs -g -C split-debuginfo=packed -Z unstable-options
+	ls $(TMPDIR)/*.dwp
+	ls $(TMPDIR)/*.dwo && exit 1 || exit 0
+	rm -rf $(TMPDIR)/*.dwp
+
+unpacked:
+	$(RUSTC) foo.rs -g -C split-debuginfo=unpacked -Z unstable-options
+	ls $(TMPDIR)/*.dwp && exit 1 || exit 0
+	ls $(TMPDIR)/*.dwo
+	rm -rf $(TMPDIR)/*.dwo
+endif
+endif

src/test/run-make-fulldeps/split-debuginfo/foo.rs

@@ -0,0 +1 @@
+fn main() {}

src/test/run-make-fulldeps/split-dwarf/Makefile

@@ -3,6 +3,6 @@
 # only-linux
 
 all:
-	$(RUSTC) -Z split-dwarf=split foo.rs
+	$(RUSTC) -Z unstable-options -C split-debuginfo=packed foo.rs -g
 	rm $(TMPDIR)/foo.dwp
 	rm $(TMPDIR)/$(call BIN,foo)

src/test/ui/backtrace-apple-no-dsymutil.rs

@@ -0,0 +1,30 @@
+// run-pass
+
+// compile-flags:-g -Csplit-debuginfo=unpacked
+// only-macos
+
+#![feature(backtrace)]
+
+use std::process::Command;
+use std::str;
+
+#[inline(never)]
+fn main() {
+    let args: Vec<String> = std::env::args().collect();
+    if args.len() >= 2 {
+        println!("{}", std::backtrace::Backtrace::force_capture());
+        return;
+    }
+    let out = Command::new(&args[0]).env("RUST_BACKTRACE", "1").arg("foo").output().unwrap();
+    let output = format!(
+        "{}\n{}",
+        str::from_utf8(&out.stdout).unwrap(),
+        str::from_utf8(&out.stderr).unwrap(),
+    );
+    if out.status.success() && output.contains(file!()) {
+        return;
+    }
+    println!("status: {}", out.status);
+    println!("child output:\n\t{}", output.replace("\n", "\n\t"));
+    panic!("failed to find {:?} in output", file!());
+}

src/tools/compiletest/src/runtest.rs

@@ -2015,10 +2015,10 @@ impl<'test> TestCx<'test> {
                 rustc.args(&["-Zchalk"]);
             }
             Some(CompareMode::SplitDwarf) => {
-                rustc.args(&["-Zsplit-dwarf=split"]);
+                rustc.args(&["-Csplit-debuginfo=unpacked", "-Zunstable-options"]);
             }
             Some(CompareMode::SplitDwarfSingle) => {
-                rustc.args(&["-Zsplit-dwarf=single"]);
+                rustc.args(&["-Csplit-debuginfo=packed", "-Zunstable-options"]);
             }
             None => {}
         }

src/tools/tidy/src/ui_tests.rs

@@ -7,7 +7,7 @@ use std::path::Path;
 
 const ENTRY_LIMIT: usize = 1000;
 // FIXME: The following limits should be reduced eventually.
-const ROOT_ENTRY_LIMIT: usize = 1458;
+const ROOT_ENTRY_LIMIT: usize = 1459;
 const ISSUES_ENTRY_LIMIT: usize = 2669;
 
 fn check_entries(path: &Path, bad: &mut bool) {