user0/rust
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Rollup merge of #134531 - GuillaumeGomez:extract-doctests, r=notriddle,aDotInTheVoid

Browse source 
[rustdoc] Add `--extract-doctests` command-line flag

Part of https://github.com/rust-lang/rust/issues/134529.

It was discussed with the Rust-for-Linux project recently that they needed a way to extract doctests so they can modify them and then run them more easily (look for "a way to extract doctests" [here](https://github.com/Rust-for-Linux/linux/issues/2)).

For now, I output most of `ScrapedDoctest` fields in JSON format with `serde_json`. So it outputs the following information:

 * filename
 * line
 * langstr
 * text

cc `@ojeda`
r? `@notriddle`
This commit is contained in:
Matthias Krüger 2025-01-31 12:28:15 +01:00 • committed by GitHub
commit 86595e4c88
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 308 additions and 39 deletions
Download patch file Download diff file Expand all files Collapse all files

64
src/doc/rustdoc/src/unstable-features.md
View file

@ -524,6 +524,8 @@ use `-o -`.
## `-w`/`--output-format`: output format
### json
`--output-format json` emits documentation in the experimental
[JSON format](https://doc.rust-lang.org/nightly/nightly-rustc/rustdoc_json_types/). `--output-format html` has no effect,
and is also accepted on stable toolchains.
@ -542,6 +544,68 @@ It can also be used with `--show-coverage`. Take a look at its
[documentation](#--show-coverage-calculate-the-percentage-of-items-with-documentation) for more
information.
### doctest
`--output-format doctest` emits JSON on stdout which gives you information about doctests in the
provided crate.
Tracking issue: [#134529](https://github.com/rust-lang/rust/issues/134529)
You can use this option like this:
```bash
rustdoc -Zunstable-options --output-format=doctest src/lib.rs
```
For this rust code:
```rust
/// ```
/// let x = 12;
/// ```
pub trait Trait {}
```
The generated output (formatted) will look like this:
```json
{
"format_version": 1,
"doctests": [
{
"file": "foo.rs",
"line": 1,
"doctest_attributes": {
"original": "",
"should_panic": false,
"no_run": false,
"ignore": "None",
"rust": true,
"test_harness": false,
"compile_fail": false,
"standalone_crate": false,
"error_codes": [],
"edition": null,
"added_css_classes": [],
"unknown": []
},
"original_code": "let x = 12;",
"doctest_code": "#![allow(unused)]\nfn main() {\nlet x = 12;\n}",
"name": "foo.rs - Trait (line 1)"
}
]
}
```
* `format_version` gives you the current version of the generated JSON. If we change the output in any way, the number will increase.
* `doctests` contains the list of doctests present in the crate.
* `file` is the file path where the doctest is located.
* `line` is the line where the doctest starts (so where the \`\`\` is located in the current code).
* `doctest_attributes` contains computed information about the attributes used on the doctests. For more information about doctest attributes, take a look [here](write-documentation/documentation-tests.html#attributes).
* `original_code` is the code as written in the source code before rustdoc modifies it.
* `doctest_code` is the code modified by rustdoc that will be run. If there is a fatal syntax error, this field will not be present.
* `name` is the name generated by rustdoc which represents this doctest.
## `--enable-per-target-ignores`: allow `ignore-foo` style filters for doctests
* Tracking issue: [#64245](https://github.com/rust-lang/rust/issues/64245)

60
src/librustdoc/config.rs
View file

@ -33,6 +33,7 @@ pub(crate) enum OutputFormat {
Json,
#[default]
Html,
Doctest,
}
impl OutputFormat {
@ -48,6 +49,7 @@ impl TryFrom<&str> for OutputFormat {
match value {
"json" => Ok(OutputFormat::Json),
"html" => Ok(OutputFormat::Html),
"doctest" => Ok(OutputFormat::Doctest),
_ => Err(format!("unknown output format `{value}`")),
}
}
@ -445,14 +447,42 @@ impl Options {
}
}
let show_coverage = matches.opt_present("show-coverage");
let output_format_s = matches.opt_str("output-format");
let output_format = match output_format_s {
Some(ref s) => match OutputFormat::try_from(s.as_str()) {
Ok(out_fmt) => out_fmt,
Err(e) => dcx.fatal(e),
},
None => OutputFormat::default(),
};
// check for `--output-format=json`
if !matches!(matches.opt_str("output-format").as_deref(), None | Some("html"))
&& !matches.opt_present("show-coverage")
&& !nightly_options::is_unstable_enabled(matches)
{
dcx.fatal(
"the -Z unstable-options flag must be passed to enable --output-format for documentation generation (see https://github.com/rust-lang/rust/issues/76578)",
);
match (
output_format_s.as_ref().map(|_| output_format),
show_coverage,
nightly_options::is_unstable_enabled(matches),
) {
(None | Some(OutputFormat::Json), true, _) => {}
(_, true, _) => {
dcx.fatal(format!(
"`--output-format={}` is not supported for the `--show-coverage` option",
output_format_s.unwrap_or_default(),
));
}
// If `-Zunstable-options` is used, nothing to check after this point.
(_, false, true) => {}
(None | Some(OutputFormat::Html), false, _) => {}
(Some(OutputFormat::Json), false, false) => {
dcx.fatal(
"the -Z unstable-options flag must be passed to enable --output-format for documentation generation (see https://github.com/rust-lang/rust/issues/76578)",
);
}
(Some(OutputFormat::Doctest), false, false) => {
dcx.fatal(
"the -Z unstable-options flag must be passed to enable --output-format for documentation generation (see https://github.com/rust-lang/rust/issues/134529)",
);
}
}
let to_check = matches.opt_strs("check-theme");
@ -704,8 +734,6 @@ impl Options {
})
.collect();
let show_coverage = matches.opt_present("show-coverage");
let crate_types = match parse_crate_types_from_list(matches.opt_strs("crate-type")) {
Ok(types) => types,
Err(e) => {
@ -713,20 +741,6 @@ impl Options {
}
};
let output_format = match matches.opt_str("output-format") {
Some(s) => match OutputFormat::try_from(s.as_str()) {
Ok(out_fmt) => {
if !out_fmt.is_json() && show_coverage {
dcx.fatal(
"html output format isn't supported for the --show-coverage option",
);
}
out_fmt
}
Err(e) => dcx.fatal(e),
},
None => OutputFormat::default(),
};
let crate_name = matches.opt_str("crate-name");
let bin_crate = crate_types.contains(&CrateType::Executable);
let proc_macro_crate = crate_types.contains(&CrateType::ProcMacro);

51
src/librustdoc/doctest.rs
View file

@ -1,3 +1,4 @@
mod extracted;
mod make;
mod markdown;
mod runner;
@ -30,7 +31,7 @@ use tempfile::{Builder as TempFileBuilder, TempDir};
use tracing::debug;
use self::rust::HirCollector;
use crate::config::Options as RustdocOptions;
use crate::config::{Options as RustdocOptions, OutputFormat};
use crate::html::markdown::{ErrorCodes, Ignore, LangString, MdRelLine};
use crate::lint::init_lints;
@ -209,15 +210,8 @@ pub(crate) fn run(dcx: DiagCtxtHandle<'_>, input: Input, options: RustdocOptions
let args_path = temp_dir.path().join("rustdoc-cfgs");
crate::wrap_return(dcx, generate_args_file(&args_path, &options));
let CreateRunnableDocTests {
standalone_tests,
mergeable_tests,
rustdoc_options,
opts,
unused_extern_reports,
compiling_test_count,
..
} = interface::run_compiler(config, |compiler| {
let extract_doctests = options.output_format == OutputFormat::Doctest;
let result = interface::run_compiler(config, |compiler| {
let krate = rustc_interface::passes::parse(&compiler.sess);
let collector = rustc_interface::create_and_enter_global_ctxt(compiler, krate, |tcx| {
@ -226,22 +220,53 @@ pub(crate) fn run(dcx: DiagCtxtHandle<'_>, input: Input, options: RustdocOptions
let opts = scrape_test_config(crate_name, crate_attrs, args_path);
let enable_per_target_ignores = options.enable_per_target_ignores;
let mut collector = CreateRunnableDocTests::new(options, opts);
let hir_collector = HirCollector::new(
ErrorCodes::from(compiler.sess.opts.unstable_features.is_nightly_build()),
enable_per_target_ignores,
tcx,
);
let tests = hir_collector.collect_crate();
tests.into_iter().for_each(|t| collector.add_test(t));
if extract_doctests {
let mut collector = extracted::ExtractedDocTests::new();
tests.into_iter().for_each(|t| collector.add_test(t, &opts, &options));
collector
let stdout = std::io::stdout();
let mut stdout = stdout.lock();
if let Err(error) = serde_json::ser::to_writer(&mut stdout, &collector) {
eprintln!();
Err(format!("Failed to generate JSON output for doctests: {error:?}"))
} else {
Ok(None)
}
} else {
let mut collector = CreateRunnableDocTests::new(options, opts);
tests.into_iter().for_each(|t| collector.add_test(t));
Ok(Some(collector))
}
});
compiler.sess.dcx().abort_if_errors();
collector
});
let CreateRunnableDocTests {
standalone_tests,
mergeable_tests,
rustdoc_options,
opts,
unused_extern_reports,
compiling_test_count,
..
} = match result {
Ok(Some(collector)) => collector,
Ok(None) => return,
Err(error) => {
eprintln!("{error}");
std::process::exit(1);
}
};
run_tests(opts, &rustdoc_options, &unused_extern_reports, standalone_tests, mergeable_tests);
let compiling_test_count = compiling_test_count.load(Ordering::SeqCst);

141
src/librustdoc/doctest/extracted.rs Normal file
View file

@ -0,0 +1,141 @@
//! Rustdoc's doctest extraction.
//!
//! This module contains the logic to extract doctests and output a JSON containing this
//! information.
use serde::Serialize;
use super::{DocTestBuilder, ScrapedDocTest};
use crate::config::Options as RustdocOptions;
use crate::html::markdown;
/// The version of JSON output that this code generates.
///
/// This integer is incremented with every breaking change to the API,
/// and is returned along with the JSON blob into the `format_version` root field.
/// Consuming code should assert that this value matches the format version(s) that it supports.
const FORMAT_VERSION: u32 = 1;
#[derive(Serialize)]
pub(crate) struct ExtractedDocTests {
format_version: u32,
doctests: Vec<ExtractedDocTest>,
}
impl ExtractedDocTests {
pub(crate) fn new() -> Self {
Self { format_version: FORMAT_VERSION, doctests: Vec::new() }
}
pub(crate) fn add_test(
&mut self,
scraped_test: ScrapedDocTest,
opts: &super::GlobalTestOptions,
options: &RustdocOptions,
) {
let edition = scraped_test.edition(&options);
let ScrapedDocTest { filename, line, langstr, text, name } = scraped_test;
let doctest = DocTestBuilder::new(
&text,
Some(&opts.crate_name),
edition,
false,
None,
Some(&langstr),
);
let (full_test_code, size) = doctest.generate_unique_doctest(
&text,
langstr.test_harness,
&opts,
Some(&opts.crate_name),
);
self.doctests.push(ExtractedDocTest {
file: filename.prefer_remapped_unconditionaly().to_string(),
line,
doctest_attributes: langstr.into(),
doctest_code: if size != 0 { Some(full_test_code) } else { None },
original_code: text,
name,
});
}
}
#[derive(Serialize)]
pub(crate) struct ExtractedDocTest {
file: String,
line: usize,
doctest_attributes: LangString,
original_code: String,
/// `None` if the code syntax is invalid.
doctest_code: Option<String>,
name: String,
}
#[derive(Serialize)]
pub(crate) enum Ignore {
All,
None,
Some(Vec<String>),
}
impl From<markdown::Ignore> for Ignore {
fn from(original: markdown::Ignore) -> Self {
match original {
markdown::Ignore::All => Self::All,
markdown::Ignore::None => Self::None,
markdown::Ignore::Some(values) => Self::Some(values),
}
}
}
#[derive(Serialize)]
struct LangString {
pub(crate) original: String,
pub(crate) should_panic: bool,
pub(crate) no_run: bool,
pub(crate) ignore: Ignore,
pub(crate) rust: bool,
pub(crate) test_harness: bool,
pub(crate) compile_fail: bool,
pub(crate) standalone_crate: bool,
pub(crate) error_codes: Vec<String>,
pub(crate) edition: Option<String>,
pub(crate) added_css_classes: Vec<String>,
pub(crate) unknown: Vec<String>,
}
impl From<markdown::LangString> for LangString {
fn from(original: markdown::LangString) -> Self {
let markdown::LangString {
original,
should_panic,
no_run,
ignore,
rust,
test_harness,
compile_fail,
standalone_crate,
error_codes,
edition,
added_classes,
unknown,
} = original;
Self {
original,
should_panic,
no_run,
ignore: ignore.into(),
rust,
test_harness,
compile_fail,
standalone_crate,
error_codes,
edition: edition.map(|edition| edition.to_string()),
added_css_classes: added_classes,
unknown,
}
}
}

10
src/librustdoc/lib.rs
View file

@ -814,7 +814,12 @@ fn main_args(early_dcx: &mut EarlyDiagCtxt, at_args: &[String]) {
}
};
match (options.should_test, config::markdown_input(&input)) {
let output_format = options.output_format;
match (
options.should_test || output_format == config::OutputFormat::Doctest,
config::markdown_input(&input),
) {
(true, Some(_)) => return wrap_return(dcx, doctest::test_markdown(&input, options)),
(true, None) => return doctest::run(dcx, input, options),
(false, Some(md_input)) => {
@ -849,7 +854,6 @@ fn main_args(early_dcx: &mut EarlyDiagCtxt, at_args: &[String]) {
// plug/cleaning passes.
let crate_version = options.crate_version.clone();
let output_format = options.output_format;
let scrape_examples_options = options.scrape_examples_options.clone();
let bin_crate = options.bin_crate;
@ -899,6 +903,8 @@ fn main_args(early_dcx: &mut EarlyDiagCtxt, at_args: &[String]) {
config::OutputFormat::Json => sess.time("render_json", || {
run_renderer::<json::JsonRenderer<'_>>(krate, render_opts, cache, tcx)
}),
// Already handled above with doctest runners.
config::OutputFormat::Doctest => unreachable!(),
}
})
})

2
tests/rustdoc-ui/coverage/html.stderr
View file

@ -1,2 +1,2 @@
error: html output format isn't supported for the --show-coverage option
error: `--output-format=html` is not supported for the `--show-coverage` option

1
tests/rustdoc-ui/doctest-output.rs Normal file
View file

@ -0,0 +1 @@
//@ compile-flags:-Z unstable-options --show-coverage --output-format=doctest

2
tests/rustdoc-ui/doctest-output.stderr Normal file
View file

@ -0,0 +1,2 @@
error: `--output-format=doctest` is not supported for the `--show-coverage` option

15
tests/rustdoc-ui/extract-doctests.rs Normal file
View file

@ -0,0 +1,15 @@
// Test to ensure that it generates expected output for `--output-format=doctest` command-line
// flag.
//@ compile-flags:-Z unstable-options --output-format=doctest
//@ normalize-stdout: "tests/rustdoc-ui" -> "$$DIR"
//@ check-pass
//! ```ignore (checking attributes)
//! let x = 12;
//! let y = 14;
//! ```
//!
//! ```edition2018,compile_fail
//! let
//! ```

1
tests/rustdoc-ui/extract-doctests.stdout Normal file
View file

@ -0,0 +1 @@
{"format_version":1,"doctests":[{"file":"$DIR/extract-doctests.rs","line":8,"doctest_attributes":{"original":"ignore (checking attributes)","should_panic":false,"no_run":false,"ignore":"All","rust":true,"test_harness":false,"compile_fail":false,"standalone_crate":false,"error_codes":[],"edition":null,"added_css_classes":[],"unknown":[]},"original_code":"let x = 12;\nlet y = 14;","doctest_code":"#![allow(unused)]\nfn main() {\nlet x = 12;\nlet y = 14;\n}","name":"$DIR/extract-doctests.rs - (line 8)"},{"file":"$DIR/extract-doctests.rs","line":13,"doctest_attributes":{"original":"edition2018,compile_fail","should_panic":false,"no_run":true,"ignore":"None","rust":true,"test_harness":false,"compile_fail":true,"standalone_crate":false,"error_codes":[],"edition":"2018","added_css_classes":[],"unknown":[]},"original_code":"let","doctest_code":"#![allow(unused)]\nfn main() {\nlet\n}","name":"$DIR/extract-doctests.rs - (line 13)"}]}