Auto merge of #32133 - alexcrichton:linkchecker, r=brson

Add a link validator to rustbuild

This commit was originally targeted at just adding a link checking script to the rustbuild system. This ended up snowballing a bit to extend rustbuild to be amenable to various tools we have as part of the build system in general.

There's a new `src/tools` directory which has a number of scripts/programs that are purely intended to be used as part of the build system and CI of this repository. This is currently inhabited by rustbook, the error index generator, and a new linkchecker script added as part of this PR. I suspect that more tools like compiletest, tidy scripts, snapshot scripts, etc will migrate their way into this directory over time.

The commit which adds the error index generator shows the steps necessary to add new tools to the build system, namely:

1. New steps are defined for building the tool and running the tool
2. The dependencies are configured
3. The steps are implemented

In terms of the link checker, these commits do a few things:

* A new `src/tools/linkchecker` script is added. This will read an entire documentation tree looking for broken relative links (HTTP links aren't followed yet).
* A large number of broken links throughout the documentation were fixed. Many of these were just broken when viewed from core as opposed to std, but were easily fixed.
* A few rustdoc bugs here and there were fixed

src/doc/book/choosing-your-guarantees.md

@@ -204,7 +204,7 @@ borrow checker. Generally we know that such mutations won't happen in a nested f
 to check.
 
 For large, complicated programs, it becomes useful to put some things in `RefCell`s to make things
-simpler. For example, a lot of the maps in [the `ctxt` struct][ctxt] in the Rust compiler internals
+simpler. For example, a lot of the maps in the `ctxt` struct in the Rust compiler internals
 are inside this wrapper. These are only modified once (during creation, which is not right after
 initialization) or a couple of times in well-separated places. However, since this struct is
 pervasively used everywhere, juggling mutable and immutable pointers would be hard (perhaps
@@ -235,7 +235,6 @@ At runtime each borrow causes a modification/check of the refcount.
 [cell-mod]: ../std/cell/
 [cell]: ../std/cell/struct.Cell.html
 [refcell]: ../std/cell/struct.RefCell.html
-[ctxt]: ../rustc/middle/ty/struct.ctxt.html
 
 # Synchronous types
 

src/doc/book/compiler-plugins.md

@@ -8,12 +8,12 @@ extend the compiler's behavior with new syntax extensions, lint checks, etc.
 A plugin is a dynamic library crate with a designated *registrar* function that
 registers extensions with `rustc`. Other crates can load these extensions using
 the crate attribute `#![plugin(...)]`.  See the
-[`rustc_plugin`](../rustc_plugin/index.html) documentation for more about the
+`rustc_plugin` documentation for more about the
 mechanics of defining and loading a plugin.
 
 If present, arguments passed as `#![plugin(foo(... args ...))]` are not
 interpreted by rustc itself.  They are provided to the plugin through the
-`Registry`'s [`args` method](../rustc_plugin/registry/struct.Registry.html#method.args).
+`Registry`'s `args` method.
 
 In the vast majority of cases, a plugin should *only* be used through
 `#![plugin]` and not through an `extern crate` item.  Linking a plugin would
@@ -30,7 +30,7 @@ of a library.
 Plugins can extend Rust's syntax in various ways. One kind of syntax extension
 is the procedural macro. These are invoked the same way as [ordinary
 macros](macros.html), but the expansion is performed by arbitrary Rust
-code that manipulates [syntax trees](../syntax/ast/index.html) at
+code that manipulates syntax trees at
 compile time.
 
 Let's write a plugin
@@ -120,11 +120,8 @@ The advantages over a simple `fn(&str) -> u32` are:
 
 In addition to procedural macros, you can define new
 [`derive`](../reference.html#derive)-like attributes and other kinds of
-extensions.  See
-[`Registry::register_syntax_extension`](../rustc_plugin/registry/struct.Registry.html#method.register_syntax_extension)
-and the [`SyntaxExtension`
-enum](https://doc.rust-lang.org/syntax/ext/base/enum.SyntaxExtension.html).  For
-a more involved macro example, see
+extensions.  See `Registry::register_syntax_extension` and the `SyntaxExtension`
+enum.  For a more involved macro example, see
 [`regex_macros`](https://github.com/rust-lang/regex/blob/master/regex_macros/src/lib.rs).
 
 
@@ -132,7 +129,7 @@ a more involved macro example, see
 
 Some of the [macro debugging tips](macros.html#debugging-macro-code) are applicable.
 
-You can use [`syntax::parse`](../syntax/parse/index.html) to turn token trees into
+You can use `syntax::parse` to turn token trees into
 higher-level syntax elements like expressions:
 
 ```ignore
@@ -148,30 +145,21 @@ Looking through [`libsyntax` parser
 code](https://github.com/rust-lang/rust/blob/master/src/libsyntax/parse/parser.rs)
 will give you a feel for how the parsing infrastructure works.
 
-Keep the [`Span`s](../syntax/codemap/struct.Span.html) of
-everything you parse, for better error reporting. You can wrap
-[`Spanned`](../syntax/codemap/struct.Spanned.html) around
-your custom data structures.
+Keep the `Span`s of everything you parse, for better error reporting. You can
+wrap `Spanned` around your custom data structures.
 
-Calling
-[`ExtCtxt::span_fatal`](../syntax/ext/base/struct.ExtCtxt.html#method.span_fatal)
-will immediately abort compilation. It's better to instead call
-[`ExtCtxt::span_err`](../syntax/ext/base/struct.ExtCtxt.html#method.span_err)
-and return
-[`DummyResult`](../syntax/ext/base/struct.DummyResult.html),
-so that the compiler can continue and find further errors.
+Calling `ExtCtxt::span_fatal` will immediately abort compilation. It's better to
+instead call `ExtCtxt::span_err` and return `DummyResult` so that the compiler
+can continue and find further errors.
 
-To print syntax fragments for debugging, you can use
-[`span_note`](../syntax/ext/base/struct.ExtCtxt.html#method.span_note) together
-with
-[`syntax::print::pprust::*_to_string`](https://doc.rust-lang.org/syntax/print/pprust/index.html#functions).
+To print syntax fragments for debugging, you can use `span_note` together with
+`syntax::print::pprust::*_to_string`.
 
-The example above produced an integer literal using
-[`AstBuilder::expr_usize`](../syntax/ext/build/trait.AstBuilder.html#tymethod.expr_usize).
+The example above produced an integer literal using `AstBuilder::expr_usize`.
 As an alternative to the `AstBuilder` trait, `libsyntax` provides a set of
-[quasiquote macros](../syntax/ext/quote/index.html).  They are undocumented and
-very rough around the edges.  However, the implementation may be a good
-starting point for an improved quasiquote as an ordinary plugin library.
+quasiquote macros. They are undocumented and very rough around the edges.
+However, the implementation may be a good starting point for an improved
+quasiquote as an ordinary plugin library.
 
 
 # Lint plugins
@@ -239,12 +227,11 @@ foo.rs:4 fn lintme() { }
 
 The components of a lint plugin are:
 
-* one or more `declare_lint!` invocations, which define static
-  [`Lint`](../rustc/lint/struct.Lint.html) structs;
+* one or more `declare_lint!` invocations, which define static `Lint` structs;
 
 * a struct holding any state needed by the lint pass (here, none);
 
-* a [`LintPass`](../rustc/lint/trait.LintPass.html)
+* a `LintPass`
   implementation defining how to check each syntax element. A single
   `LintPass` may call `span_lint` for several different `Lint`s, but should
   register them all through the `get_lints` method.

src/doc/guide-plugins.md

@@ -1,4 +1,4 @@
 % The (old) Rust Compiler Plugins Guide
 
 This content has moved into
-[the Rust Programming Language book](book/plugins.html).
+[the Rust Programming Language book](book/compiler-plugins.html).

src/doc/style/README.md

@@ -53,7 +53,7 @@ This document is broken into four parts:
   cross-cutting topic, starting with
   [Ownership and resources](ownership/README.md).
 
-* **[APIs for a changing Rust](changing/README.md)**
+* **APIs for a changing Rust**
   discusses the forward-compatibility hazards, especially those that interact
   with the pre-1.0 library stabilization process.
 

src/doc/style/features/functions-and-methods/input.md

@@ -76,7 +76,7 @@ needs to make about its arguments.
 On the other hand, generics can make it more difficult to read and understand a
 function's signature. Aim for "natural" parameter types that a neither overly
 concrete nor overly abstract. See the discussion on
-[traits](../../traits/README.md) for more guidance.
+[traits](../traits/README.md) for more guidance.
 
 
 #### Minimizing ownership assumptions:

src/doc/style/style/naming/README.md

@@ -101,7 +101,7 @@ The convention for a field `foo: T` is:
   here may take `&T` or some other type, depending on the context.)
 
 Note that this convention is about getters/setters on ordinary data types, *not*
-on [builder objects](../ownership/builders.html).
+on [builder objects](../../ownership/builders.html).
 
 ### Escape hatches [FIXME]